unannotated document pages dropped in exported pdf

[wuschel-brompf]

RM on 2.15.1.1189
docker tag 0.0.13

Example

• Import a 10 page document
• export will show a 10 page document (as expected)
• annotate pages 5 and 8, without making ANY changes to any other page
• export will show a 2 paged document:
  • source doc page 1/10 with overlayed notes of page 5 and
  • source doc page 2/10 with overlayed notes of page 8.

Problem Description

When annotating n pages in a multipage pdf, exported pdf has the FIRST n pages of the source pdf as background (irrespective of which pages were annotated) and overlays them with the annotations made.

Once a page is annotated, and even if the annotation is deleted immediately, it will be counted as an annotated page.

Suggested (dirty) fix

Add and remove an annotation to every page automatically.

related issue

When there are more annotated pages (eg by adding note pages) than pages in the source pdf, "invalid page number (page count too short)" error occurs, see #185

[ddvk]

problably to do with the redirection, or something

[wuschel-brompf]

what do you mean - is this a general issue or an error in my setup?

[ddvk]

I was not clear enough, at some point page redirection was introduced in note files. It got implemented in rmapi, but I haven't tested whether it got updated here.

[damienchallet]

(edit): Similar problem here, the annotations do not skip pages without annotations, which would make annotations be put in page 5 and 6 in the example above. The exported pdf has all the pages, though. Sending via email works perfectly well.

[damienchallet]

Update, I just experienced the same problem for a long document (160 pages), with 4 random pages annotated. In rmfakecloud, the document is reported as having 4 pages: only the first 4 pages of the original document appear, with the annotations of the 4 pages (so missing pages and annotations on the wrong page).

If this issue is not being fixed, I would like either to help bissect it, or to be pointed to a version that works.

(rM1, firmware 2.8.0.98)

[Eeems]

You could be encountering a known issue with the OS that causes data loss for OS versions older than 3.3, which would not be something rmfakecloud can fix.
https://support.remarkable.com/s/article/Tech-bulletins

[damienchallet]

That is a possibility, but 10.11.99.1 works perfectly, and sending documents via email also works (and this is how I deal with the rmfakecloud issue at the moment).

…

On Fri, 13 Sept 2024 at 08:45, Nathaniel van Diepen < ***@***.***> wrote: You could be encountering a known issue with the OS that causes data loss for OS versions older than 3.3, which would not be something rmfakecloud can fix. https://support.remarkable.com/s/article/Tech-bulletins — Reply to this email directly, view it on GitHub <#228 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADRKPVRSKVQWTH36MMGJEDZWKCYFAVCNFSM6AAAAABNMK5D6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBYGE2TINZYGU> . You are receiving this because you commented.Message ID: ***@***.***>

[Eeems]

That is a possibility, but 10.11.99.1 works perfectly, and sending documents via email also works (and this is how I deal with the rmfakecloud issue at the moment).

Emails are generated on the device and then sent through the cloud interface, and 10.11.99.1 is fully local. As both are local, they would be unrelated to a dataloss caused by synchronising with the cloud, which is what you are describing.

[giovi321]

Still reproducible on current master, and the original description is exactly right.

Measured on a 28 page document annotated on 14 of them. The export has 14 pages, and matching each exported page against the source shows the backgrounds are source pages 1 through 14 in order, regardless of which pages carry ink. The ink is the annotated pages in their own order, laid over them. On this document, the ink from page 8 is drawn over the background of page 3.

So it is the pairing rather than redirection: the index used to fetch the background is the position among annotated pages, not the page the ink belongs to. With every page annotated the two agree, which is why this only shows up on partly annotated documents.

@damienchallet's later comment is the other half of it: the unannotated pages are dropped rather than kept, so nothing lines up afterwards either.

I have this fixed on a branch, though as part of a larger change to the export path rather than something I can send here on its own. The pages a document declares are collected in the order its content file gives, each is rendered against its own background, and a page with no ink is kept rather than skipped so the numbering holds. On the library I tested, all 44 annotated pdfs then export with the full page count and the ink on the right pages.

Writing the cause down here mainly because this has been open a while. It is also worth knowing when reviewing #483, which fixes landscape documents rendering sideways: a landscape export still looks wrong afterwards for this separate reason, so that one has to be judged on measured ink positions rather than on how the page looks.

[giovi321]

Correcting myself on the mechanism, and I have opened #484 for it.

Above I wrote that "the index used to fetch the background is the position among annotated pages". That was an inference from the behaviour rather than something I had read, and I also said this was not something I could send here on its own. Both were wrong.

It is ArchiveFromHashDoc in internal/storage/models/archive.go, which walks the pages the content file declares and appends an entry only where a .rm blob exists. Pages nobody drew on are never added, so the list ends up indexed by annotated page, and pdf.go then takes each page's background from that index. @ddvk's first reply guessed redirection, which turned out not to be it: on the document I measured, the redirection map is present and is the identity.

Adding an entry for every declared page, with no data where nobody drew, puts each page back over its own background and keeps the untouched pages. On that 28 page document, annotated on 14: it exported 14 pages with backgrounds 1 to 14 in order, and now exports 28, each over its own source page, with none mismatched.

#484 is that change, one file. It is independent of #483, which is about where ink sits on a page rather than which page it lands on. A landscape document needs both.