Drop the pages that carry nothing - #23
Merged
Merged
Conversation
A duplex stack run through a single-sided feeder comes back with a blank behind every one-sided sheet, and a scanner set to "both sides" makes one for every sheet that only had one. They are the pages nobody wants and everybody has — and with Interleave in ops v0.9.0 putting the two halves of such a scan back together, this is the other half of that job. The threshold was measured rather than decreed. Over 395 pages of government forms and library scans, the ink on a page runs to a median of 4.7% and falls away sharply below one part in a thousand: eleven pages are under it, two are under a twentieth of it, and the pages between are covers carrying a rule and nothing else. A page with a line of text is around three parts in a thousand, so this keeps one and drops a page number alone — which is what a blank back with a footer is. Picking that by eye would have been wrong twice over. The first measurement used a threshold of 200 rather than 128 and reported a median of 18% ink and not one blank page in 395: at that setting the off-white background of a scan counts as ink. Which threshold is used decides the answer, so the answer has to say which. A page that cannot be DRAWN is not counted blank. Not being able to see a page is not evidence that there is nothing on it, and deleting on that footing would throw away exactly the pages this program had most trouble with. A document whose every page is blank is refused rather than emptied, and that is not a corner case: nine of two hundred real forms are single-page documents whose one page reads "Please wait... your PDF viewer may not be able to display this type of document". They are XFA forms, whose real content is XML that no PDF viewer draws. The page is blank; the document is not. Over those two hundred forms — 773 pages — it drops 42, which is 5.4%. 100% statement coverage, go vet and -race clean, the wasm build, and the browser check.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A duplex stack run through a single-sided feeder comes back with a blank behind every one-sided sheet. With
Interleavein ops v0.9.0 putting the two halves of such a scan back together, this is the other half of that job.The threshold was measured, not decreed
Over 395 pages of government forms and library scans, the ink on a page runs to a median of 4.7% and falls away sharply below one part in a thousand: eleven pages are under it, two are under a twentieth of it, and the pages between are covers carrying a rule and nothing else. A page with a line of text is around three parts in a thousand — so this keeps one and drops a page number alone, which is what a blank back with a footer is.
Picking it by eye would have been wrong twice over. The first measurement used a threshold of 200 rather than 128 and reported a median of 18% ink and not one blank page in 395: at that setting the off-white background of a scan counts as ink. Which threshold is used decides the answer, so the answer has to say which.
Two refusals, both load-bearing
A page that cannot be drawn is not counted blank. Not being able to see a page is not evidence that there is nothing on it, and deleting on that footing would throw away exactly the pages this program had most trouble with.
A document whose every page is blank is refused rather than emptied — and that is not a corner case. Nine of two hundred real forms are single-page documents whose one page reads "Please wait... your PDF viewer may not be able to display this type of document". They are XFA forms, whose real content is XML that no PDF viewer draws. The page is blank; the document is not.
Over those two hundred forms — 773 pages — it drops 42, which is 5.4%.
100% statement coverage,
go vetand-raceclean, the wasm build, and the browser check.