fix(deps): update the fleet - #18
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
v0.3.0→v0.6.0v0.11.0→v0.12.0v0.277.0→v0.284.0Release Notes
go-pdfkit/reader (github.com/go-pdfkit/reader)
v0.6.0: — salvage, kept out of the field callers paint fromCompare Source
Three defects were put to this release. One of them did not exist, and that is reported as plainly as the two that did.
A broken filter chain kept what it had, and now says it is salvage
Half of this was already here and lying:
flateDecodereturned a damaged stream's inflated prefix with anilerror, so 37 streams — 33 in 18 real forms, 4 in fixtures — were reported as clean decodes of a fragment. The other half was missing: 263 streams in 212 of the 1 633 real forms (13.0%) failed and yielded nothing at all.The recovered bytes are now kept structurally apart from the clean ones.
Decoded.Dataholds only what a filter decoded;Decoded.Undecodedholds what the chain could not get past, set instead ofData, never beside it. Writing that guarantee as a test found two live holes:cryptFilterwas returning ciphertext asData, and v0.5.0's own CCITT change meant a fax with a bad/Columnsfailed inside the chain with its encoded bytes landing inData— a fax handed to a stencil path as if it were samples, which is exactly the shape of the defectrenderv0.10.0 had just fixed. The guarantee is a type, not a flag./Lengthagainst the cross-reference table — there was nothing to fixThe cross-check is already in
parse.go. Measured over 344 367 streams in 4 283 files: 47 streams need the scan fallback, 0 are ambiguous, 0 truncate, 0 overrun the next object header. No change was made, and none was needed.A masked error, worse than described
Two faults. The masking bites 30 of 30 arXiv open failures: every one reported
no startxref in the last 4096 byteswhere the rebuild had actually foundthe file holds no indirect objects(28) orno document catalogue found(2). And an ordering fault —repair()read object streams before the decryption key existed, so an encrypted file with broken tables was rebuilt from unreadable bytes. That one bites 0 files in either corpus; it ships anyway, with the negative result stated rather than dressed up.Three the search turned up on its own
/Cryptis the biggest real gap: 209 of the 263 failures, in 209 files — 12.8% of the real forms. Implementing only the filter would be measurably the worst state (209 of 209 decode "cleanly", 0 of 209 give readable XMP); both halves together give 209 of 209 readable.A second map-order non-determinism, in
indexObjectStreams— distinct from theExpandedfix in v0.4.2, and still giving 5 answers in 5 runs with that fix in. The tie-break now matches what the header scan already does: later in the file wins./StmFand/StrFboth/Identitymeans nothing in the file is encrypted and no password is needed. 3 of 495 encrypted files; all three now open with 30, 1 and 30 pages — poppler's exact numbers.Measured
The two arXiv files differing by one byte differ by a redundant newline for an empty stream in a
/Contentsarray; operation count and operator-sequence hash are identical on both.100% statement coverage,
go vetand-raceclean, nine cross-compile targets — each built one per line, with bogusGOOS/GOARCHas controls. Two fuzz targets added where there were none. Nothing outside the standard library.v0.5.0: — a scanned form is a fax, and a fax was a blank pageCompare Source
A scanned form is a fax, and a fax was a blank page
67 of the 1 633 real forms in the corpus — the eleven issuing bodies, not the vendor test suites — carry a CCITT-encoded image, 263 images between them, and 6 of their pages have nothing else on them at all. Those pages drew entirely blank, and a blank page is the failure a reader notices before any other.
Group 3 one-dimensional, Group 3 mixed and Group 4 are all decoded, with
/K,/Columns,/Rows,/BlackIs1,/EncodedByteAlignand/EndOfBlockread.Why it belongs in the reader
The other filters this package stops at carry an image with its own idea of how many components it has and how deep they are. A fax does not: it produces bilevel samples, one bit a pixel, rows padded to a byte, and the stream dictionary says what they mean. That is a byte stream, so it is a filter — and decoding it here means every caller gets it rather than each writing its own.
ImageFilterno longer returns true for/CCITTFaxDecodeor/CCF. That is a change to a documented contract: a caller that switched on the filter name and handled the fax itself will now receive samples with an empty filter name instead.The reference, read before writing
ITU-T T.4 and T.6, by way of the tables and the changing-element algorithm in
golang.org/x/image/ccitt— read, not imported, because this package has no dependencies outside the standard library and gains none. The 218 code table entries were extracted mechanically from that package'sgen.gorather than retyped, because a table of 218 variable-length codes transcribed by hand is a table with a mistake in it.Measured against that reference, image by image
Every CCITT image in the corpus, decoded by both and compared byte for byte, with both sides working the height out for themselves where
/Rowsis absent:The three the reference gives up on — "invalid code", "invalid offset" — are pages 2, 3 and 4 of
fr-cerfa/cerfa_10455.pdf, where it stops and we decode all 3 504 rows, exactly the/Heightthe image dictionary names. Being more forgiving is deliberate: a damaged scan comes back as far as it got, because refusing it turns a form into a blank page.Two things the measurement found that reading the specification did not
Zero bits decode as perfectly good two-dimensional modes — pass mode is
0001, vertical-left-3 is0000010— so a decoder that does not stop at the padding goes on inventing rows that look like the last real one.fr-cerfa/cerfa_10701gave 1 636 rows for a 208-row image that way, every surplus row with plausible ink on it.The fuzz target found the second: on a truncated fax the early return skipped the padding that makes the answer as long as
/Rowspromised, so two bytes of input gave one row where the caller had asked for four, with no error to say so. 6.7 million fuzz executions since, none failing.What it changes when a page is drawn
Through
render, on the first page of all 1 633 real forms, 69 pages change: one goes from blank to drawn (fr-cerfa/cerfa_11818, 0 inked pixels to 28 989), fourteen gain ink, and fifty-one lose ink.That last number is the interesting one. Those pages were not blank before — they were noise. 273 of the corpus's image masks carry an encoded filter, and
renderhanded a mask to its stencil path before asking whether the bytes were still compressed, so a fax was drawn as a stencil of its own compressed bytes. Decoding it replaces the noise with the picture, which has less ink in it because most of a form is white. Fixed on the other side in go-pdfkit/render#20; the 9 JBIG2 masks that remain are now not drawn rather than drawn wrong.The same file drew a different picture on different runs
An inline image dictionary may carry both spellings of a key and disagree with itself —
/W 20beside/Width 10— and expanding it in one pass over the map let Go's deliberately randomised iteration order pick the winner. Eight renders of safedocs'Inline_Image_Abbreviationsfixture gave five different answers; now they give one. Measured scope: of 3 668 corpus files rendered three times over, exactly one disagrees with itself, and it is the fixture built for this case.100% statement coverage,
go vetand-raceclean, nine cross-compile targets, nothing outside the standard library.v0.4.2: — the same file drew a different picture on different runsCompare Source
The same file drew a different picture on different runs
An inline image dictionary may say the same thing twice and disagree with itself:
/W 20beside/Width 10,/CS /RGBbeside/ColorSpace /3chanRGB,/Dbeside/Decodewith the components reversed.Expanded()walked the dictionary in one pass, so when both spellings were present the winner was whichever Go's map iteration happened to yield last — and that order is deliberately randomised per range.Eight renders of safedocs'
Inline_Image_Abbreviationsfixture, same binary, same file:79b9edef79b9edefb2594aef6bf897476bf89747544c8e6b79b9edef43030bf387f31291× 8Five different answers out of eight, against one.
The scope is measured, not guessed
Rendering the first page of 3 668 corpus files three times over — the 1 633 real forms, 1 400 arXiv papers and all 635 vendor fixtures — turns up exactly one file that disagrees with itself, and it is that fixture, which safedocs built for this case: its seven images each pair an abbreviation against the written-out name, with comments marking the line to remove "to see the effect".
So the corpus measurements already taken are not contaminated. One file is, and it is the one designed to be.
Which spelling wins
The specification permits both in an inline image and says nothing about precedence, and the public implementations disagree: pdf.js asks for the abbreviation and falls back to the written-out name; MuPDF asks the other way round.
The abbreviation wins here, for two reasons: it is the spelling Table 93 gives for an inline image, the written-out name being the tolerated alias; and safedocs marks the abbreviation as the line whose removal changes what a viewer shows, which is a statement about the reference viewer.
Two passes settle it — the written-out names first, then the abbreviations over them. No order within either pass can matter, because the names in each are distinct, so the answer no longer depends on the run whichever way it is read.
The test expands the same dictionary two hundred times and requires every answer to match the first. Against the parent commit it fails on the first comparison, reporting
Height = 40where the first gave10.100% statement coverage,
go vetand-raceclean, nine cross-compile targets, no dependencies outside the standard library.v0.4.1: — a 219-byte denial of serviceCompare Source
Repairing a damaged file means listing its objects in order, and that was done by counting from nought to the highest object number and asking the cross-reference table about each one.
A file may name any object number it likes. mozilla/pdf.js's
bug1980958.pdfis 219 bytes and declares object2147483beside its three real ones. Counting up to that is two thousand million map lookups, done twice, for three objects.219 bytes in, Open took 25.39s219 bytes in, Open took 0sNo allocation either way — it was a loop, not a memory bomb, which is why nothing caught it. Any file a reader opens could do this.
The numbers that exist are collected, sorted and walked, which is what the order was wanted for in the first place.
Found by battle-testing
go-pdfkit/formsagainst 1 046 files — 69 public forms plus the whole of mozilla/pdf.js's test corpus, which is adversarial by construction. 1 029 opened, 14 580 fields read and filled, no panics, and this one file took 12.87 of the run's 13.1 seconds.100% statement coverage,
go vetclean.v0.4.0: — say how a file is protectedCompare Source
Document.Protection()reports what a file said about itself on the way in: the method in words (RC4-40,RC4-128,AES-128,AES-256, ornonefor a file that declares a security handler and protects nothing with it), the handler's revision, the permissions it grants, and whether the password it opened with was the owner's — in which case those permissions do not apply.PermissionsgainsAllowsandString, so what a file grants reads asprint, copyrather than as a bit pattern.100% statement coverage.
go-widgets/painter (github.com/go-widgets/painter)
v0.12.0Compare Source
go-widgets/toolkit (github.com/go-widgets/toolkit)
v0.284.0Compare Source
v0.283.0Compare Source
v0.282.0Compare Source
v0.281.0Compare Source
v0.280.0Compare Source
v0.279.0Compare Source
v0.278.0Compare Source
Configuration
📅 Schedule: (UTC)
🚦 Automerge: Enabled.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.
This PR has been generated by Mend Renovate CLI.