From 94cd157eb151e821648fa72d3834cdd60310eee2 Mon Sep 17 00:00:00 2001 From: tannevaled Date: Mon, 31 Aug 2026 15:15:55 +0200 Subject: [PATCH 1/3] Put a JPEG's chroma back the way every other reader does MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `render` decodes JPEG with Go's `image/jpeg`, which hands back an `*image.YCbCr`; turning that into pixels through `raster.FromImage` reads chroma through `image.YCbCr.COffset`, and COffset divides x and y by the sampling factor (image/ycbcr.go:101). A 4:2:0 picture's chroma is therefore REPLICATED: four pixels share one sample exactly. libjpeg interpolates instead, weighting the nearer chroma sample 3 and the further one 1 in each direction — "fancy upsampling", on by default (jdapimin.c:229). poppler never touches the flag (DCTStream.cc:98), and neither do pdfium, mupdf or Ghostscript, so all four interpolate. The two are not close, and the difference is the whole of go-pdfkit/render#40. Measured against `pdfimages` on the conformance corpus, replication differs from poppler by 16 to 62 levels of 255 across a third to a half of a 4:2:0 picture's pixels. Nothing else in the decode disagrees: over 112 real greyscale JPEGs the largest channel difference from poppler is ONE level, the ISO/IEC 10918-2 IDCT allowance exactly, and a synthetic 4:4:4 picture comes out of Go's decoder byte for byte identical to libjpeg's. Neither has any chroma to put back. So this reproduces libjpeg's filter, and only where libjpeg applies it: 4:2:0, 4:2:2 and 4:4:0, and within those only when the chroma plane is more than two samples wide, because libjpeg falls back to replication below that (jdsample.c, `compptr->downsampled_width > 2`). 4:1:1 and 4:1:0 have no fancy method there at all — they go through int_upsample, which replicates — so they are left alone too, and a test asserts each of those cases comes out exactly as `raster.FromImage` would have made it. # WHAT IT IS WORTH On the picture the investigation reduced to — a 75x75 4:2:0 logo in `us-dol/CA-10.pdf` — our output went from 56 levels away from poppler on 48.3% of its pixels to 3 levels away on 0.018% of them. The 3 that remain are two conformant IDCTs each within the ISO allowance of the reference and so up to 2 apart in Y, Cb and Cr, carried through the colour matrix; they are not this. # ROWS RATHER THAN PLANES The reconstruction is made one row at a time into a buffer the plane owns, not into two whole extra planes beside the picture. A picture is already the largest thing a page allocates, and reconstructing both planes in full would have raised what a JPEG costs by half for no gain. An edge row reads the last real row twice rather than reading the MCU padding that follows it in memory, which is what libjpeg does (jdmainct.c:217) and is the only reading under which the two agree. --- chroma.go | 196 +++++++++++++++++++++++++++++++++++++++++++++++++ chroma_test.go | 163 ++++++++++++++++++++++++++++++++++++++++ image.go | 2 +- 3 files changed, 360 insertions(+), 1 deletion(-) create mode 100644 chroma.go create mode 100644 chroma_test.go diff --git a/chroma.go b/chroma.go new file mode 100644 index 0000000..00f360f --- /dev/null +++ b/chroma.go @@ -0,0 +1,196 @@ +// Copyright (c) 2026, the go-pdfkit/render authors +// All rights reserved. +// +// SPDX-License-Identifier: BSD-3-Clause + +package render + +import ( + "image" + "image/color" + + "github.com/go-gfx/gfx/raster" +) + +// jpegPixels turns a decoded JPEG into pixels, putting subsampled chroma back +// to full resolution the way every libjpeg-based PDF reader does. +// +// # WHY THIS EXISTS +// +// A JPEG that stores its chroma at half resolution — 4:2:0, which is what +// nearly every colour JPEG in the corpus is — has to have that chroma put back +// before it can be turned into RGB, and ISO/IEC 10918 does not say how. There +// are two answers in the field: +// +// - REPLICATE each chroma sample over the pixels it covers. That is what +// [image.YCbCr] does: its COffset divides x and y by the sampling factor +// (image/ycbcr.go:101), so four pixels share one chroma sample exactly. It +// is also what pdf.js does — src/core/jpg.js truncates with +// "0 | (x * componentScaleX)". +// - INTERPOLATE, weighting the nearer chroma sample 3 and the further one 1 +// in each direction. That is libjpeg's "fancy upsampling", which is on by +// default (jdapimin.c:229), and so it is what poppler (DCTStream.cc:98 +// never touches the flag), pdfium, mupdf and Ghostscript all do. +// +// The two are not close. Both were measured against poppler over the whole +// conformance corpus for go-pdfkit/render#40: replication differs from it by +// 16 to 62 levels of 255 over a third to a half of a 4:2:0 picture's pixels, +// and interpolation differs by at most 3, on a hundredth of a percent of them. +// Every other part of the decode already agreed to the bit — a greyscale JPEG +// and a 4:4:4 one come out of Go's decoder byte for byte identical to +// libjpeg's, because neither has any chroma to put back. +// +// It reproduces libjpeg's filter only where libjpeg applies it: 4:2:0, 4:2:2 +// and 4:4:0, and within those only when the chroma plane is more than two +// samples wide, because below that libjpeg falls back to replication +// (jdsample.c:503 and :534, "compptr->downsampled_width > 2"). 4:1:1 and 4:1:0 +// have no fancy method in libjpeg at all — they go through int_upsample, which +// replicates (jdsample.c:552) — so they are left alone here too. +func jpegPixels(img image.Image) *raster.Image { + yc, ok := img.(*image.YCbCr) + if !ok { + return raster.FromImage(img) + } + hf, vf := chromaFactors(yc.SubsampleRatio) + b := yc.Bounds() + w, h := b.Dx(), b.Dy() + // The dimensions libjpeg's upsampler reads, its downsampled_width and + // downsampled_height. The planes are wider than this when the picture does + // not fill its last MCU, and what is past them is padding rather than + // picture: libjpeg reconstructs an edge by repeating the last real sample + // (jdmainct.c:217), not by reading the padding. + cw, ch := (w+hf-1)/hf, (h+vf-1)/vf + if hf*vf == 1 || cw <= 2 { + return raster.FromImage(img) + } + cb := chromaRows(yc.Cb, yc, cw, ch, hf, vf) + cr := chromaRows(yc.Cr, yc, cw, ch, hf, vf) + out := raster.New(w, h) + for y := 0; y < h; y++ { + u, v := cb.at(y), cr.at(y) + for x := 0; x < w; x++ { + l := yc.Y[yc.YOffset(b.Min.X+x, b.Min.Y+y)] + r, g, bl := color.YCbCrToRGB(l, u[x], v[x]) + o := (y*w + x) * 4 + out.Pix[o], out.Pix[o+1], out.Pix[o+2], out.Pix[o+3] = r, g, bl, 255 + } + } + return out +} + +// chromaFactors is how many pixels one chroma sample covers each way, and is +// 1 by 1 — meaning "leave it to [raster.FromImage]" — for every ratio libjpeg +// reconstructs by replication rather than by interpolating. +func chromaFactors(r image.YCbCrSubsampleRatio) (hf, vf int) { + switch r { + case image.YCbCrSubsampleRatio420: + return 2, 2 + case image.YCbCrSubsampleRatio422: + return 2, 1 + case image.YCbCrSubsampleRatio440: + return 1, 2 + } + return 1, 1 +} + +// A plane is one chroma plane and how to read a full-width row out of it. +// +// The rows are made one at a time into a buffer this owns rather than into a +// second picture, because a picture is already the largest thing a page +// allocates and reconstructing two whole planes beside it would raise what a +// JPEG costs by half. +type plane struct { + src []uint8 + base, stride int + cw, ch int + hf, vf int + buf []uint8 +} + +// chromaRows is the row source for one plane of a subsampled picture. +func chromaRows(src []uint8, yc *image.YCbCr, cw, ch, hf, vf int) *plane { + b := yc.Bounds() + return &plane{src: src, base: yc.COffset(b.Min.X, b.Min.Y), stride: yc.CStride, + cw: cw, ch: ch, hf: hf, vf: vf, buf: make([]uint8, 2*cw)} +} + +// row is one row of stored chroma, with an index past either end reading the +// row at that end. libjpeg reconstructs a picture's edge from a duplicate of +// its last real row rather than from the padding that follows it in memory +// (jdmainct.c:217), and the padding is what the difference would be made of. +func (p *plane) row(cy int) []uint8 { + if cy < 0 { + cy = 0 + } + if cy >= p.ch { + cy = p.ch - 1 + } + o := p.base + cy*p.stride + return p.src[o : o+p.cw] +} + +// at is one full-width row of chroma for output row y. +func (p *plane) at(y int) []uint8 { + switch { + case p.vf == 1: + h2v1Row(p.buf, p.row(y), p.cw) + case p.hf == 1: + // The upper output row of a pair takes its second-nearest row from + // above and the lower one from below, and the two round the other way + // from each other (jdsample.c, h1v2_fancy_upsample). + h1v2Row(p.buf, p.row(y/2), p.row(farRow(y)), p.cw, 1+y%2) + default: + h2v2Row(p.buf, p.row(y/2), p.row(farRow(y)), p.cw) + } + return p.buf +} + +// farRow is the second-nearest chroma row of output row y, which is the one +// above for the upper of a pair and the one below for the lower. +func farRow(y int) int { + if y%2 == 0 { + return y/2 - 1 + } + return y/2 + 1 +} + +// h2v1Row is libjpeg's h2v1_fancy_upsample for one row: three quarters of the +// nearer chroma sample and one quarter of the further one. +func h2v1Row(dst, in []byte, cw int) { + dst[0] = in[0] + dst[1] = byte((int(in[0])*3 + int(in[1]) + 2) >> 2) + for c := 1; c < cw-1; c++ { + v := int(in[c]) * 3 + dst[2*c] = byte((v + int(in[c-1]) + 1) >> 2) + dst[2*c+1] = byte((v + int(in[c+1]) + 2) >> 2) + } + dst[2*cw-2] = byte((int(in[cw-1])*3 + int(in[cw-2]) + 1) >> 2) + dst[2*cw-1] = in[cw-1] +} + +// h1v2Row is libjpeg's h1v2_fancy_upsample for one row, which interpolates +// down the column only. The bias is 1 for the upper row of a pair and 2 for +// the lower, which is how libjpeg keeps the pair's rounding from drifting. +func h1v2Row(dst, near, far []byte, cw, bias int) { + for c := 0; c < cw; c++ { + dst[c] = byte((int(near[c])*3 + int(far[c]) + bias) >> 2) + } +} + +// h2v2Row is libjpeg's h2v2_fancy_upsample for one row: nine sixteenths of the +// nearest chroma sample, three of each neighbour and one of the diagonal. +func h2v2Row(dst, near, far []byte, cw int) { + col := func(c int) int { return int(near[c])*3 + int(far[c]) } + last := col(0) + dst[0] = byte((last*4 + 8) >> 4) + this := col(1) + dst[1] = byte((last*3 + this + 7) >> 4) + for c := 1; c < cw-1; c++ { + next := col(c + 1) + dst[2*c] = byte((this*3 + last + 8) >> 4) + dst[2*c+1] = byte((this*3 + next + 7) >> 4) + last, this = this, next + } + dst[2*cw-2] = byte((this*3 + last + 8) >> 4) + dst[2*cw-1] = byte((this*4 + 7) >> 4) +} diff --git a/chroma_test.go b/chroma_test.go new file mode 100644 index 0000000..764cf17 --- /dev/null +++ b/chroma_test.go @@ -0,0 +1,163 @@ +// Copyright (c) 2026, the go-pdfkit/render authors +// All rights reserved. +// +// SPDX-License-Identifier: BSD-3-Clause + +package render + +import ( + "bytes" + "image" + imgcolor "image/color" + "image/jpeg" + "testing" + + "github.com/go-gfx/gfx/raster" +) + +// ycbcr is a chroma plane written into a picture of w by h pixels, so a test +// can say what one plane holds without saying anything about the other two. +func ycbcr(t *testing.T, w, h int, r image.YCbCrSubsampleRatio, cb [][]uint8) *image.YCbCr { + t.Helper() + im := image.NewYCbCr(image.Rect(0, 0, w, h), r) + for y, row := range cb { + copy(im.Cb[y*im.CStride:], row) + copy(im.Cr[y*im.CStride:], row) + } + return im +} + +// TestChromaIsPutBackTheWayLibjpegPutsItBack pins the reconstruction against +// libjpeg's, one sampling at a time. +// +// The expected rows are not this code's output written down. They are +// libjpeg's expressions evaluated on these inputs: h2v1_fancy_upsample, +// h2v2_fancy_upsample and h1v2_fancy_upsample from jdsample.c, including the +// asymmetric biases — 1 for the upper row of a pair and 2 for the lower — that +// keep a pair's rounding from drifting one way. The whole point of this +// function is to agree with a decoder written in another language, so a test +// that only agreed with itself would assert nothing. +func TestChromaIsPutBackTheWayLibjpegPutsItBack(t *testing.T) { + for _, c := range []struct { + name string + ratio image.YCbCrSubsampleRatio + plane [][]uint8 + want [][]uint8 + }{{ + name: "4:2:0, interpolated both ways", + ratio: image.YCbCrSubsampleRatio420, + plane: [][]uint8{{0, 60, 120}, {180, 240, 255}}, + want: [][]uint8{ + {0, 15, 45, 75, 105, 120}, + {45, 60, 90, 117, 142, 154}, + {135, 150, 180, 202, 215, 221}, + {180, 195, 225, 244, 251, 255}, + }, + }, { + name: "4:2:2, interpolated across only", + ratio: image.YCbCrSubsampleRatio422, + plane: [][]uint8{{0, 60, 120}, {180, 240, 255}, {10, 20, 30}, {200, 100, 50}}, + want: [][]uint8{ + {0, 15, 45, 75, 105, 120}, + {180, 195, 225, 244, 251, 255}, + {10, 13, 17, 23, 27, 30}, + {200, 175, 125, 88, 62, 50}, + }, + }, { + name: "4:4:0, interpolated down only", + ratio: image.YCbCrSubsampleRatio440, + plane: [][]uint8{{0, 60, 120, 180, 240, 255}, {255, 240, 180, 120, 60, 0}}, + want: [][]uint8{ + {0, 60, 120, 180, 240, 255}, + {64, 105, 135, 165, 195, 191}, + {191, 195, 165, 135, 105, 64}, + {255, 240, 180, 120, 60, 0}, + }, + }} { + t.Run(c.name, func(t *testing.T) { + im := ycbcr(t, 6, 4, c.ratio, c.plane) + hf, vf := chromaFactors(c.ratio) + rows := chromaRows(im.Cb, im, (6+hf-1)/hf, (4+vf-1)/vf, hf, vf) + for y, row := range c.want { + got := rows.at(y) + for x, want := range row { + if got[x] != want { + t.Errorf("chroma at %d,%d is %d, libjpeg makes it %d", + x, y, got[x], want) + } + } + } + }) + } +} + +// TestChromaIsLeftAloneWhereLibjpegLeavesItAlone checks the pictures this must +// not touch: the ones libjpeg itself reconstructs by replication, where +// interpolating would make us the odd one out in the other direction. +func TestChromaIsLeftAloneWhereLibjpegLeavesItAlone(t *testing.T) { + grey := image.NewGray(image.Rect(0, 0, 4, 4)) + grey.Set(1, 1, imgcolor.Gray{200}) + for _, c := range []struct { + name string + img image.Image + }{ + {"no chroma to put back at all", grey}, + {"4:4:4 is already full size", + ycbcr(t, 6, 4, image.YCbCrSubsampleRatio444, [][]uint8{{1, 2, 3, 4, 5, 6}})}, + {"4:1:1 has no fancy method in libjpeg", + ycbcr(t, 8, 4, image.YCbCrSubsampleRatio411, [][]uint8{{1, 2}})}, + {"4:1:0 has no fancy method in libjpeg", + ycbcr(t, 8, 4, image.YCbCrSubsampleRatio410, [][]uint8{{1, 2}})}, + {"a chroma plane two samples wide is below libjpeg's own threshold", + ycbcr(t, 4, 4, image.YCbCrSubsampleRatio420, [][]uint8{{7, 9}})}, + } { + t.Run(c.name, func(t *testing.T) { + want := raster.FromImage(c.img) + got := jpegPixels(c.img) + if !bytes.Equal(got.Pix, want.Pix) { + t.Error("the picture was reconstructed where libjpeg replicates") + } + }) + } +} + +// TestASubsampledJPEGComesOutInterpolated is the whole path: a JPEG with a +// hard colour edge, encoded 4:2:0, drawn through the renderer. +// +// Under replication the two pixels either side of the edge are the two source +// colours exactly and the pair inside one chroma block is identical; under +// libjpeg's filter the edge is graded, so the assertion is that the two +// columns differ. That is the difference the corpus measures as 16 to 62 +// levels, seen in the small. +func TestASubsampledJPEGComesOutInterpolated(t *testing.T) { + src := image.NewRGBA(image.Rect(0, 0, 16, 16)) + for y := 0; y < 16; y++ { + for x := 0; x < 16; x++ { + if x < 8 { + src.Set(x, y, imgcolor.RGBA{255, 0, 0, 255}) + } else { + src.Set(x, y, imgcolor.RGBA{0, 0, 255, 255}) + } + } + } + var buf bytes.Buffer + if err := jpeg.Encode(&buf, src, nil); err != nil { + t.Fatal(err) + } + img, err := jpeg.Decode(bytes.NewReader(buf.Bytes())) + if err != nil { + t.Fatal(err) + } + if _, ok := img.(*image.YCbCr); !ok { + t.Fatalf("the encoder stopped subsampling: %T", img) + } + got, plain := jpegPixels(img), raster.FromImage(img) + if bytes.Equal(got.Pix, plain.Pix) { + t.Fatal("a 4:2:0 JPEG came out exactly as replication would have made it") + } + // The pair of pixels inside one chroma block, at the edge. + a, b := got.Pix[(8*16+6)*4:], got.Pix[(8*16+7)*4:] + if a[0] == b[0] && a[2] == b[2] { + t.Error("the two pixels of one chroma block are still identical") + } +} diff --git a/image.go b/image.go index 8bb61d7..cf0b1ba 100644 --- a/image.go +++ b/image.go @@ -312,7 +312,7 @@ func (r *renderer) decodeJPEG(data []byte, w, h int, inverted bool) *sampled { if b.Dx() != w || b.Dy() != h { w, h = b.Dx(), b.Dy() } - src := raster.FromImage(img) + src := jpegPixels(img) return &sampled{w: w, h: h, pix: src.Pix} } From 1679d8218f38e4e85185749d479df6933d238aa7 Mon Sep 17 00:00:00 2001 From: tannevaled Date: Mon, 31 Aug 2026 15:23:01 +0200 Subject: [PATCH 2/3] Cite the two selection sites exactly jdsample.c:506 is the 2h1v arm and :534 the 2h2v one; the generic replicating method is chosen at :553. --- chroma.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/chroma.go b/chroma.go index 00f360f..a0d6749 100644 --- a/chroma.go +++ b/chroma.go @@ -43,9 +43,9 @@ import ( // It reproduces libjpeg's filter only where libjpeg applies it: 4:2:0, 4:2:2 // and 4:4:0, and within those only when the chroma plane is more than two // samples wide, because below that libjpeg falls back to replication -// (jdsample.c:503 and :534, "compptr->downsampled_width > 2"). 4:1:1 and 4:1:0 +// (jdsample.c:506 and :534, "compptr->downsampled_width > 2"). 4:1:1 and 4:1:0 // have no fancy method in libjpeg at all — they go through int_upsample, which -// replicates (jdsample.c:552) — so they are left alone here too. +// replicates (jdsample.c:553) — so they are left alone here too. func jpegPixels(img image.Image) *raster.Image { yc, ok := img.(*image.YCbCr) if !ok { From 57eb4242bd4b62f7b0cee0abbc514006cfd73154 Mon Sep 17 00:00:00 2001 From: tannevaled Date: Mon, 31 Aug 2026 15:24:56 +0200 Subject: [PATCH 3/3] Name the two readers whose source is here, not a third whose is not The clone of ghostpdl in the reference library is a stub with no source tree in it, so "Ghostscript does this too" was a claim nothing here could be pointed at. pdfium and mupdf can be. --- chroma.go | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/chroma.go b/chroma.go index a0d6749..174f673 100644 --- a/chroma.go +++ b/chroma.go @@ -30,7 +30,8 @@ import ( // - INTERPOLATE, weighting the nearer chroma sample 3 and the further one 1 // in each direction. That is libjpeg's "fancy upsampling", which is on by // default (jdapimin.c:229), and so it is what poppler (DCTStream.cc:98 -// never touches the flag), pdfium, mupdf and Ghostscript all do. +// never touches the flag), pdfium (core/fxcodec/jpeg/jpeg_common.c:29) +// and mupdf (source/fitz/filter-dct.c:284) do too. // // The two are not close. Both were measured against poppler over the whole // conformance corpus for go-pdfkit/render#40: replication differs from it by