Fix image masks (PDFBOX-6077)

[valerybokov]

PDFBOX-6077: Fix stencil masks filled with a pattern

Summary

A PDF image used as a stencil mask (an ImageMask) can be filled with a pattern instead of a solid color. PDFBox renders this case specially: it draws the pattern's paint and the mask into two separate scratch images, then combines them pixel-by-pixel before compositing the result onto the page. Two independent bugs in that combine step caused patterns used this way to render incorrectly.

Bug 1 — mask alpha overwrote the paint's own alpha

The combine step did:
rasterPixel[3] = alphaPixel[0];
unconditionally replacing the paint's alpha with the mask's alpha. Any pixel the pattern itself never painted into — for example the gaps between tiles of a tiling pattern — has alpha 0 in the paint image, but that got overwritten with the mask's (opaque) alpha, turning transparent gaps into solid black.

Fix: combine the two alphas by multiplication instead of overwriting:
rasterPixel[3] = rasterPixel[3] * alphaPixel[0] / 255;
so a pixel is only visible where both the pattern painted something and the mask allows it through.

Bug 2 — soft-masked patterns rendered fully transparent

A pattern can itself have a soft mask applied to it (SoftMask, wrapping the pattern's own Paint). SoftMask looks up its backing (grayscale) raster using absolute page-device pixel coordinates, fixed at the point the soft mask's transparency group was rendered. Because the stencil-mask-with-pattern code fills into an isolated scratch image — not the real page Graphics2D — those coordinates no longer lined up with anything, and the soft mask silently applied zero alpha everywhere, making the pattern disappear entirely.

Fix: unwrap the soft mask, fill the scratch image with its plain underlying paint, then apply the soft mask's own alpha afterward via a new PageDrawer.applySoftMaskAlpha(), which looks up the mask's backing raster directly using a per-pixel device-coordinate transform — correct regardless of the scratch image's resolution — rather than relying on the Paint/PaintContext machinery that assumes it's rendering onto the real page raster. SoftMask gains a few narrow package-private accessors for this.

Commits

1. PDFBOX-6077: combine stencil mask alpha with the pattern's own alpha
2. PDFBOX-6077: fix soft-masked patterns used as a stencil mask fill

---

Per https://www.apache.org/legal/generative-tooling.html: portions of this PR were
produced with assistance from Claude Code (Anthropic), based on a bug was described in Apache PDFBOX Issue Tracker.

I've reviewed the generated code and confirm to the best of my knowledge that the output does not include any
third-party copyrighted material and is compatible with the Apache License 2.0.

[THausherr]

This change produces a white line in the rendering of the file from PDFBOX-5403, in the text "Enzian Immobilien".

update: I tried "fix 1" alone and had just that.

[valerybokov]

This change produces a white line in the rendering of the file from PDFBOX-5403, in the text "Enzian Immobilien".

update: I tried "fix 1" alone and had just that.

Root cause: the alpha-combine fix (fix 1) correctly exposes any place where the pattern's own alpha isn't fully opaque — but TilingPaint has hairline (1-pixel), fully-transparent seams at tile boundaries from sub-pixel rounding, which used to be invisible (the old code just overwrote alpha with the mask's, ignoring the paint's). In the PDFBOX-5403 file, a pattern is tiled across many repeated ~14pt-tall bands (one per text line), and the seam lands right in the middle of each line, showing as a light gray/white streak through the text.

Fix: before combining, widen the paint's own alpha to the max of its 4-neighbors (a 1px dilate) — absorbs hairline seams while leaving genuinely large gaps (many pixels wide, like the original PDFBOX-6077 case) untouched.