Skip to content

fix: REQ sketch loses min/max when merging into a new sketch - #170

Merged
proost merged 1 commit into
apache:mainfrom
jaideeppyne:fix-req-merge-minmax
Aug 31, 2026
Merged

fix: REQ sketch loses min/max when merging into a new sketch#170
proost merged 1 commit into
apache:mainfrom
jaideeppyne:fix-req-merge-minmax

Conversation

@jaideeppyne

Copy link
Copy Markdown

req.NewSketch() leaves minItem/maxItem at Go's zero value, but Merge and Reset both use NaN as the "unset" sentinel. So merging into a freshly created sketch computes min(0, other.minItem) and max(0, other.maxItem), and the 0 wins whenever the stream doesn't straddle it.

src, _ := req.NewSketch()
for i := 1001; i <= 1100; i++ { src.Update(float32(i)) }

dst, _ := req.NewSketch()
dst.Merge(src)

dst.MinItem()   // 0, want 1001
dst.Quantile(0) // 0, want 1001
dst.Rank(500)   // 0.01, want 0

It's not just the getters. The bogus 0 is reinserted into the sorted view as a retained quantile, so Quantile, Rank, CDF and PMF all shift, and it survives MarshalBinary + Decode. Chained merges into an empty accumulator, which is the usual map-reduce shape, hit it too. NewSketch() then Reset() then Merge works fine, which is the tell.

ReqSketch.java initialises the fields at declaration:

private float minItem = Float.NaN;
private float maxItem = Float.NaN;

Go has no field initialisers so the port dropped it. I set them in NewSketch instead. That covers every construction site: the decoder's empty and raw-items paths both go through NewSketch, and the exact and estimation paths read min/max out of the buffer. Update guards on IsEmpty() so it was never affected, and every reader of the fields (MinItem, MaxItem, refreshSortedView, the estimation-format encoder) is already behind an emptiness check, so no path can now observe the NaN.

How I found it: I built an invariant harness over the quantile families asserting that min/max after merging into a fresh sketch equals min/max after merging into a reset sketch, and equals the source's. 5 value patterns across REQ, KLL and t-digest. KLL and t-digest are clean; REQ fails 4 of 5. The one that passes is the case whose data straddles zero, which is why the existing Merge Multiple test misses it. I checked the intended semantics against ReqSketch.java rather than guessing at it. The same harness also ran 200-trial randomised sweeps per family for quantile monotonicity in rank, CDF within [0,1] and non-decreasing, PMF summing to 1, and serialize/deserialize identity; nothing else came out of it, and I confirmed tdigest.Quantile matches tdigest_impl.hpp line for line.

7 subtests added in TestSketchMergeIntoEmptyMinMax. All 7 fail on main with only the test file applied, all 7 pass with the two-line change. make lint-check and go test -race ./req/ are clean and the full suite is green.

Written with AI assistance (Claude). I reviewed the change and ran everything above myself.

NewSketch left minItem/maxItem at Go's zero value while Merge and Reset
use NaN as the unset sentinel, so merging into a freshly created sketch
computed min(0, other.minItem) and max(0, other.maxItem).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@proost proost left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice Catch! Thank you!

@proost
proost merged commit 38cfc6c into apache:main Aug 31, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants