test(out): publish the insert-lane batch from one source transaction - #159
Merged
Merged
Conversation
`insert_lane_position_is_the_sequence` fails 10/10 on main (Concourse `tests` #177 and #178, both on 8f6cda8; #176 on 6cbaf3d was green): assertion `left == right` failed: the batch ended on two skipped events, so its watermark is 3 left: EventSequence(1) right: EventSequence(3) That assertion is the test's own precondition, not the invariant. The invariant — `FlushOp::position()` is the last fully handled event, skips included — is checked on every flush a few lines above and passes; the failing line only says the batch never ended on skips, because it closed after the first event instead of spanning all three. It closed there legitimately. The runner holds a batch open only over events already buffered ("a pending stream is itself the flush trigger"); the one exception, `None if in_group => persistent.next().await`, is the commit lane's group atomicity, and `Transport::insert` hardcodes `boundary: true`, so `in_group` is never set on the insert lane. Spanning two separate commits was therefore always a race, never a promise. #157 made that race deterministic rather than merely likely. `PersistEvents::post_commit` used to `sender.send` each committed event inline; it now calls `CacheFeeder::accept`, and the feeder task feeds one batch per iteration with `await_consumed(last).await` between them. Two commits are two batches, so the second cannot reach the broadcast until the runner has consumed the first — and the first is exactly what closes the batch. The insert lane's delivery contract is unchanged. Publishing all three from one transaction gives the feeder one page, which `feed_page` sends without an await, so the runner finds events 2 and 3 buffered when event 1's handler returns. The gate that stood in for this is now redundant and removed. Discriminating power is unchanged: the flush carrying only row 1 must still land at position 3, which a `max` over the collected rows would report as 1. Verified: 20/20 on the fixed test, 192/192 on the full suite with `--no-fail-fast` (`cargo nextest run --workspace`), and `nix flake check` all green. Test-only change; no source file is touched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mainis red on Concourse —tests#178 and #177, both on8f6cda8(the #156 merge). #176, on6cbaf3d(the #157 merge), was green. One test fails, 10/10 deterministically, and it is the only failure — the rest of the suite is 191/192 with--no-fail-fast.It is the test's precondition, not the invariant
The invariant the test exists for —
FlushOp::position()is the last fully handled event, skips included — is asserted over every flush a few lines above, and it passes. The failing line asserts only the shape of the batch: that it ended on two skipped events. It did not, because it closed after the first event.Closing there is correct
The runner keeps a batch open only over events it finds already buffered:
The single exception is
None if in_group => persistent.next().await, which exists for the commit lane's group atomicity. On the insert laneTransport::inserthardcodesboundary: true, soin_groupis never set — the insert lane makes no group promise at all. A batch spanning two separate commits was always a race the test happened to win, never a guarantee.What #157 changed
PersistEvents::post_commitused to push each committed event into the cache-fill broadcast inline:It now hands the batch to
CacheFeeder::accept, and the feeder task feeds one batch per loop iteration, withawait_consumed(last).awaitbetween them. Two commits are two batches, so the second cannot reach the broadcast until the runner has consumed the first — and consuming the first is exactly what closes the batch. A latent race became a certainty.No delivery semantics changed. Ordering, exactly-once and the position/checkpoint contract are untouched; what moved is when a locally committed batch becomes visible to an in-process listener.
The fix
Publish all three events from one source transaction. That is one feeder batch, one
feed_page, andfeed_pagesends its whole page with no await — so events 2 and 3 are buffered when event 1's handler returns, independent of scheduling. TheNotifygate that stood in for this is redundant and removed.Discriminating power is unchanged: the flush carrying only row
1must still land at position3, which amaxover the collected rows would report as1.Verification
mainbefore the change.cargo nextest run --workspace --no-fail-fast: 192/192.nix flake check(fmt, clippy--deny warnings, audit, deny, alejandra): all checks passed — reported separately from the test run.cargo docbaseline is unaffected.Worth a second opinion
The insert lane's opportunistic batching is now noticeably harder to hit across separate in-process commits, since each one waits for the previous to be consumed. That costs throughput, not correctness, and only for handlers fast enough to drain between commits — but it is a real behavioural shift from #157 that no test pins, so flagging it rather than deciding it.
🤖 Generated with Claude Code
Note
Low Risk
Test-only change in
tests/lane_typed_delivery.rs; no production or delivery semantics touched.Overview
Fixes
insert_lane_position_is_the_sequence, which started failing deterministically after cache-feed batching made separate in-process commits visible one at a time.The test still checks that
FlushOp::position()on the insert lane reflects the last fully handled event (including skips), not just the max of collected rows. To get a batch that spans Ping(1) plus two skipped events, it now publishes[1, 2, 3]in a single transaction instead of two commits synchronized with aNotifygate.InsertPositionRecorderno longer blocks the handler on a gate; that coordination was only papering over a race that insert-lane batching never guaranteed across commits.Reviewed by Cursor Bugbot for commit f0adc1b. Bugbot is set up for automated code reviews on this repo. Configure here.