Skip to content

Reject malformed live budget counters - #62

Closed
albertovincenzi wants to merge 3 commits into
fix/fail-closed-live-statefrom
fix/reject-malformed-budget-state
Closed

Reject malformed live budget counters#62
albertovincenzi wants to merge 3 commits into
fix/fail-closed-live-statefrom
fix/reject-malformed-budget-state

Conversation

@albertovincenzi

Copy link
Copy Markdown
Collaborator

Problem

A present Queen KV budget row is currently decoded with defaults: a non-integer or missing value becomes 0, and a missing or malformed expiresAt becomes "retry now". Unexpected and duplicate keys are accepted too.

That makes corrupt or version-skewed live state look healthy. Operator APIs can show unused budget, ETA can become too optimistic, and the relay can repeatedly retry a type-refused counter with no real window deadline.

There is a second integrity edge: the batched charge may have applied some increments before its trailing read reveals malformed state. Returning only an error without compensation would leak those known charges.

Fix

  • Decode present counter rows strictly: integer value, valid mandatory expiry, requested key only, and one row per returned key.
  • Keep absent keys valid: getMany omits them, so no State is fabricated.
  • Use the same strict decoder in read-side APIs and in the relay's trailing post-charge read.
  • If a post-charge decode fails, refund every applied increment whose returned post-value proves the exact window it reached before returning the error.
  • Preserve the existing safe degradation only when the broker omits the entire trailing read result; admission verdicts remain usable and an unknown deadline remains retry-now.

Before / after

Before: corrupt present state could be reported as value 0 with no wait, or could drive a rapid refusal loop.

After: reads fail explicitly (HTTP 502 through #59), relay work is released for later redelivery, and any provably applied increments are guarded and refunded.

Verification

  • cargo test --workspace
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo fmt --check
  • Unit coverage for non-integer values, missing/invalid expiry, duplicate keys, and unexpected keys.
  • Live broker coverage plants a forever counter, checks graph/ETA/target/shared-budget/product endpoints return 502, applies a charge, and verifies the counter is restored to its original value before the decode error escapes.

Review / dependency

This PR is intentionally based on #59 so read-side Queen errors already have one consistent HTTP 502 contract. Review commit 031beb4 as this complete fix. It is independent of #61 and can be rebased after #59 lands.

albertovincenzi and others added 3 commits September 5, 2026 02:07
The two halves of a counter row cannot fail the same way, and treating them
alike put an unbounded outage on the admission path.

`charge` is the hot path, and a failed charge is NOT a refusal: the relay reads
it as "the batch did not happen" and releases the claim, so the identical batch
comes back and fails again. A single row with a missing or unrenderable expiry
therefore stopped every path on that counter for ever, at twice the kv write
volume, admitting nothing and healing never. The row is reachable — an operator
`put` without a TTL is exactly what this branch's own test plants.

An expiry costs the PARK DEADLINE and nothing else, and `None` there has always
meant "retry now" rather than "wait for ever". So it degrades, loudly, and the
charge is still decided by the counter.

A VALUE that is not an integer keeps failing closed: there is nothing to reason
about, and the read-side lie this branch exists to remove is precisely that one
being reported as zero. The live test now plants that case for the 502
assertions and proves the expiryless counter still admits.

Claude-Session: https://claude.ai/code/session_012K8u7BEJyd6nDNMCQAgH3z
@alice-viola

Copy link
Copy Markdown
Contributor

Landed on master via #67 (merge commit 944ee9b) as part of the 62-PR integration — this PR's head commit 0b92473 is an ancestor of master. GitHub could not mark it merged automatically because its base is fix/fail-closed-live-state, not master. Closing as landed.

@alice-viola alice-viola closed this Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants