test(retry): pin the 429 header-before-message ordering the contract publishes - #601
Conversation
…publishes `exitcodes_doc.go` and README publish a 🔴 bullet: THE HEADER IS CONSULTED BEFORE THE MESSAGE, so a cap-worded 429 carrying `Retry-After` exits 5, NOT 2. Nothing guarded it. Consulting the message first — return terminal on cap wording, whatever the header — inverts that bullet from 5 to 2 with the whole suite green, because retry_test.go's two 429 neighbours each hold one half of the input fixed: TestRead429WithRetryAfterExhaustsGeneric sends an EMPTY body, TestRead429WithoutRetryAfterIsTerminalWithCursorHint sends no header. Neither can distinguish the two orderings; it takes both at once. TestCapWorded429WithRetryAfterIsRetriedNotReclassified sends both: a cap-worded body (isDeepPagingCap matches) plus `Retry-After` on every response. It asserts the header decided (4 requests, not 1) and, per AGENTS.md item 7, asserts the classification by errors.Is — ErrNetwork (5), not ErrBadRequest (2), not ErrRateLimited (6) — since the sentinels carry no visible text. Matrix: green at HEAD (21 ok / 0 FAIL); under the mutant, 20 ok packages and exactly one `--- FAIL`, this test. So every pre-existing test stays green under it, confirming round 4's 21 ok / 0 FAIL reading from the other side. Also corrects the claim row's `pinnedBy`, which read "nothing local — the assumption is about the server". That conflated two claims: the REACHABILITY (does the server ever attach Retry-After to a cap 429) is genuinely unguardable and is why the bullet is published; the CLI's own ORDERING is local and now pinned. As written the field told the next maintainer no guard was possible — the description-reads-as-coverage failure that table exists to stop. The published bullets are unchanged: "that assumption is vendored and has no local guard" refers to the server half, which is still true. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013JQ4B1VHew33RNX5Ck7ELS
|
CI settled: 13/13 checks terminal, all SUCCESS, That closes the linter version caveat in the body from the other side — the |
…of only pinning it Round 0 of the audit on this PR returned `requirement questioned` with two 🟡 and two 🟢. All four taken. F1 (🟡, payload) — the guard pinned the SENTINEL; the requirement of record and the published bullet are about an EXIT CODE. cmd/civitai's TestCapWorded429WithRetryAfterExitsFiveNotTwo now composes cap-body + Retry-After -> exitCode() == 5, using the idiom the adjacent 429 guard already states: "pkg/civitai owns the sentinel; this owns the number a script reads from `$?`." Nothing composed those two inputs at the rc level before, so the seam was unowned for the one 429 case the contract marks 🔴. Carries the same instrument positive control (exitCode must be able to return 2). F2 (🟡, payload) — the precedence was described in three surfaces and argued in none, so a maintainer reading the bullet call it "the exact hazard the 2 reclassification exists to prevent" would reasonably invert it, and be blocked by a test asserting a contract without saying why. retry.go's 429 branch now argues it: a Retry-After header is a STRUCTURED signal the server sent deliberately; isDeepPagingCap is a three-phrase substring match over prose its own doc comment calls "deliberately narrow" and which any proxy or copy-edit can produce or destroy. Letting the heuristic outrank the structured signal trades this hazard for a worse one — a genuine throttle refused as a usage error, which a backoff loop never retries. It also names the four surfaces that move if the precedence ever changes, and corrects "there is no local guard on it": there are now two on the ORDERING; the REACHABILITY is the half that stays unguardable. F3 (🟢, scaffolding) — errors.Is(err, ErrRateLimited) is an INVARIANT guard and is now labelled one. Measured: it does not fire under the ordering mutant, since errkind.go attaches exactly one sentinel and the cap matcher still matches, so ErrRateLimited is unreachable by construction on this input. Kept, not counted as regression coverage. F4 (🟢, scaffolding) — dropped the restated 20-ok/1-FAIL matrix from the `pinnedBy` comment. It was a fourth in-tree copy, hardcoded a package count nothing checks, and RULES puts the measurement in the commit and the claim in the comment. 🔴 Correcting a claim in the previous commit message and the PR body: they say the mutant fails "all five of its own assertions incl. both `errors.Is` ones". The count 5 is right but the phrasing is wrong twice — the test has SIX assertions and THREE `errors.Is` ones, and the one that does not fire is F3's. Corrected publicly on the PR rather than silently. Matrix, re-measured on this tree: `make ci` rc=0, 21 ok / 0 FAIL; gofmt 0 of 487 files; golangci-lint 0 issues; `-race` clean on both guards. Under the message-before-header mutant: 19 ok packages and exactly TWO `--- FAIL`s, both new guards — the sentinel one on 5 of its 6 assertions, the rc one on "got 1 request(s)" and "exits 2, want 5 (exitNetwork)", with its CONTROL assertion not firing. Mutant written as a diff, anchor asserted to occur exactly once, retry.go restored from a `cp -a` copy and confirmed byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013JQ4B1VHew33RNX5Ck7ELS
🔴 Correction to the PR body, plus round 0's findings — all four taken (
|
make ci |
rc=0, 21 ok / 0 FAIL |
| gofmt | 0 flagged of 487 .go files |
golangci-lint |
0 issues (v2.13.2 local; CI's pinned v2.12.2 job is the authority and was green on cc24058) |
-race |
clean on both guards |
| mutant (message before header) | 19 ok packages, exactly TWO --- FAILs — both new guards. Sentinel guard: 5 of 6 assertions. rc guard: "got 1 request(s)" and "exits 2, want 5 (exitNetwork)", with its CONTROL assertion not firing |
Mutant written down as a diff, anchor asserted to occur exactly once, retry.go restored from a cp -a copy and confirmed byte-identical. A mid-battery timeout did leave the mutant in the tree once; it was caught by an explicit re-check before the commit, not by luck.
Overlap with #602
Both edit internal/cmd/exitcodes_claims_test.go. The auditor test-merged them (git merge-tree --write-tree, branching on the exit code, not a marker grep) → exit 0, merged tree 21 ok / 0 FAIL, hunks disjoint (#602 appends floor entries and prepends rows; this edits the code-6 row's pinnedBy). That measurement is against cc24058 + f02c2003 only — whoever merges second should re-run it, since my later commit af35fbf touches that file again.
Round-0 trial ledger for this PR: ran: 1 · changed the outcome: 1
It ran before the merge decision (the audit-pr-nudge.py routing working as intended) and changed what shipped: two payload findings, one of which closes the requirement of record.
Round 1 (the nine correctness axes) has not been run. These four fixes carry my own verification, not an auditor's.
…not just routed rank 22 — DONE. cli#601 merged (b727a83). The published header-before-message ordering is now pinned TWICE: the sentinel in pkg/civitai/retry_test.go and the exit code in cmd/civitai/read_error_stderr_test.go. Round 0 of the audit ran before the merge decision and found the first guard was one hop short — the requirement of record and the 🔴 bullet are about an `rc`, and nothing composed cap-body + Retry-After -> exitCode(). That is the durable finding: a classification test and an exit-code test are two claims. rank 9 — DONE, and closed in the terms it was stated in. The streak's end was INFERRED from a workflow_dispatch run; it is now OBSERVED in scheduled run 34848144324 (2026-09-14T13:15 UTC): drift=success, 0 failing steps and 15 EXECUTED, notify=success. The step count is the load-bearing half — a run that skipped its steps reports success too. rank 3 — no longer "unrouted". An agent is dispatched against civitai/civitai to fix the Meilisearch-side coercion and open a PR; the brief's constraint is recorded here because it is the easy mistake: a boundary String(...) cast is NOT the fix, since "0222" is already 222 by the time the read path sees it. New rank 25 carries the outcome check so the next session does not re-dispatch it. rank 26 — cli#602's merged-tree re-run is recorded as DELIBERATELY SKIPPED (operator's call), not absent. The mechanical check is posted on that PR. State now: main moved 7467c62 -> 426288f (#601, #596, #603). Flags that this repo has a SECOND live handoff doc maintained by a concurrent session, with its own ranked list — "rank 23 closes" in a commit subject may not mean this doc's. Four new lessons, all from this session's own mistakes: a timed-out mutation battery leaves the mutant in the tree (the restore line is the one that dies); a grep for "the old text is gone" cannot distinguish REMOVED from QUOTED-IN-THE- RETRACTION; `| head` ate grep's exit status again; and the merge drop-check has a premise — it reported 103 "dropped" lines that were this PR's own replacements, because main had not touched that path at all. Doc: 64,635 B of the 65,536 B ceiling. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013JQ4B1VHew33RNX5Ck7ELS
What
internal/cmd/exitcodes_doc.goand README publish a 🔴 bullet on exit code 6:Nothing in the repo guarded it. This adds the guard and corrects the claim row that said none was possible.
The gap, measured
Consult the message first — return terminal on cap wording, whatever the header — and the published bullet inverts from 5 to 2 while the full suite stays green.
retry_test.go's two 429 neighbours each hold one half of the input fixed:Retry-AfterTestRead429WithRetryAfterExhaustsGenericTestRead429WithoutRetryAfterIsTerminalWithCursorHintIt takes both at once, which nothing sent.
The guard
TestCapWorded429WithRetryAfterIsRetriedNotReclassified(pkg/civitai) serves a cap-worded body (isDeepPagingCapmatches) plusRetry-After: 0on every response, and asserts:readMaxAttempts(4) requests, not 1;errors.Isper AGENTS.md item 7, since the sentinels carry no visible text:ErrNetwork(exit 5), notErrBadRequest(exit 2), notErrRateLimited(exit 6);readError's. The cap wording still reaches the string viasnippet(raw), so the discriminating absences arerate limited (429)and--cursor—readError's own text, absent from the body.Verification matrix
make circ=0, 21 ok / 0 FAIL;-raceclean;gofmtclean--- FAIL— this test, failing on all five of its own assertions incl. botherrors.IsonesThe mutant was written down as a diff and applied from it; the anchor was asserted to match exactly once first;
retry.gowas backed up withcp -aand restored by copy (nevergit checkout --), andgit diffconfirms it is byte-identical to HEAD. That single-FAILreading confirms round 4's 21 ok / 0 FAIL from the other side: nothing that existed before could see the swap.Linter:
golangci-lint run ./...→ 0 issues, and it was validated with a negative control (a deliberate unused variable → rc=1, 1 issue). Scope caveat: that was v2.13.2 vianix-shell; CI pins v2.12.2, so the zero is a claim about 2.13.2, not about the pinned job.The
pinnedBycorrectionThe claim row read:
That conflated two claims. The reachability — whether the server ever attaches
Retry-Afterto a cap 429 — is genuinely unguardable and is exactly why the bullet is published. The CLI's own ordering is local, trivially observable, and was unguarded. As written the field told the next maintainer no guard was possible, which is the description-reads-as-coverage failure that table exists to stop.The published bullets are unchanged: "that assumption is vendored and has no local guard" refers to the server half, which is still true. No generated region is touched.
Residual, not fixed here
pinnedByis free prose —exitcodes_claims_test.goonly prints it in a failure message, so nothing checks that a named test exists. The sibling ledger inapp_newest_submission_test.godoes check this (testFuncBody(...)→ "the ledger names coverage that does not exist"), but several rows here name tests in other packages, so an existence check is a cross-package change and out of scope for this PR.Closes the rank-22 item in
claudedocs/handoff-external-issue-513-numeric-username.md(cli#591round-4 F2).