fix(verify): the runner shall not report success having executed zero commands (#403)

[avrabe]

Closes the remaining live half of #403 — the issue's reported title.

The defect

tools/run_verification.py skips an artifact with no fields.steps[].run and counts the skip toward exit 0. Its terminal decision was return 0 if failed_count == 0 else 1, so a run that matched artifacts and executed zero commands returned 0 — and Verification Gate (rivet-driven) is a REQUIRED context, so it reported green having verified nothing.

Measured on the live plane:

$ tools/run_verification.py --filter '(and (= type "feature") (has-field "test-name"))'
passed: 0   failed: 0   skipped: 64
EXIT=0

The sibling if not ids: return 1 had already closed "the filter matched nothing"; "the filter matched, and nothing ran" sat open one branch below.

Scope — the runner half, ratchet-bounded

REQ-GUARD-GATE-EVIDENCE-003 (check_evidence_steps.py) already guards the artifact plane via a ratchet that deliberately does not hard-fail the skip backlog (failing all 85 claiming-but-stepless artifacts at once is "a gate that gets disabled"). It named the runner's exit logic as a separate, still-open question. This PR is that half, and it stays compatible with the ratchet:

• A pure decide_exit(total, passed, failed, skipped) reds a run where nothing executed (every match skipped) and names the cause.
• A mixed run where something ran and none failed still passes — so it does not re-introduce the "fail the whole backlog" posture the ratchet avoids. It closes only the hole where nothing at all was verified.

REQ-GUARD-GATE-EVIDENCE-003 (tagged human-scoped) is untouched.

What's here

• run_verification.py: decide_exit() + --self-test (no rivet, no shell). The load-bearing case all skipped -> exit 1 pins both the exit code and the verified nothing reason text. Mutating the guard to never fire reds the self-test.
• ci.yml: Verification-runner guardrail self-test step in the rivet-validate job, beside its siblings.
• check_self_test_potency.py: rv-zero-executed-lenient mutant — CAUGHT (5/5 declared mutants caught).
• REQ-GUARD-GATE-EVIDENCE-005 + TEST-GUARD-EVIDENCE-RUNNER (release v0.43.0), tracing to -003.

Blast radius

The Verification Gate default filter (v093/v0100) selects 20 type: feature artifacts, all with runnable steps, so the default gate cannot red on this change. Evidence-step floor unchanged (78).

Verification (local)

• run_verification.py --self-test → 9/9; proven non-vacuous by the mutation above.
• check_self_test_potency.py → 5/5 CAUGHT; --self-test 12/12.
• check_evidence_steps.py --max-stepless 78 → green (floor held).
• All sibling stdlib guard self-tests green; YAML parses; no duplicate ids; 0 dangling cross-refs.
• rivet validate runs in CI (no local rivet, as expected).
• An independent clean-room subagent re-derived all of the above and returned SHIP.

Not claimed

• That a present step is the right step or that it passes — that remains check_verification_filters.py's and the per-step cargo exit's question.
• That the skip backlog is failed — by design it is not; bounding it stays -003's job.

🤖 Generated with Claude Code

---

Generated by Claude Code

[github-actions]

Rivet verification gate

✅ 20/20 passed

	count
Passed	20
Failed	0
Skipped (no steps)	0

Filter: (and (= type "feature") (or (has-tag "v093") (has-tag "v0100")))

Failed artifacts

(none)

_{Updated automatically by tools/post_verification_comment.py. Source of truth: artifacts/verification.yaml.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[avrabe]

Clippy came back red — diagnosed as a transient runner flake, not this PR's

The Clippy check failed on the first run (job 98065186444): the guardrail reported DIRTY Cargo.toml (exit 101) on the root workspace. I could not reproduce it, and it is not attributable to this PR:

This diff touches zero Rust. The change is tools/*.py (Python), artifacts/*.yaml, and one added step in the unrelated rivet-validate job. The Rust tree on this branch is byte-identical to main @ v0.42.0 (branched from 138a693, no Rust files modified), so clippy's verdict on the Rust code is by construction identical to main.

It does not reproduce under the exact CI conditions. Locally I ran, on this branch's checkout:

• cargo clippy --workspace --all-targets -- -D warnings on the CI toolchain stable 1.98.0 (clippy 0.1.98, the same dtolnay/rust-toolchain@stable resolves to) → clean, exit 0.
• The exact guardrail, tools/check_clippy_workspaces.py, with the exact CI env (RUSTFLAGS=-D warnings, CARGO_INCREMENTAL=0) → 4 workspace(s) checked, 4 lint-clean, exit 0.
• Also clean on the older stable 1.94.1.

The failure signature is transient, not a lint. The job log shows No cache found, an sccache orphan process terminated at teardown (Terminate orphan process: pid (2204632) (sccache)), and the run exiting 101 mid-compile with no reproducible lint on any toolchain — consistent with an sccache/runner-state hiccup on the self-hosted runner, not a clippy finding.

I could not re-run it myself: rerun-failed-jobs returns HTTP 403 for this integration. I'm deliberately not widening this focused Python/YAML PR with an unrelated Rust or toolchain-pinning change to chase a flake that reproduces nowhere.

Ask: a maintainer re-running the Clippy job (or any subsequent push) should clear it. I'll keep watching this PR until it's green, and re-check on a scheduled cadence; if a re-run reproduces the failure, it's real and I'll dig in rather than call it a flake again.

The remaining required checks (Mutation ~2h, Lean ~90m, etc.) are still running.

---

Generated by Claude Code

[avrabe]

Correction: this is NOT a flake — it's a real, repo-wide nightly-clippy regression. My previous comment was wrong.

I have to correct my earlier comment. I diagnosed the Clippy failure against stable 1.98.0 and called it a transient flake. That was a mistake: the Clippy job runs on dtolnay/rust-toolchain@nightly, not stable (step 3 of the job is Run dtolnay/rust-toolchain@nightly). On stable the tree is clean, which is exactly why my stable repro misled me.

Reproduced on the actual toolchain — rustc 1.100.0-nightly (787af2b8c, 2026-08-25) / clippy 0.1.100 — with the guardrail's own env (RUSTFLAGS=-D warnings): it fails deterministically, 4 workspace(s) checked, 3 lint-clean, root DIRTY (exit 101).

Root cause: clippy::needless_range_loop, newly stricter on nightly

Three sites, all -D clippy::needless_range_loop (implied by -D warnings):

error: the loop variable `j` is used to index `matrix.matrix[i]`
  --> crates/spar-cli/src/main.rs:1337
error: the loop variable `i` is used to index `m.matrix`
  --> crates/spar-analysis/src/mode_reachability.rs:770
error: the loop variable `j` is used to index `m.matrix[i]`
  --> crates/spar-analysis/src/mode_reachability.rs:771   (a #[test])

These lints do not fire on stable 1.98.0; nightly clippy 0.1.100 tightened the heuristic.

Why it is not this PR's

This PR's diff is tools/*.py + artifacts/*.yaml + one added step in the rivet-validate job — zero Rust. crates/spar-cli/src/main.rs and crates/spar-analysis/src/mode_reachability.rs are byte-identical to main @ v0.42.0. So main (and every other open PR) fails this identically on today's nightly — it is a repo-wide nightly-clippy drift, surfaced here only because @nightly moved since main's last green run (2026-08-22). Everything else on this PR is green, including the new Verification-runner guardrail self-test, Rivet validate, Test, Proptest, Fuzz smoke, and Coverage.

Proposed patch (not applied here — keeping this PR focused; no fix exists to port yet)

Per the repo's focus discipline I'm not bundling an unrelated Rust lint fix into this verification-gate PR, and no base fix exists to port. Options, for whoever picks this up (a separate focused change):

1. Refactor to iterators (the lint's own suggestion) — e.g. crates/spar-cli/src/main.rs:1337, iterate matrix.modes.iter().enumerate() for the inner loop instead of for j in 0..matrix.modes.len(); similarly .enumerate() the mode_reachability.rs test loops.
2. Localised allow — #[allow(clippy::needless_range_loop)] at the three sites (the mode_reachability.rs one is a 0..3 test loop where the range is arguably clearer than an iterator).
3. Pin the Clippy job's nightly — toolchain: nightly-2026-08-22 (or move the Clippy gate to @stable), so daily nightly clippy churn stops gating merges.

This blocks #440 from going green until the base is fixed (Clippy is a required check). I'll keep watching and re-check on a cadence; once the base is clippy-clean on nightly again, this PR should go green with no change of its own.

Apologies for the incorrect first read — testing the wrong toolchain produced a wrong conclusion, and I'd rather correct it plainly than let it stand.

---

Generated by Claude Code

[avrabe]

Standing down this run — the CI red here is a base-branch clippy break, not this PR.

State of the head (eb64446 merged onto main@138a693): of the 17 CI jobs, 16 are green (Format, Test, Miri, Proptest, Fuzz smoke, Bench compile smoke, Cargo Deny, Security Audit, Supply Chain, Rivet validate, Code Coverage, Mutation Testing, Kani, Bazel, plus Verification Gate and Lean Proofs). The only red job is Clippy, and it fails on the root workspace Cargo.toml (DIRTY Cargo.toml (exit 101)); the three sub-workspaces (codegen-exec-oracle, codegen-kiln-oracle, fuzz) are all clean.

This PR's diff is five files — ci.yml, artifacts/requirements.yaml, artifacts/verification.yaml, tools/check_self_test_potency.py, tools/run_verification.py — and touches zero Rust. The Clippy job checks out pull/440/merge, so the failing root-workspace crates come entirely from the base; nothing in this diff can produce a clippy lint.

Root cause is the toolchain: the job installs nightly 1.100.0 (787af2b8c, 2026-08-25) and a lint in that fresh nightly reds the existing spar-* crates under -D warnings. The same wall reds #442 and #443 (both run the Rust suite); #441 is artifacts-only, so its changed-paths classifier skips the Rust jobs and it stays green — which is exactly why only the code-touching PRs are red.

Reproduce on the base: cargo +nightly clippy --manifest-path Cargo.toml --all-targets -- -D warnings (nightly ≥ 2026-08-25).

I'm not widening this no-Rust PR with an unrelated, repo-wide clippy fix. The base needs a fix (or a nightly-clippy pin) first; once that lands, this PR should go green on a re-run with no change here. Not merging.

Autonomous issue-hunt run: 4 PRs open (#440–#443), so the overlap guard held and no new work was started this run.

---

Generated by Claude Code

[avrabe]

CI red on this PR is a repo-wide nightly-clippy drift, not this change. Standing down rather than widening a Python-only PR — this needs a maintainer decision (see below).

What is failing

Only the required Clippy check is red. Every other required check on this HEAD is success, including the ~2h Mutation Testing, Lean proof typecheck, Miri, Proptest, Coverage, Rivet validate, etc.

The Clippy job uses dtolnay/rust-toolchain@nightly (unpinned). The run resolved nightly to 1.100.0-nightly (787af2b8c 2026-08-25), which promotes clippy::needless_range_loop into the lint set caught by -D warnings. That lint fires on pre-existing Rust identical to main — this PR's diff is .github/workflows/ci.yml + artifacts/*.yaml + two tools/*.py files, zero Rust — so the failure is not caused by this change.

Evidence it is toolchain drift and repo-wide, not this PR:

• cargo clippy --workspace --all-targets -- -D warnings on stable 1.94.1 is clean (exit 0).
• Reproduced on nightly locally (1.100.0-nightly (5db7f4be8 2026-09-01)): the same 3 errors, exit 101.
• main's last CI (2026-08-22, older nightly) was green on the same Rust; a re-run on current nightly would red identically.
• PR feat(cli): one table drives both dispatch and --help #443 (and, by the same mechanism, the other open PRs) fail only Clippy too.

The 3 hits (all pre-existing, none touched here)

crates/spar-cli/src/main.rs:1337        for j in 0..matrix.modes.len()   -> matrix.matrix[i][j]
crates/spar-analysis/src/mode_reachability.rs:770   for i in 0..3        -> m.matrix[i][j]   (#[test])
crates/spar-analysis/src/mode_reachability.rs:771   for j in 0..3        -> m.matrix[i][j]   (#[test])

All three are symmetric 2D-matrix accesses that use both indices, so clippy's .iter().enumerate() rewrite is awkward; #[allow(clippy::needless_range_loop)] on each loop (or a small idiomatic rewrite) is the pragmatic fix.

Why I did not just fix it here

• This is a Python-runner fix (Verification Gate is green having executed zero commands: run_verification.py counts 'skipped' as pass #403); adding unrelated Rust lint edits would break its scope and its clean-room ledger.
• The overlap guard forbids me opening a separate fix PR while other PRs are open, and I will not touch PRs that are not mine.
• The real choice is a design decision for the maintainer: patch the 3 loops, or pin the nightly toolchain in ci.yml (e.g. dtolnay/rust-toolchain@nightly with a dated toolchain: such as nightly-2026-08-01) so unattended nightly clippy churn stops silently blocking every PR. Pinning is the more durable fix — otherwise the next new nightly lint reds all PRs again.

This blocks all four currently-open PRs (#440, #441, #442, #443). Once a fix or pin lands on main, this PR only needs a rebase/merge of the base to go green — nothing else on it is outstanding.

---

Generated by Claude Code