mutants-nightly has produced zero data in 87 runs: every leg dies in ~30s on a missing target/ parent, and the step exits 0 so it can never go red

[FerroxLabs]

The one instrument in this workspace that could mechanically detect a test which runs but cannot reach the defect it names has never emitted a single data point, and cannot report that it hasn't.

Measured

gh run list --workflow mutants-nightly.yml --limit 100 → 87 runs, conclusion success on all 87. Oldest 27133754825 (2026-06-08, the workflow's inception), newest 33599619308 (2026-09-02).

Newest run, all five legs, 27–38 seconds each, every one success:

mutants / wcore-config   success  06:36:45 → 06:37:13
mutants / wcore-agent    success  06:36:46 → 06:37:23
mutants / wcore-providers success 06:36:45 → 06:37:20
mutants / wcore-cli      success  06:36:46 → 06:37:18
mutants / wcore-cron     success  06:36:46 → 06:37:22

and every one of them died the same way:

Error: create output parent directory "target/mutants-wcore-config"
    No such file or directory (os error 2)

There is no cargo build and no mkdir -p target before cargo mutants --output target/..., so on a fresh runner the parent does not exist. cargo mutants exits non-zero, the step swallows it, and the job concludes success.

A real leg takes minutes and reports e.g. 54 mutants tested in 2m: 9 missed, 28 caught, 17 unviable. Thirty seconds is not a fast run; it is no run.

Why this is worth its own ticket

A gate that cannot fail is worth exactly as much as one that cannot pass. This one has read green 87 times, and anything built on top of it would be graded on a shrug. It was found while trying to answer "is mutation coverage already guarding the vacuous-by-data test class?" — the answer is no, and it never has been.

The two known-positive controls matter here: the only run that ever filed issues is 27133754825, and those issues (#1, #3, #5) read **Catch rate**: N/A%, **Summary**: no summary, none under Top surviving mutants. So even the run that produced output produced no data.

Acceptance

• c1 RED ARM — the diagnosis reproduces before anything else counts. On a checkout with no target/ present, cargo mutants -p wcore-cron --no-shuffle --timeout 90 --output target/mutants-wcore-cron exits non-zero in under 60 seconds printing create output parent directory / No such file or directory (os error 2).
• c2 THE FIX PRODUCES REAL DATA, read from the artifact, not the job log. After the fix, one workflow_dispatch run has a mutants / wcore-cron leg whose uploaded artifact contains a line matching ^[0-9]+ mutants tested in .
• c3 THE LEG CAN NOW CONCLUDE failure. With the fix in place, a run in which cargo-mutants produces no summary line makes the job conclude failure. Demonstrated, not asserted — this is the half that stops it being unfalsifiable, and it is the half that has been missing for 87 runs.
• c4 THE "no data ever" CLAIM CARRIES ITS CONTROL. Any scan asserting a log contains no summary line is run in the same invocation against a copy of that log with a real summary line inserted, and both arms reported: real logs → 0 hits, seeded copy → exactly 1 hit.
• c5 SCOPE, RECORDED SO THE FIX IS NOT OVER-CLAIMED. The closing comment records the wcore-cron summary line from c2 verbatim as the first mutation-coverage baseline this repository has ever measured, and states that fixing the harness does not by itself establish coverage for any other crate.

Found while adjudicating whether the "vacuous-by-data test" class needed a ticket of its own. It does not — journal_envelope_roundtrip.rs already pins the float case with a 2048-value sweep whose assert names a dropped float_roundtrip feature. This is the real gap underneath that question.

[FerroxLabs]

Closing record — scope, so the fix is not over-claimed

The harness is fixed and it now produces data. The first workflow_dispatch run in
this workflow's history (run 34438249721, head fdf4b1e1c, the 94 runs before it all
event=schedule) has a mutants / wcore-cron leg that concluded success and uploaded
an artifact. Read from that artifact — mutants-log-wcore-cron, id 10138518656, at
.blackboard/E2E-MUTATION-BASELINE/wcore-cron.log:134 — the summary line, verbatim:

339 mutants tested in 28m: 64 missed, 196 caught, 77 unviable, 2 timeouts

Exactly one line in that log matches ^[0-9]+ mutants tested in , so the match is not an
artefact of a loose pattern. The per-outcome files in the same artifact hold 196, 64 and 2
lines respectively, and 196 + 64 + 2 + 77 = 339 — the summary and the outcome files agree,
so neither is a stale copy of the other.

This is the first mutation-coverage baseline this repository has ever measured.

Catch rate, both denominators, because they differ and the choice is not neutral:

• 74.8% — 196 caught / 262 (caught + missed + timeouts). This is the figure to quote.
  A timeout is not a kill: the test suite did not demonstrate it detects that mutant, it
  demonstrated it hangs.
• 75.4% — 196 / 260, excluding the 2 timeouts. This is the friendlier number and it is the
  one to distrust, for the reason above.

Unviable mutants are excluded from both: an unviable mutant does not compile, so no test
could have caught it and counting it against the suite would understate coverage.

What this does NOT establish, stated plainly:

1. It is one crate. wcore-agent, wcore-config, wcore-providers and wcore-cli
   were still in progress when this was written. Fixing the harness does not by itself
   establish mutation coverage for any other crate, and this comment should not be cited
   as if it did. That is wayland-core#451's subject, not this ticket's.
2. A produced baseline is not coverage. 64 mutants survive wcore-cron's test suite.
   Those survivors are real findings about those tests, they are what the instrument is
   for, they were filed as wayland-core#449, and they need their own triage. Recording
   them here would let a harness fix be mistaken for a coverage result.
3. The 64-row missed set is byte-identical to the earlier run's. That is expected and
   is not a finding about test quality: crates/wcore-cron/tests/store_hardening.rs and
   trigger_arithmetic.rs — the 14 tests written against those survivors — are absent from
   fdf4b1e1c (git ls-tree -r fdf4b1e1c -- crates/wcore-cron/tests/ lists five files,
   neither of them). This run graded a tree that does not contain them, so it says nothing
   in either direction about whether they kill. #449 still owes a mutation run on a tree
   that has them.

Baseline the leg measured for itself, from the same artifact:
{"crate": "wcore-cron", "measured": true, "build_seconds": 116, "test_seconds": 4, "timeout_multiplier": 5, "minimum_test_timeout": 90, "runner_os": "macOS"}