Fuzz nightly: fuzz_scheduler_solver OOMs (solve_milp >2GB RSS) on a small bounded task set — new on 2026-08-25

[avrabe]

Surfaced by the issue-hunt health check, not yet tracked.

Measurement

Fuzz (nightly) run 125 (2026-08-25, main @ 138a693 / v0.42.0) failed. The failure is isolated to one target — fuzz_scheduler_solver — and it is an OOM, not a panic or hang:

Error: Fuzz target exited with exit status: 71
Reproduce with:
  cargo fuzz run --target=x86_64-unknown-linux-gnu fuzz_scheduler_solver \
    fuzz/artifacts/fuzz_scheduler_solver/oom-c9dae275e8e855ad1dd29dcf09051fe27471730f

• The job uploaded a crash artifact (fuzz-artifacts-fuzz_scheduler_solver, artifact ID 9549504175, 270 bytes — the reproducer input is oom-c9dae275…), whereas the two passing targets (fuzz_aadl_parse, fuzz_codegen_roundtrip) skipped that upload step. So a concrete reproducer exists.
• New this run: the two prior nightlies — run 124 (08-24) and run 123 (08-23) — were green across all three targets. main is unchanged since v0.42.0, so this is a newly-explored corpus input rather than a code regression.

Why this looks like a real bug, not a harness artifact

fuzz/fuzz_targets/fuzz_scheduler_solver.rs bounds the input explicitly and small: n_tasks = len.min(8), n_procs = len.min(4), with period/wcet/deadline as u16 and memory_bytes as u32. Its stated contract:

The call must return Ok or Err — it must never panic, and the outer libfuzzer -timeout backstop catches hangs.

That contract covers panics and hangs but not memory. Driving >2GB RSS (libFuzzer's default rss_limit_mb) through spar_solver::milp::solve_milp on a ≤8-task / ≤4-processor set points at an unbounded/blow-up allocation in the MILP path — the search space is tiny by construction, so this is not "the fuzzer fed a huge input."

Traceability the target names: REQ-SOLVER-001, REQ-SOLVER-003, REQ-SOLVER-005.

Triage this needs (not yet done here)

I have not reproduced or minimized the input this run (no nightly cargo-fuzz toolchain to hand), so the two hypotheses are still open:

1. Real defect in solve_milp — an input shape (e.g. period/wcet = 0, a degenerate deadline, or a bind_to that forces a pathological formulation) makes the solver allocate without bound. Fix in the solver.
2. Harness gap — the MILP formulation is legitimately memory-heavy for some in-bounds shape and the target needs an explicit memory/complexity bound (or a tighter -rss_limit_mb matched to the intended budget).

Next step for whoever picks this up: cargo fuzz tmin the oom-c9dae275… artifact to get the minimal task set, then decide (1) vs (2). The reproducer is in the run's artifacts (30-day retention).

Not claimed

That this is a memory-safety issue or exploitable — it is a resource-exhaustion (OOM) finding. Also not claimed that fuzz_scheduler_solver would otherwise be green long-term; only that this specific OOM is new as of 2026-08-25 and has a saved reproducer.

[avrabe]

Triage: a single in-domain input cannot drive solve_milp to the OOM ceiling — this looks run-level, not a solver defect

spar issue-hunt run picked this up. I could not run the real libFuzzer harness (no nightly/cargo-fuzz in this session) or fetch the exact oom-c9dae275… bytes, so I reconstructed the harness's decoded input domain directly and searched it hard against the solver. Findings below; nothing is claimed beyond what I measured.

Two corrections to the premise first

• The nightly does not use the default 2048 MB limit. .github/workflows/fuzz-nightly.yml runs this leg with -max_total_time=3600 -timeout=10 -rss_limit_mb=4096. So run 125 crossed ~4 GB RSS, not 2 GB.
• The reachable single-input domain is fully bounded and small. With -max_len=128 and the harness's tasks.len().min(8) / processors.len().min(4), the workflow's own comment notes ~120 bytes saturate the reachable domain. And solve_milp only reads num_procs + per-thread period_ps / wcet_ps / current_binding / name — deadline_ps, priority, and memory_bytes are inert to it. So the behaviourally-relevant space is exactly: n_procs ∈ 1..=4, n_tasks ∈ 1..=8, period,wcet ∈ 0..=65535, optional pre-binding.

What I measured (built against the CI-pinned highs 2.0.0 / good_lp 1.15.0)

Direct-construction search over that whole domain — random full-range u16 period/wcet, random n_tasks/n_procs, random current_binding — one build+solve per iteration, VmRSS sampled after each solve, each solve timed:

run	backend	solves	peak VmRSS	solves ≥ 400 ms
direct search	highs 2.0.0 (CI-pinned)	335,647	11.5 MB	0
direct search	highs 2.4.0 (latest)	~335,000	12.8 MB	0

Per-call leak probes (build+solve in a tight loop, pinned 2.0.0):

path	iterations	VmRSS behaviour
feasible (8 tasks / 4 procs)	8,000	plateaus by iter ~2,000, flat after (~11 MB)
infeasible (all util > 1)	200,000	flat at 8.5 MB start→finish

Plus hand-picked extremes (max-util wcet=65535/period=1, wide coefficient range, fully symmetric, near-capacity, coprime periods): all solve in single-digit ms with a few-MB footprint and the correct feasible/infeasible verdict.

Combined ~671k solves across both backend versions; the ceiling was ~13 MB — roughly 300× below the 4 GB rss_limit. No per-call leak (RSS plateaus, it does not climb).

Conclusion

• Hypothesis (1), a solver defect where some in-bounds shape allocates without bound — strongly excluded. No single reachable input gets solve_milp anywhere near 4 GB, and there is no per-call leak that could accumulate.
• Because the input domain is capped so small, ~4 GB is only reachable cumulatively over the 1-hour session — libFuzzer's own process RSS (coverage counters + the corpus restored from cache and grown during the run + mutation/bookkeeping), not a per-input blow-up in the solver. The saved oom-c9dae275… is simply the input in flight when the ceiling was crossed. This matches the two facts already in the issue: runs 123/124 were green on byte-identical code, and the reporter never reproduced it standalone.

Recommended next step (needs the toolchain + artifact I don't have here)

1. Replay the artifact standalone: cargo +nightly fuzz run --target x86_64-unknown-linux-gnu fuzz_scheduler_solver fuzz/artifacts/fuzz_scheduler_solver/oom-c9dae275…. Prediction: exit 0, well under 15 MB.
2. If it does not reproduce standalone → it's a run-level RSS event, not a bug in solve_milp. Levers that actually bound cumulative growth (a tighter per-target -rss_limit_mb would not, since the growth is the session's, not the input's): periodic corpus trim, or -runs=<N> batching / a fresh-exec cadence so libFuzzer re-execs and resets process RSS. Given the 97-run history where this is the only failing leg, "benign session-RSS flake" is a defensible call too.
3. If it does reproduce standalone → a rare shape escaped 671k samples; cargo fuzz tmin it and share the minimized bytes and I'll fix solve_milp against it.

I did not ship a code change: the evidence points away from a solver defect, and adding a guard to code that never blows up would be a vacuous "fix." Leaving this as a diagnostic + handoff rather than guessing.

---

Generated by Claude Code