Workstream B Stage 1: a closed diffusion-and-binding benchmark on frozen MCS-0 geometry - #33
Workstream B Stage 1: a closed diffusion-and-binding benchmark on frozen MCS-0 geometry#33aurascoper wants to merge 1 commit into
Conversation
…zen MCS-0 geometry A numerical benchmark and nothing else. It integrates a dissolved and a bound pool on the labels and mask of one frozen snapshot and asks whether the implemented scheme conserves what it should, loses only what its ledger books, and converges at the order it claims. No chemistry is asserted, no molecule, species or isotope is named, and the thorium model stays deferred to a separate specification packet. The timestep restriction is derived from this stencil, not imported. In 3-D the diffusion bound is h2/(6D), not the 1-D h2/(2D): at h = D = 1 and dt = 0.25, which the 1-D bound admits, a unit centre with six zero neighbours lands at -0.5. The suite asserts that value and asserts that step! refuses the step producing it. Reaction and decay are unsplit, so they enter the same restriction, and each of the three terms is pinned by its own assertion rather than by one combined number. The rate is re-checked every step because max(c) and max(B-b) both move. The Laplacian is in flux form, iterating faces rather than sites, because 6,766 of this geometry's 50,200 interior sites have a ragged boundary and a site-centred stencil leaks at exactly those. Every ledger term is accumulated from the term the step computes, never by differencing the fields, so the closure identity has two independent sides. The Laplacian's floating-point residue is reported separately and is deliberately not a term in that identity: folding it in would let a real boundary leak close the books. Twelve controls, runnable, exiting non-zero on an unexpected verdict. K1 is the matched- total-capacity comparison: integrals agree to 2.6e-15 at every comparison time and the bound inventory differs by up to 1.89%, with the two runs cross-checked structurally since a difference in the bound pool must be mirrored exactly in the dissolved pool. K8 is the deliberately omitted transfer and FIRES is its correct verdict: closure_residual excludes the dropped accumulator on purpose, because a ledger that books the material a bug destroys cannot report the bug. It comes out at exactly minus the dropped amount, to 6e-11 on an inventory of 5e4. Red before green, on three counts. The capacity guard in make_state was checking b0 <= B at every interior site rather than where the pool is placed, so any positive b0 was refused; four assertions caught it. The first fixture written for the degenerate-modulator guard was itself degenerate, a solid 2x2x2 block in which every site has exactly three occupied neighbours, and the guard fired on it. And mutation_controls.jl plants seven single-line defects in a scratch copy of the module: all seven are caught, the unmutated baseline is green, and each patch is verified to have applied exactly once before its suite runs, because a patch that quietly matched nothing is indistinguishable in a report from a defect the suite missed. Chain of custody has one pin. run_manifest.json is hashed against it, the snapshot against the entry the manifest carries for it, and the snapshot's own mcs, run_id, label_state_hash and mask_sha256 against the configuration. The 101-frame authoritative trajectory is not regenerated; the root Project.toml and both protected CSVs are byte-identical to HEAD. The diagnostic carries its own Project.toml: the root project lists AMDGPU, which cannot instantiate on darwin, so a benchmark sharing it could not run on the machine it was written on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01V1GLLCekStea7UHjRF51aw
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fc0b26a662
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| (led.min_dissolved >= 0 && led.min_bound >= 0 && overshoot_final <= 0) ? | ||
| "PASS" : "FAIL", |
There was a problem hiding this comment.
Fail K6 on recorded transient overshoot
The overflow loop clamps b back to B after every step, so overshoot_final <= 0 is guaranteed even if the Euler update exceeded capacity during the run. In that scenario the nonnegative pools let K6 report PASS, although led.max_overshoot records the violation and the control explicitly asks whether the bound pool ever exceeded capacity; include that recorded maximum in the verdict.
Useful? React with 👍 / 👎.
| p = cfg["params"] | ||
| cfg, Params(p["D_c"], p["lambda"], p["k_on"], p["k_off"], | ||
| p["B0"], p["dB"], p["K"], p["n"], p["q_ext"]) |
There was a problem hiding this comment.
Validate parameter domains before constructing Params
A custom --config can currently supply negative transport or kinetic constants and still produce successful-looking receipts. For example, lambda = -0.01 makes both pools grow, records negative values under decayed_from_*, and reduces the computed stability rate, while the pure-decay and closure controls can still pass because they use the same invalid sign. Reject negative D_c, lambda, k_on, and k_off (and enforce the declared capacity/Hill domains) before constructing Params.
Useful? React with 👍 / 👎.
| function fresh_destination(dir::AbstractString) | ||
| refuse_existing(dir) | ||
| mkpath(dir) |
There was a problem hiding this comment.
Create the output destination atomically
If another process creates dir after refuse_existing(dir) but before mkpath(dir), mkpath accepts the now-existing directory and the subsequent open(..., "w") calls can overwrite its files. This defeats the explicitly documented second-check protection during a long run; the final leaf directory must be created with an operation that fails atomically when it already exists.
Useful? React with 👍 / 👎.
| "parent" => Dict("snapshot" => cfg["parent_snapshot"], "snapshot_sha256" => snap_sha, | ||
| "manifest_sha256" => cfg["parent_manifest_sha256"], | ||
| "run_id" => geo.run_id, "mcs" => geo.mcs), | ||
| "expected_red" => ["K8 the same transfer, deliberately omitted"], |
There was a problem hiding this comment.
Declare K12 in the expected-red metadata
The script identifies both K8 and K12 as expected-red checks, and a K12 DID-NOT-FIRE causes a nonzero exit, but the emitted receipt declares only K8 in expected_red. Consumers relying on this machine-readable field will therefore treat K12's intentional FIRES result as undocumented or unexpected even though it is required for a successful control run.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
🟡 Changes recommended
Moderate issues remain in dependency setup, configuration validation/provenance, control correctness, and mutation receipt handling.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a standalone Julia diffusion-and-binding benchmark on frozen MCS-0 geometry.
Changes:
- Implements solver, transport, binding, decay, ledgers, and timestep checks.
- Adds numerical, control, and mutation-testing suites.
- Includes configuration, receipts, trajectory data, documentation, and dependencies.
File summaries
| File | Description |
|---|---|
diagnostics/binding_benchmark/timeseries.csv |
Shipped benchmark trajectory |
diagnostics/binding_benchmark/test_numerics.jl |
Data-free numerical tests |
diagnostics/binding_benchmark/setup.jl |
Configuration and provenance validation |
diagnostics/binding_benchmark/run.jl |
Production benchmark runner |
diagnostics/binding_benchmark/README.md |
Documentation and reproduction instructions |
diagnostics/binding_benchmark/Project.toml |
Dedicated Julia environment |
diagnostics/binding_benchmark/mutation_verification.json |
Mutation-test results |
diagnostics/binding_benchmark/mutation_controls.jl |
Mutation-testing harness |
diagnostics/binding_benchmark/controls.jl |
K1–K12 validation controls |
diagnostics/binding_benchmark/control_verification.json |
Control results |
diagnostics/binding_benchmark/BindingBenchmark.jl |
Core benchmark implementation |
diagnostics/binding_benchmark/benchmark.toml |
Benchmark configuration |
diagnostics/binding_benchmark/benchmark_receipt.json |
Production-run receipt |
Review details
Suppressed comments (8)
diagnostics/binding_benchmark/Project.toml:17
- This standalone environment declares only version ranges and ships no
Manifest.toml; the documented commands merely select the project and do not instantiate it. A clean depot therefore cannot run without an extra setup step, and different dependency resolutions are not pinned for a numerical benchmark whose receipts record only the Julia version. Commit the lockfile or document and capture a pinned instantiation.
[compat]
HDF5 = "0.17"
JSON3 = "1"
julia = "1.12"
diagnostics/binding_benchmark/README.md:22
- These reproduction commands assume the diagnostic dependencies are already installed.
--projectselects the environment but does not instantiate it, so on a clean checkout the first command fails atusing HDF5(and the data-dependent commands also need JSON3); add an explicit instantiate step before the commands.
J="julia --project=diagnostics/binding_benchmark"
$J diagnostics/binding_benchmark/test_numerics.jl # 81 assertions, no data
diagnostics/binding_benchmark/README.md:29
mutation_controls.jldoes take one argument (<out_dir>), as shown in the invocation immediately above; onlytest_numerics.jlis argument-free. Clarify this statement so the documented reproduction instructions are not self-contradictory.
`test_numerics.jl` and `mutation_controls.jl` take no data path, no argument and no
environment variable, so their assertions cannot be skipped by a missing bundle. That is
diagnostics/binding_benchmark/README.md:140
- The shipped
stability_marginis not 0.37 at every recorded time:timeseries.csvdecreases from 0.3634 at t=2.5 to 0.3527 at t=50. Only the initial margin is 0.37, so this sentence contradicts the receipt and its own statement that both maxima decrease.
configuration the initial rate is 0.74, the bound is 1.3514, the declared step is 0.5, and
the margin is 0.37 at every recorded time (both maxima start at their largest and decrease
from there).
diagnostics/binding_benchmark/controls.jl:187
- K6 asks whether the bound pool ever exceeds capacity, but
overshoot_finalonly checks the final state.step!already accumulatesled.max_overshootbefore the release, so a transient overflow that is released before T would make this control pass. Use the all-time ledger value for this verdict.
(led.min_dissolved >= 0 && led.min_bound >= 0 && overshoot_final <= 0) ?
"PASS" : "FAIL",
diagnostics/binding_benchmark/mutation_controls.jl:112
- The initial
ispath(out)check has the same TOCTOU gap: another process can create the destination during the baseline/mutation runs, after whichmkpath(out)silently accepts it and writes the receipt there. Use exclusive final-directory creation instead ofmkpathfor the output commit.
mkpath(out)
diagnostics/binding_benchmark/setup.jl:52
- Because both CLI entry points accept
--config, this parser can accept negative or non-finite diffusion/loss/reaction/capacity parameters and negative initial pools. For example, a negativelambdais treated as growth and can still pass the timestep check, so the run writes a receipt for a scheme that violates the documented positivity/loss assumptions. Validate the allowed parameter and initial-state domains before constructingParams.
require(cfg["dt"] > 0 && cfg["total_time"] > 0, "dt and total_time must be positive")
require(cfg["record_every"] isa Integer && cfg["record_every"] >= 1,
"record_every must be a positive integer")
p = cfg["params"]
cfg, Params(p["D_c"], p["lambda"], p["k_on"], p["k_off"],
p["B0"], p["dB"], p["K"], p["n"], p["q_ext"])
diagnostics/binding_benchmark/setup.jl:115
- This is not an atomic second check: another process can create
dirafterrefuse_existing(dir)returns and beforemkpath(dir), andmkpathwill succeed. The subsequent writes can then mix with or overwrite a pre-existing output despite the documented refusal. Use an atomic creation of the final directory (after preparing its parent) or an exclusive sentinel.
function fresh_destination(dir::AbstractString)
refuse_existing(dir)
mkpath(dir)
- Files reviewed: 13/13 changed files
- Comments generated: 5
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| A = neighbour_fraction(geo.occupied) | ||
| B = capacity_field(geo, p; A = A) | ||
| dt = Float64(cfg["dt"]); total = Float64(cfg["total_time"]) | ||
| nsteps = round(Int, total / dt); every = max(1, nsteps ÷ 10) |
| "parent" => Dict("snapshot" => cfg["parent_snapshot"], "snapshot_sha256" => snap_sha, | ||
| "manifest_sha256" => cfg["parent_manifest_sha256"], | ||
| "run_id" => geo.run_id, "mcs" => geo.mcs), | ||
| "expected_red" => ["K8 the same transfer, deliberately omitted"], |
| failing = [strip(l) for l in eachline(log) | ||
| if occursin("Test Failed", l) || occursin("Error During Test", l)] |
| "Hashes of the code and configuration that produced a receipt, recorded beside it." | ||
| function code_hashes() | ||
| Dict(f => sha256_file(joinpath(@__DIR__, f)) for f in | ||
| ("BindingBenchmark.jl", "setup.jl", "run.jl", "controls.jl", | ||
| "mutation_controls.jl", "test_numerics.jl", "benchmark.toml")) |
| Uniform initial pools over the interior. `b0` is clipped to the local capacity, and a | ||
| `b0` that would exceed capacity anywhere is refused rather than quietly clipped. |
A numerical benchmark and nothing else. It integrates a dissolved pool and a bound pool on
the labels and mask of one frozen snapshot, and asks whether the implemented scheme
conserves what it should, loses only what its ledger books, and converges at the order it
claims. No chemistry is asserted, no molecule, species or isotope is named, and it does not
establish that the generating CPM ever had a binding loop of this shape. The
thorium/oxalate model is not coded here and stays deferred to a separate specification
packet.
Based on
master, not on the six open PRs: this depends on none of them.Closed means
q_ext = 0and no flux through the boundary, so the only thing that leaves isdecay. Units are the diagnostic's own:
dtu, lattice sites, a declared arbitraryconcentration unit. The days used by the decay-reference illustration are that diagnostic's
clock and are not imported here.
The timestep bound is derived from this scheme, not imported
In 3-D the diffusion bound is
h²/(6D), not the one-dimensionalh²/(2D). Ath = D = 1and
Δt = 0.25, which the 1-D bound admits, a unit centre with six zero neighbours landsat −0.5. The suite asserts that value and asserts that
step!refuses the step producingit. Reaction and decay are unsplit, so they enter the same restriction, and each of the
three contributing terms is pinned by its own assertion rather than by one combined number.
The rate is re-evaluated inside every step, because
max(c)andmax(B − b)both move.Δt ≤ 1/λis a positivity condition and not an accuracy condition: atΔt = 1/λthescheme returns exactly zero against a continuum value of
1/e, and both halves areasserted.
Two structural choices
Transport is in flux form, iterating faces rather than sites, so each interior–interior
face contributes
+fto one cell and−fto the other. 6,766 of this geometry's 50,200interior sites are boundary-ragged, and a site-centred stencil with boundary special cases
leaks at exactly those.
Every ledger term is accumulated from the term the step computes, never by differencing the
fields, so the closure identity has two independent sides. The Laplacian's floating-point
residue is reported separately and is deliberately not a term in that identity: folding
it in would let a real boundary leak close the books.
What the shipped run measured
100 steps,
T = 50dtu, 50,200 interior sites, 3,038 occupied, capacity integral 7017.9486.Binding is internal and both pools lose at the same rate, so the total obeys
dI/dt = −λIexactly. The final total, 30409.6759, is
50200 · (1 − λΔt)¹⁰⁰to the digits printed: anindependent check on the whole integration that costs nothing.
Both bound-fraction denominators are reported separately throughout. They differ by 4.33×
here, so a single number called "the bound fraction" would be wrong under one reading of it
whichever one was meant.
Controls
controls.jlwritescontrol_verification.jsonand exits non-zero on an unexpectedverdict. Two rows are expected red.
ΔB = 0reduction0 ≤ b ≤ BK1 measures rather than judges: at equal integrated capacity at every comparison time,
moving capacity from uniform to
h(A)changes the bound inventory by up to 1.89%. The tworuns are cross-checked structurally as well, since the total inventory must be identical
between them and so any difference in the bound pool has to be mirrored exactly in the
dissolved pool. It is.
K8's verdict is FIRES because red is the correct outcome.
closure_residualdoes notinclude the
droppedaccumulator, on purpose: a ledger that books the material a bugdestroys is a ledger that cannot report the bug.
Red before green, three times
make_statecheckedb0 <= Bat every interior site rather than where the pool isplaced. Capacity is zero off the occupied set, so any positive
b0was refused by a guardlooking at sites that never receive a bound pool. Four assertions caught it.
The first fixture written for the degenerate-modulator guard was itself degenerate: a solid
2×2×2 block, in which every site has exactly three occupied face neighbours, so
A ≡ 1/2.The guard fired on it. That case is now asserted.
mutation_controls.jlplants seven single-line defects in a scratch copy of the module andruns the data-free suite against each. All seven are caught (by 6, 11, 2, 1, 2, 1 and 2
assertions), and the unmutated baseline is green. Each patch is verified to have applied
exactly once before its suite runs, and a patch matching zero or many times is reported
as DID-NOT-APPLY rather than as a result: a textual patch that quietly matched nothing is
indistinguishable in a report from a defect the suite failed to catch, which is precisely
what happened to the colour-table control in #28.
Provenance and blast radius
One pin.
run_manifest.jsonis hashed againstf00e470a…, the snapshot against the entrythe manifest itself carries for it, and the snapshot's own
mcs,run_id,label_state_hashandmask_sha256against the configuration. Carrying agit_shawouldsay which code produced the bytes; it would not say that these are the bytes.
The 101-frame authoritative trajectory is not regenerated. The root
Project.tomland bothprotected CSVs are byte-identical to
HEAD. Nothing outsidediagnostics/binding_benchmark/is added or changed. Receipts record paths as configured rather than as absolute paths on
the machine that ran them, since the hashes are what bind a receipt to bytes.
The diagnostic carries its own
Project.toml: the root project lists AMDGPU, which cannotinstantiate on darwin, so a benchmark sharing it could not be run on the machine it was
written on.
Declarations, proposed and not ratified
Recorded as choices, because no one ratified them on the operator's behalf. Bound inventory
fixed at lattice sites with no advection; volumetric finite capacity, one per occupied site;
max(b − B, 0)released locally to dissolved, recorded, and tested by a forced capacitydecrease; decayed material accumulated in a loss ledger without occupying capacity; the
modulator taken as the 6-connected occupied face-neighbour fraction.
clamp_dissolved = true, used only by K3, is a different scheme and not a view of this one:holding
cfixed makes the system open. The material the clamp returns is booked aschemostat_inputso the ledger still closes, and the production run never sets it.Not done here
Recomputing the inert signal itself on these frozen labels belongs to
diagnostics/inert_signal, which lives on #30's branch. Importing it would make this branchdepend on an unmerged PR, so it is left out and the README says so.
Reproducing
test_numerics.jlandmutation_controls.jltake no data path, no argument and noenvironment variable, so their assertions cannot be skipped by a missing bundle.
🤖 Generated with Claude Code
https://claude.ai/code/session_01V1GLLCekStea7UHjRF51aw