Render-truth backbone: Rust fips-render producer + causal ledger in the per-event timeline

[pablof7z-agent]

Render-truth backbone: move the render-frame producer to Rust + put the causal ledger in the per-event timeline

Summary

Today the versioned render stream (render-frame-v1alpha1) is produced client-side in Swift, and the exact per-cause causal ledger never enters the per-event timeline (it exists only on the final RunArtifact). This issue proposes the right long-term move:

1. Move the render-frame producer into a pure Rust fips-render crate — a deterministic project(trace, fidelity, provenance) -> Vec<RenderFrame> fold over the ordered event trace — so render truth is derived from the same Rust source as everything else and cannot drift from the engine.
2. Put the causal ledger waterfall into the replayable per-event timeline (as per-event deltas), so a run can be scrubbed/seeked to an exact causal state without re-running, and the flame/stage views replay from the stream instead of a bolted-on artifact summary.

Both require a render-frame-v1alpha2 schema cut and a run-hash/golden regeneration. This is intentionally deferred behind a lean Swift-first visualization pass (see "Relationship to current work"); it is filed now so the rationale and investigation aren't lost.

Related: closed #56 (M6 artifact query/downsampling). Builds on the renderer-truth work already on main (feat: independently verify renderer semantics).

Background — what prompted this

We ran a visualization-coverage audit of the Wind Tunnel: the app's purpose is to visualize deterministic mesh-protocol simulations (observe exact causal propagation, congestion, protocol limits), but a lot of important simulation state the engine computes is invisible on the canvas or buried in panels. While scoping how to surface it, we mapped the render data path end to end and found two structural facts that motivate this issue.

What we investigated (findings)

The render-frame producer already exists — in Swift, not Rust. FIPSDPackage/Sources/FIPSDFeature/RenderFrame.swift folds raw SimulationState into the exact render-frame-v1alpha1 shape (nodes, edges, cohorts, reconciliation, violations), including world coordinates via a splitmix64 radial hash, and the workbench writes render-frames.v1.jsonl evidence. So render-frame-v1alpha1 is not an unproduced contract — but the producer is a Swift reimplementation that:

• can drift from Rust truth (two independent projections of the same semantics),
• sees only top-level event fields (it can't reach engine-internal state the events don't already carry),
• and is duplicated effort every time the render model evolves.

The causal ledger is computed exactly but never enters the timeline. The engine's add_ledger calls (scattered through the handlers, e.g. crates/fips-engine/src/engine_tree.rs) build a complete per-cause waterfall (requested → coalesced → constructed → transmitted → delivered/rejected). But it is assembled onto the final RunArtifact only (crates/fips-engine/src/engine_finish.rs); a naive "replay the event stream" viewer never sees it. Relatedly, LedgerEntry.causal_parent is currently always None, and partition/node-loss blast-radius counts (sessions-disrupted, caches-invalidated) are ledger-only, absent from the events that cause them.

Wire/const fidelity is high and pinned. Separately confirmed during the audit: wire sizes are pinned to fixtures generated by the real FIPS serializers, and the engine follows executable codecs over prose. This is why a Rust projector is attractive — render truth should be derived on the same trusted Rust side, gated by parity, rather than re-derived in Swift.

The long-term move (proposed design)

A. fips-render crate (kill drift)

• New crate crates/fips-render exposing pure deterministic project(trace, fidelity, provenance) -> Vec<RenderFrame> folding the already-ordered EventRecord trace.
• Not in the engine hot loop (render truth is a separate concern/cadence); not left in Swift (that's the drift source).
• Port world_x/world_y layout into the projector so the evidence log is authoritative. Live-window fit stays client-side (WorldViewport / RenderFrame.positions(in:)). Swift decodes frames into the existing RenderNode/RenderCohort types; client-side derivation is deleted once byte-parity is proven.

B. Causal ledger in the timeline (enable causal replay)

• Instrument add_ledger to capture per-event ledger mutations; flush them into a new EventRecord.ledger_delta field (skip_serializing_if empty so unrelated runs stay byte-identical).
• The projector accumulates cumulative totals; seek = sum of deltas ≤ ordinal, giving the absolute causal waterfall at any point without replay.
• Populate LedgerEntry.causal_parent (today always None) via an explicit primary-cause rule for multi-cause events — one mechanism feeding both the timeline and the end-of-run aggregate.

C. render-frame-v1alpha2 schema

v1alpha1 objects are additionalProperties:false + all-required, so any addition is breaking → clean version cut. The v2 additions also make room for the visualization signals the audit wants surfaced (these can land incrementally once the projector owns them):

• Congestion on link: queue_occupancy_bytes, queue_capacity_bytes, congestion_ratio, bandwidth_bps, latency_ns, loss_ppm, mtu_bytes, shared_medium_group.
• Cause-of-inactivity on node+link: inactive_cause enum (node-death|partition|transport-class-failure|link-conditions), inactive_cause_event_id, intervention_id.
• Bloom new directional bloom_edges[]: from/to/edge_id, role, occupied_bits, size_bits, hash_count, fill_ratio, fpr_ppb, estimated_cardinality, saturated (null under Occupancy/SparseBits fidelity), fresh.
• Adversarial on node: sybil_descended (sticky), sybil_ordinal, attacker_operations, transport_class_faulted, fault_class.
• Lookup new lookups[]: flow_id, origin, attempt, ttl, path_len, outcome enum (in-flight|succeeded|retry|ttl-exhausted|failed-discovery), retry_at_ns.
• Causal ledger per-frame: causal_ledger{ deltas[{causal_id, causal_parent, stage, delta, cumulative, event_id, evidence}], stage_order }.
• Plus emit bloom size_bits/hash_count from the engine (static filter params).

Rationale — why this is the right long-term move

• Single source of render truth. Deriving frames in Rust from the same trace the engine emits removes an entire class of Swift/Rust divergence bugs; the evidence log becomes the authority, gated by parity.
• Causal replay is the product thesis. The app exists to "observe exact causal propagation." Putting the ledger in the timeline is what makes seek/scrub/flame views replay faithfully rather than reconstruct from a summary.
• Extensibility. New render signals become projector fields validated against a versioned schema, instead of ad-hoc Swift reconstructions.
• It compounds with existing renderer-truth work already on main.

Relationship to current work (why this is deferred)

We are shipping the user-visible goal first via a lean, all-Swift pass: surface congestion, cause-of-failure, adversarial lineage, lookup-failure, and a bloom overlay/inspector on the canvas using the existing Swift producer, deriving fields from the already-replayable event stream (e.g. bloom role from tree topology; size_bits/hash_count are constants). That path needs zero engine changes, no schema bump, and no artifact re-hashing, so it delivers value fast and low-risk.

This issue is the backbone those signals should eventually sit on. Doing the lean pass first does not burn any bridges: the projector and causal-timeline remain independently justified, and the Swift field-derivations become parity references for the Rust projector.

Risks / decisions

1. Golden blast radius / run-hash break. ledger_delta shifts trace bytes → changes the run hash (engine_finish.rs) → every run_id/artifact_id for runs that exercise ledger deltas. skip_serializing_if empty keeps current runs byte-identical, but this is inherent once the ledger is in the timeline. Decision (owner): accept and regenerate all goldens via the blessed generators (no fidelity-flag gating).
2. Cross-language layout parity. splitmix64 + cos/sin can diverge at the last ULP between Swift and Rust. Gate on a byte-parity test; prefer fixed/normalized space; if exact parity is unreachable, ratify a quantized comparison explicitly.
3. Multi-cause events need an explicit primary-cause rule for causal_parent.
4. Bloom saturated is only valid under ExactBits fidelity (null otherwise).

Phased plan

• P0 — engine enrichment. EventRecord.ledger_delta (skip-if-empty) instrumented at add_ledger; populate LedgerEntry.causal_parent via the primary-cause rule; emit bloom size_bits/hash_count. Regenerate goldens.
• P1 — fips-render crate + byte-parity. Port the Swift projection to Rust; gate on a byte-parity test vs the current render-frames.v1.jsonl. No new signals; Swift keeps self-producing in parallel until parity proven.
• P2 — v1alpha2 signal fields. Add congestion + cause-of-inactivity + bloom_edges + adversarial + lookups + causal_ledger to the projector/schema.
• P3 — Swift consumes. Swift decodes projector frames; delete client-side derivation; wire causal_ledger to the scrubber/flame panel.

Acceptance criteria (definition of done)

• crates/fips-render produces frames byte-identical (or ratified-quantized) to the current Swift render-frames.v1.jsonl, enforced by a CI parity test.
• EventRecord.ledger_delta present and skip_serializing_if empty; runs without ledger activity are byte-unchanged.
• LedgerEntry.causal_parent populated via a documented primary-cause rule.
• render-frame-v1alpha2 schema published with the six field groups; v1alpha1 retained/migrated.
• Timeline seek reconstructs the exact causal waterfall from deltas without re-running.
• Swift renderer consumes Rust frames; client-side projection removed; all goldens regenerated the blessed way; scripts/check.sh green.

---

Filed from a visualization-coverage audit (2026-07-24). Full design specs (projector, visual-layer, coverage matrix) were produced during that audit and can be attached on request.