Skip to content

fix(testing)!: grade every verifier properly - #66

Open
sephynox wants to merge 1 commit into
fix/audit-phase-6from
fix/audit-phase-6-1
Open

sephynox wants to merge 1 commit into
fix/audit-phase-6from
fix/audit-phase-6-1

Conversation

@sephynox

@sephynox sephynox commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Two harness verifiers were reporting results nobody computed: Layer 3's verdict had no reader since 596d3e1, and the gate check compared each spec against a decision the harness itself had written moments earlier. Both now answer from the evidence they hold, which turned 282 silently-wrong results into real verdicts.

Related Issues

Refs .audit/60-synthesis/REMEDIATION.md phase 6

Changes Made

  • Layer 3 reports one decision per assertion, not five booleans
  • A recorded trace is checked in the traces model only
  • An undecided check is reported as such, not as a violation
  • gate: is optional, and the fabricated gate decision is gone
  • A run with no gate decision is accepted, not read as an error

Testing

  • make test-all: passes

Breaking Changes

  • FdrVerdict fields are Decision. passed and complete are gone, use outcome()
  • FdrConfig.expect_failure is gone. Use .with_expect(Expect::Violation(layer))
  • gate: at a site that takes no gate now fails. Remove the declaration
  • Process is #[non_exhaustive]. Construct through Process::builder
  • ScenarioResult fields are private. Use trace(), verdict(), passed()

Note

Medium Risk
Changes core test verdict semantics and breaks callers that relied on implicit gate: Ok or old FDR/ScenarioResult APIs; incorrect specs may now fail where they previously passed silently.

Overview
Gate expectations are no longer fabricated or mandatory. The scenario harness stops writing TransitStatus::Ok into every consumed trace, and tb_assert_spec! treats gate: as optional—omitting it means the spec requires no gate decision. Specs that still declare gate: Ok are checked against the trace’s real decision; a gateless run now fails Layer 1 assertion verification instead of passing because the harness lied.

Trace semantics align with “no gate taken.” ConsumedTrace::execution_mode() treats missing gate decisions as Accept, not Error, and FDR trace helpers treat no decision as valid/terminal where appropriate instead of requiring a gate for every run.

Mechanical fallout: AssertSpecBuilder::gate_decision becomes expected_gate(Option<TransitStatus>), macro plumbing passes Option::None or Some(status) from optional gate: blocks, and hundreds of tests/fuzz specs drop redundant gate: Ok. New tests/gate scenarios lock in gateless acceptance vs. gate: Ok rejection.

Reviewed by Cursor Bugbot for commit 4382258. Bugbot is set up for automated code reviews on this repo. Configure here.

@sephynox sephynox self-assigned this Sep 12, 2026
@sephynox
sephynox added this pull request to stack #55 September 12, 2026 08:00
@sephynox sephynox added the bug Something isn't working label Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant