Skip to content

fix(answer-key): model multi-program sources (sibling / nested programs) (#4206) - #4226

Merged
squid-protocol merged 1 commit into
mainfrom
fix/4206-multi-program-answer-key
Oct 3, 2026
Merged

squid-protocol merged 1 commit into
mainfrom
fix/4206-multi-program-answer-key

Conversation

@squid-protocol

Copy link
Copy Markdown
Owner

Root cause

The COBOL answer key held one program per source file and keyed dead verdicts by unit name. _units read from the first PROCEDURE DIVISION to end of file. On a source holding several programs (DBB epscsmrd.cbl: 11 batch-compiled siblings plus XCHRFLTR and XWSPFLTR nested in EPSCSMRC, each opening with MAINLINE SECTION), the draft read later programs' PROGRAM-ID / AUTHOR / INSTALLATION / WORKING-STORAGE lines as units, and ran reachability across program boundaries.

Fix (tests/tools only, no engine change)

  • program_spans splits a source into its programs. The first is the key entry; the rest go in a new siblings block (line, end_line, nested_in, units, dead), read from each program's own PROCEDURE DIVISION, which stops at its END PROGRAM or its first nested program. A CALL of a sibling's PROGRAM-ID resolves to the file that holds it. Single-program sources keep the whole-file reading.
  • Score and census name a sibling unit PROG:NAME (keyed_units / keyed_dead). Engine units are placed in a program by start line, the innermost program winning (keyed_unit_name). The census brief explains PROG:NAME only when it asks about one. Readers that treat a file as one program (the refraction differential and the call-graph COBOL score) use bare names (bare_units / bare_dead). test_key_integrity checks every sibling the way it checks a program.
  • Four draft-reader errors found while verifying epscsmrd, each with a test:
    1. _GOTO consumed the whole name run, which also spans the next statements, so a later GO TO in that run was never seen. EPSCSMRI's GO TO X000000CC … DEPENDING ON targets all read as dead (~160 phantom dead units). The capture is now a lookahead.
    2. GO TO a b DEPENDING ON x GO TO y . is terminal. The DEPENDING exclusion now applies only when the final transfer is the DEPENDING one.
    3. XML PARSE … PROCESSING PROCEDURE A THRU B now counts as a reach of A..B.
    4. END-XML / END-JSON / END-INVOKE close an ON EXCEPTION phrase. EPSCSMRI MAINLINE's … END-XML GOBACK. no longer reads as falling through.
  • No program in the other six keyed corpora drafts differently. All 129 programs were drafted with origin/main's tool and with this one, then diffed. Only epscsmrd.cbl moved.
  • The answer-key README documents siblings. The stale excluded_programs reason in the DBB key is rewritten; that is its only data change.

epscsmrd.cbl stays excluded (follow-up)

#4216 merged before this PR, so the exclusion is already on main. This PR makes the key able to hold the file but does not add it:

  • Draft is correct. It reads 13 programs and 6 dead verdicts, all EPSCSMRI section headers entered only through their paragraphs. Each was checked against the source.
  • Engine agrees on units and dead code. Its usage_status marks the same six headers. Its only miss is that it records no PROGRAM-ID for the file (PROGRAM-ID. 'EPSCSMRD'., quoted, column 9).
  • Forge produces about 1,000 new ledger mismatches. It reads the file as one program. No issue owns that yet, so the program waits until one does, then gets a blind census.
  • Dropping the exclusion is then draft → insert the entry → cross_verify.py census → ground_truth_ledger.py update/assign.

Tests

New: test_program_spans_split_siblings_and_nested_programs, test_draft_keys_sibling_programs_with_their_own_units, test_score_places_engine_units_in_their_sibling_program, test_every_go_to_in_a_name_run_is_seen, test_xml_parse_processing_procedure_is_a_range_and_end_xml_closes_its_phrase, test_census_asks_about_sibling_programs_units_by_program. All five draft tests fail against origin/main's tool and pass here.

Golden drift

None. No engine code changed, and the golden gate passes without a re-bless. Sibling bug-sweep PRs do re-bless golden masters, so if one merges first, merge main and re-bless (crucible_check.py --update --yes). Never hand-merge the golden files.

Gates

  • Ground-truth ledger check: clean, no ledger change.
  • pr_gates.py, run in chunks under a shared lock: lint, audits, secrets, xray, gauntlet and golden PASS. Suite: see the comment below.

Closes #4206

🤖 Generated with Claude Code

https://claude.ai/code/session_013sNMtAg6s9dyyjNgnjRHi8

…ograms, units per PROCEDURE DIVISION (#4206)

The draft reader now splits a source into its programs (program_spans). The
first program is the entry; the rest are `siblings`, each with units and dead
verdicts from its own PROCEDURE DIVISION. Score pairs and census claims name a
sibling's unit PROG:NAME, and engine units are placed by start line. Readers
that treat a file as one program check against bare names.

Fixes four draft-reader errors found on DBB epscsmrd.cbl: a consuming GO TO
match swallowed later GO TOs in the same name run; GO TO ... DEPENDING ON
followed by a plain GO TO is terminal; XML PARSE PROCESSING PROCEDURE is a
reach; END-XML/END-JSON/END-INVOKE close an ON EXCEPTION phrase. No program in
the other six corpora drafts differently.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013sNMtAg6s9dyyjNgnjRHi8
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

@squid-protocol

Copy link
Copy Markdown
Owner Author

Suite gate: 13,932 passed. 3 failed, all pre-existing (they also fail on origin/main c0a7e90, and this PR touches no det code): tests/cobol_mainframe/test_det_programs.py :: test_typed_lifts_only_what_every_use_allows, test_a_move_that_can_leave_negative_zero_keeps_the_item_bytes, test_typed_groups_sync_a_groups_bytes_around_its_whole_uses (AssertionError: private Field ...).

@squid-protocol
squid-protocol merged commit 0f1be1b into main Oct 3, 2026
32 checks passed
@squid-protocol
squid-protocol deleted the fix/4206-multi-program-answer-key branch October 3, 2026 15:32
@squid-protocol

Copy link
Copy Markdown
Owner Author

Correction: the 3 tests/cobol_mainframe/test_det_programs.py failures listed above as pre-existing on main (test_typed_lifts_only_what_every_use_allows, test_a_move_that_can_leave_negative_zero_keeps_the_item_bytes, test_typed_groups_sync_a_groups_bytes_around_its_whole_uses) were measured against c0a7e90, which predates #4220. #4220 updated those tests to the one-line Field shape from #4202 and made CI install the translator's parser, so they no longer skip. All 3 pass on current main (0f1be1b). No follow-up issue needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

answer key: model multi-program sources (sibling programs / END PROGRAM)

1 participant