Fix #35: parse control identifiers by framework grammar, not by column position - #110
Merged
Merged
Conversation
`parseControlTable()` tried `looksLikeId()`, which rejects every identifier containing a space — `Req 6.2.4`, `Art. 9`, `§5.3` — then fell through to "column 0 is the id". On 611 rows that wrote the requirement prose into `control_id` and the identifier into `control_name`, so an id could not be followed back to its framework and the control-level join resolved by luck. scripts/control-ids.js holds a per-framework identifier grammar for the ten frameworks affected. A row is scanned for its framework's own id shape, wherever that sits: the id becomes `control_id`, the rest of that cell the name, and the requirement moves to `notes`. When the table has no name column the name comes from the framework registry's title for that id rather than repeating the id. Canonical forms follow the ruling on #35 — §5.3, Art. 9, Req 6.2.4, NHI-5, bare ISO clauses, SP 800-218A keeping NIST's -PS — with CIS identified by safeguard and the control carried as `parent` (A), and the SP 800-53 rows inside the 800-82 files kept and labelled `id_source` so they are not read as 800-82 sections (B). Two parser bugs surfaced on the way: - a section can hold more than one table, and the second table's header row was parsed as data. Splitting on the |---| separators drops those 10 junk rows and gives each table its own headers, which is what lets the SP 800-53 rows keep their titles. - two-column tables lost the requirement text entirely once the id stopped being mis-filed; it now becomes `notes`. Prose-shaped ids across the corpus: 611 -> 0. Mappings 3,781 -> 3,771, the difference being those 10 header rows. 969 rows changed id; no row was removed, and no severity, relationship or confidence changed. checkControlIdShapes() in validate.js fails the build if any of the 1,337 rows in those frameworks stops carrying a well-formed identifier. The registries were seeded from these broken rows and still hold prose ids; migrating them changes headline control counts and the webapp, so it follows separately. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Sep 18, 2026
emmanuelgjr
added a commit
that referenced
this pull request
Sep 18, 2026
…bout timeline (#113) The landing paragraph still claimed 3,781 control mappings; #110 removed ten rows that were table header lines the parser had mistaken for mappings, so the figure is 3,771. The About timeline gains a row for the repair itself — identifiers now parsed by each framework's grammar rather than by column position, the registries migrated off the ids that produced (1,541 → 991 controls), and the MITRE ATLAS re-transcription. The earlier rows are left as they are: they record what shipped at the time, which is what a timeline is for. Data-fed content only; no structural change. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
emmanuelgjr
added a commit
that referenced
this pull request
Sep 18, 2026
…m the published standards (#115) * Fix #112: recover 328 replacement characters in the registries, and guard data files Fourteen registry files carried 328 U+FFFD replacement characters, mostly where an em dash belongs. They ship: the registries go out in the npm package, in docs/frameworks-registry.js and in the OSCAL/STIX exports. Every character is recovered from the source text rather than assumed. The registries were extracted from the mapping rows, so for each corrupted string the text either side of the character is looked up in the Markdown and the entry JSON, and the character that sits between them is read off. A character is taken only when every match agrees; the window narrows from 24 to 8 characters when a description concatenates several rows, and keeps the raw slice when only one word follows, since trimming to a word boundary could otherwise empty it. All 328 resolved, every one to U+2014. None was guessed, and no other character was found. The guard from T-ACC06 only ever read Markdown, which is why this sat unnoticed. It now also reads data/frameworks/*.json, data/entries/*.json and data/*.json, checking for U+FFFD alone — the arrow heuristic is tuned for prose and diagrams, not JSON. Negative-tested: a replacement character injected into soc2.json fails validation, naming the file and line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Title the 19 registry entries that carried only their identifier, from the published standards After #110 and #111, 35 mapping rows still named themselves after their id — "Req 11.3 | Req 11.3" — because neither the mapping table nor the registry held a title for those 19 ids. The text now comes from the standards themselves, transcribed rather than written. PCI DSS, 17 requirements: the headings as printed in PCI SSC's own "PCI DSS v4.0 SAQ D for Merchants", extracted with pdftotext from the published PDF so the wording is the document's, not a paraphrase. Req 11.3 reads "External and internal vulnerabilities are regularly identified, prioritized, and addressed." SOC 2, PI1.3 and P7.1: TSP section 100, 2017 Trust Services Criteria (with Revised Points of Focus — 2022). The AICPA download needs a session, so the wording was taken from two independent copies of that document and compared; both agree character for character. Only entries whose title was empty or equal to their own id were filled, so no curated title is overwritten. Rows with no published title available were left alone — there are none left. Mapping rows whose control_name is merely their control_id: 35 -> 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the data half of #35. 611 prose-shaped
control_idvalues → 0.The bug
parseControlTable()askedlooksLikeId(), which rejects any identifier containing a space —Req 6.2.4,Art. 9,§5.3,SR 3.3— and then fell through to "column 0 is the id". Where a table wroteCC3.2 — Name | requirement prose, the row came out as:{ "control_id": "Prompt injection documented as a threat in LLM application risk assessment — vectors, likelihood, impact assessed", "control_name": "CC3.2" }A reader could not follow an id back to its framework, and the control-level join resolved by luck.
The fix
scripts/control-ids.js— a per-framework identifier grammar for the ten affected frameworks. Each row is scanned for its own framework's id shape, wherever it sits; the id becomescontrol_id, the remainder of that cell becomes the name, and the requirement moves tonotes. Where a table has no name column, the name is taken from the framework registry's title for that id instead of repeating the id.Canonical forms are your ruling on #35:
§5.3,Art. 9,Req 6.2.4,NHI-5, bare ISO clauses (6.1), SP 800-218A keeping NIST's-PS.control_idis the safeguard (16.1), with the control carried asparent(CIS-16); 138 rows.id_source: "NIST SP 800-53 control cited by the SP 800-82 overlay"; 30 rows.EU AI Act keeps paragraph references (
Art. 55(1)(b)), because the paragraph is what says which obligation was mapped.Two parser bugs found on the way
LLM_NISTSP80082.mdfollows its SP 800-82 sections with an SP 800-53 table, and that second table's header row was being parsed as a mapping — 10 junk rows likecontrol_id: "Title". Splitting on the|---|separators drops them and gives each table its own headers, which is what lets the SP 800-53 rows keep their titles (SI-10 → Information Input Validation) instead of naming themselves.notes, so nothing is discarded: 1,149 of 1,337 repaired rows carry their requirement.Guard
checkControlIdShapes()invalidate.jsfails the build if any of the 1,337 rows in these frameworks stops carrying a well-formed identifier.scripts/control-ids.test.mjspins the grammar against 20 row shapes taken from the files as they are today, including the ones that must yield no name rather than promote a requirement into one.Scope and verification
node scripts/validate.js: 0 errors, 87 warnings, 326 passed.npm run test:scripts: 85 passed, 0 failed, twice (81 + 4 new).npm run stats:checkcurrent; generator reruns clean.Not in this PR
The registries in
data/frameworks/were seeded from these same broken rows and still hold prose ids (NHI 104 of 117, AI Testing Guide 58 of 58). Migrating them moves headline control counts substantially and needs a note on the README and the webapp, so it follows as its own PR.🤖 Generated with Claude Code