Found while migrating the registries for #35 (PR #111).
What
15 files under data/frameworks/ contain 345 U+FFFD replacement characters, where an em dash or similar was lost in an earlier encoding round-trip. They sit mostly in description fields, and in some titles:
"description": "Short-lived credentials � tokens expire ..."
"title": "IHT � Input Handling"
Why it was not caught
T-ACC06 (#13) added an encoding guard that finds exactly this class of corruption, but it runs over the Markdown mapping files and the shared/root docs. data/**/*.json is not in its file list, so the registries were never checked.
These strings ship: the registries are published in the npm package, in docs/frameworks-registry.js, and in the OSCAL/STIX exports.
Suggested fix
- Extend the encoding guard in
validate.js to data/frameworks/*.json and data/*.json, as an error once the corpus is clean.
- Repair the 345 occurrences. Most are derivable rather than guessed: the same sentence usually exists intact in the mapping row the item was extracted from, or in a sibling entry, so the original character can be recovered rather than assumed. Anything not derivable should be left and reported, not invented.
PR #111 rewrote the titles it touched, so those are already clean; the descriptions are not.
Counted on fix/registry-prose-ids at 3310c48.
Found while migrating the registries for #35 (PR #111).
What
15 files under
data/frameworks/contain 345 U+FFFD replacement characters, where an em dash or similar was lost in an earlier encoding round-trip. They sit mostly indescriptionfields, and in some titles:Why it was not caught
T-ACC06 (#13) added an encoding guard that finds exactly this class of corruption, but it runs over the Markdown mapping files and the shared/root docs.
data/**/*.jsonis not in its file list, so the registries were never checked.These strings ship: the registries are published in the npm package, in
docs/frameworks-registry.js, and in the OSCAL/STIX exports.Suggested fix
validate.jstodata/frameworks/*.jsonanddata/*.json, as an error once the corpus is clean.PR #111 rewrote the titles it touched, so those are already clean; the descriptions are not.
Counted on
fix/registry-prose-idsat3310c48.