Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ Recommendation: **MIT** for the code. Maximum reach, minimum friction, and the s
- FCRA: prohibit use for employment, credit, or tenancy decisions in the docs.
- GDPR: officer and PSC data is personal data from public registers. Legitimate interest is the usual basis for this kind of processing; ship a short `PRIVACY.md`, and any hosted deployment must handle erasure requests.
- Companies House data reuse terms: verify the current terms before launch.
- Defamation posture: memos state record and confidence. Language rules in the synthesis prompt forbid accusatory phrasing; this is a technical control, not just a docs note. CI also scans the tool-authored statement fields on casefile.json.
- Defamation posture: memos state record and confidence. Language rules in the synthesis prompt forbid accusatory phrasing; this is a technical control, not just a docs note. The in-process whole-memo backstop scopes claim-quote exemptions to the rendered claims table. CI also scans the tool-authored statement fields on casefile.json.
- No scraping behind authentication, ever.

## 16. Verify before build
Expand Down
11 changes: 11 additions & 0 deletions DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,17 @@ The log is written from `write_case` through `write_case_text` (`O_NOFOLLOW`). B

No live Companies House, OpenSanctions, Tavily, or model calls in this sprint. Committed fixtures were not given a historical log. `scripts/check_language.py` default targets were not expanded: the log is not tool prose.

### 2026-08-20: whole-memo claim-quote exemptions stay inside the claims table
`language_backstop_failure` used to apply stored claim texts as exemption spans across the entire rendered memo. The per-field synthesis gate already gives model prose zero claim-quote exemption, but `--render-only` (and `rerun` with no model) skips synthesis. A hand-edited `casefile.json` could plant a short claim, or copy a stored claim phrase into narrative, and the backstop would treat every matching span as quoted data.

The in-process backstop now splits the memo at the template heading. The claims-table region starts at the first line that is exactly `## Claims vs evidence` and ends at the next line that starts with `## ` (that closer is not part of the region). Claim texts plus registry identity names are exempt inside that region. Identity names stay exempt before and after it. If the start heading is missing, or that heading has no following `## ` closer, the region is empty and claim-quote exemptions apply nowhere. If the heading appears more than once, only the first start/closer pair is used.

The close is region-level, not cell-level. A hand-edited `record_note` that copies a claim's banned wording and still sits inside the table region can still be span-exempt there. That residual is accepted for this sprint.

`scripts/check_language.py` memo line scanning is unchanged: it may still apply claim exemptions line-by-line across the whole memo. Label-aware CI re-verification stays in FUTURE.md. The synthesis per-field gate is unchanged (identity only). Quotation verification and match-length span advancement are unchanged. Pipe-escape in `_claim_rows` stays fail-closed: a claim that contains both `|` and a banned word is stored as the raw text and rendered escaped, so it no longer span-matches and the backstop fails closed. This sprint does not exempt the escaped form. A minimum-substance rule for stored claims is a different FUTURE bullet and was not added.

No live Companies House, OpenSanctions, Tavily, or model calls in this sprint. Committed fixture casefiles and snapshot memos were not edited.

## Section 16 verification log

Findings are recorded here as verification completes, each with source URL and retrieval date.
Expand Down
1 change: 0 additions & 1 deletion FUTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ Ideas deferred from the weekend 3 build and review:

- OCR fallback for image-only decks; the gap is currently recorded explicitly as a finding.
- Fuzzy quotation matching for claim verification, with a strict-verbatim default; per-claim provenance offsets into the extracted text for a stronger audit trail.
- Scope the whole-memo language backstop's exemptions to the rendered claims-table region, closing the hand-tampered-casefile residual on render-only reruns.
- A minimum-substance rule for stored claims so single-word quotes cannot become exemption spans.
- Sitemap-based about-page discovery and a configurable path token list for site extraction.
- Label-aware claim re-verification in the language check script (mapping site evidence back to source labels).
Expand Down
8 changes: 4 additions & 4 deletions HANDOFF.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The repository is at github.com/samrusani/coldscreen. Branch `main` is current a

A CLI that turns a UK company name into a first-pass screening memo built entirely from public sources, with every finding traceable to evidence. It is not due diligence. It is the screen that decides whether due diligence is worth anyone's time. The same pipeline is also an MCP stdio server, so the screen runs inside agent workflows without a second implementation of it.

Current state: 766 tests, 28 modules, roughly 9,600 lines of source, green on Python 3.11 through 3.13. All three milestone success tests passed, two of them against the live Companies House API. Feature-complete for v0.1; not yet published to PyPI. Rubric 0.3 adds a mechanical R4 floor for origin-year contradictions ("operating since 2015" against incorporation in 2019), so that class of claims-bearing case now anchors unconditionally. Cache UX (`--refresh`, `coldscreen cache path|clear|stats`) landed after charges pagination. `--no-write` on `screen` skips the case directory for that run and still uses the HTTP cache; the default persist path is unchanged. A screen writes `fetch_log.json` at the case directory root (URLs, sanitized params, timestamps, cache flags; no bodies, no keys); rerun and `--no-write` do not. The language check now covers tool-authored casefile statements as well as rendered memos and templates. The stage-honesty phrase set grew from observed not-run lies; the gate is still a substring check, still sanctions and media only, and still arms only when those stages are recorded not run or failed. SNI through the pinned backend is proven by a loopback HTTPS handshake; the httpx pool assignment stays fail-closed by choice.
Current state: 774 tests, 28 modules, roughly 9,600 lines of source, green on Python 3.11 through 3.13. All three milestone success tests passed, two of them against the live Companies House API. Feature-complete for v0.1; not yet published to PyPI. Rubric 0.3 adds a mechanical R4 floor for origin-year contradictions ("operating since 2015" against incorporation in 2019), so that class of claims-bearing case now anchors unconditionally. Cache UX (`--refresh`, `coldscreen cache path|clear|stats`) landed after charges pagination. `--no-write` on `screen` skips the case directory for that run and still uses the HTTP cache; the default persist path is unchanged. A screen writes `fetch_log.json` at the case directory root (URLs, sanitized params, timestamps, cache flags; no bodies, no keys); rerun and `--no-write` do not. The language check now covers tool-authored casefile statements as well as rendered memos and templates. The stage-honesty phrase set grew from observed not-run lies; the gate is still a substring check, still sanctions and media only, and still arms only when those stages are recorded not run or failed. SNI through the pinned backend is proven by a loopback HTTPS handshake; the httpx pool assignment stays fail-closed by choice.

## The five non-negotiables

Expand All @@ -27,7 +27,7 @@ The level is a pure function of the enforced trigger set. There is a property te

**4. The language control is a technical control, not a tone preference.** Memos say "contradicted by public record" with a confidence tag; they never state or imply fraud, dishonesty, or intent. Three layers: the prompt states the rules, a per-field gate scans model output with one corrective retry, and a whole-memo scan runs before any memo reaches disk. The banned-word scan strips URLs (source URLs legitimately contain words like fraud in slugs) and exempts provenance-verified quoted data.

That exemption is the subtle part and it has already been attacked successfully once. Claim text is the company's own words, so a deck saying "we fight fraud" must render. But claim text comes from the model, so an unverified exemption is a laundering channel: an early design let a model smuggle arbitrary vocabulary into memos by inventing a "quotation". The fix, which you must not weaken: a claim is stored only if it verifies as a normalized verbatim substring of its declared source section, model prose gets zero exemptions ever, and the CI language script re-verifies exemptions against the committed evidence files. Four attack shapes are permanent regression tests in `tests/test_exemption_attacks.py`.
That exemption is the subtle part and it has already been attacked successfully once. Claim text is the company's own words, so a deck saying "we fight fraud" must render. But claim text comes from the model, so an unverified exemption is a laundering channel: an early design let a model smuggle arbitrary vocabulary into memos by inventing a "quotation". The fix, which you must not weaken: a claim is stored only if it verifies as a normalized verbatim substring of its declared source section, model prose gets zero exemptions ever, and the CI language script re-verifies exemptions against the committed evidence files. Four attack shapes are permanent regression tests in `tests/test_exemption_attacks.py`. The whole-memo backstop applies claim-quote exemptions only inside the rendered claims-table region.

**5. No key material anywhere it can leak.** Secrets come from environment variables only. The Companies House key travels in a basic auth header, never in URLs, cache keys, evidence params, or `__repr__`. There is a test that greps every written file for the key. The tool bundles no OpenSanctions key: their terms make rights non-transferable, so every user brings their own key under their own licence. The MCP surface inherits this rather than reopening it: no tool schema has a field that could carry a key, there is a test that walks both schemas asserting so, and keys are read from the server process environment the host sets.

Expand Down Expand Up @@ -84,9 +84,9 @@ Findings this loop caught that the test suite did not: pagination that silently

FUTURE.md holds remaining items. My recommended ordering:

1. Scope the whole-memo language backstop's exemptions to the rendered claims-table region (FUTURE.md weekend-3 residual).
1. A minimum-substance rule for stored claims so single-word quotes cannot become exemption spans (FUTURE.md weekend-3 residual).

The per-run fetch log is done: `fetch_log.json` is written on screen, not on rerun, and not on `--no-write`. The evidence index stays a manifest. Duration and retry rows were not added. The five non-negotiables above are untouched.
The claims-table-region scope is done: the whole-memo backstop applies claim-quote exemptions only between `## Claims vs evidence` and the next `## ` heading, and fails closed when that region cannot be bounded. The per-run fetch log is done: `fetch_log.json` is written on screen, not on rerun, and not on `--no-write`. The evidence index stays a manifest. Duration and retry rows were not added. The five non-negotiables above are untouched.

The language check now covers casefile statements: CI parses `casefile.json` and scans the tool-authored fields (finding statements, record notes, verdict rationale and questions, narrative, enforcement notes, skipped reasons). Media titles and claim texts are not scanned. Identity exemptions still require sibling registry evidence; claim-quote exemptions do not apply to those fields.

Expand Down
16 changes: 9 additions & 7 deletions scripts/check_language.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,15 @@
Quoted-data exemption (memos only): a memo's claims table quotes the
company's own deck and site words verbatim, and those may legitimately
contain banned vocabulary. When a memo.md has a sibling casefile.json, its
stored claim texts are candidates for span-level exemption, the same
exemption the whole-memo backstop applies. But casefile.json is an
editable file, so a claim text is honored ONLY after re-verification
against the sibling evidence: normalized (whitespace, case, unicode quotes
and dashes), it must be a substring of the extracted source text persisted
in evidence/deck_text.json or the evidence/site_*.json records. No
evidence, no exemption: a hand-tampered casefile cannot widen this scan.
stored claim texts are candidates for span-level exemption on this
line-by-line memo scan. The in-process backstop scopes those same
claim texts to the claims-table region; this script does not. But
casefile.json is an editable file, so a claim text is honored ONLY
after re-verification against the sibling evidence: normalized
(whitespace, case, unicode quotes and dashes), it must be a substring
of the extracted source text persisted in evidence/deck_text.json or
the evidence/site_*.json records. No evidence, no exemption: a
hand-tampered casefile cannot widen this scan.
Prose outside the exact verified quoted strings stays fully gated.

That claim-quote exemption does not apply to the casefile fields above.
Expand Down
83 changes: 71 additions & 12 deletions src/coldscreen/language.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
one place the banned list and the matching rules live. Every enforcement
point imports find_banned_terms, so none of them can drift: the mechanical
gate over model output (coldscreen.synthesis), the whole-memo backstop that
runs before any memo reaches disk (coldscreen.cli), and the CI gate over
runs before any memo reaches disk (coldscreen.pipeline), and the CI gate over
rendered memos and the tool-authored fields of casefile.json
(scripts/check_language.py).

Expand All @@ -29,17 +29,23 @@
per-field gate in coldscreen.synthesis: the model references claims by id
and never repeats their wording. That gate does apply the registry
identity set, because prose has to be able to name the company and its
people. Only the whole-memo backstop and the CI memo scan take claim
texts, because the code-rendered claims table quotes the stored claim
strings verbatim. The CI casefile-field scan does not: those fields are
tool prose, same polarity as the synthesis per-field gate. Claim texts
themselves are trustworthy only because the claims stage verifies each
one is a real substring of its declared source section (after
normalize_for_match on both sides) before storing it, and
scripts/check_language.py re-verifies stored claims against the sibling
evidence files before honoring them on a memo; the script re-verifies
identity names against the registry evidence files the same way, and
that identity set is the one exemption the casefile-field scan applies.
people. The whole-memo backstop takes claim texts but applies those
exemptions only inside the rendered claims-table region: the first line
that is exactly `## Claims vs evidence` through the next ATX heading
that starts with `## ` (the closer is not part of the region). A missing
start or a missing closer leaves the region empty, so claim-quote
exemptions apply nowhere and identity exemptions still apply to the
whole memo. The CI memo scan still applies claim-quote exemptions
line-by-line across the file. The CI casefile-field scan does not take
claim texts: those fields are tool prose, same polarity as the synthesis
per-field gate. Claim texts themselves are trustworthy only because the
claims stage verifies each one is a real substring of its declared
source section (after normalize_for_match on both sides) before storing
it, and scripts/check_language.py re-verifies stored claims against the
sibling evidence files before honoring them on a memo; the script
re-verifies identity names against the registry evidence files the same
way, and that identity set is the one exemption the casefile-field scan
applies.
Occurrence discovery advances by the full match length, so overlapping
occurrences of a self-similar quote can never union into coverage of
text that was never quoted as a whole.
Expand Down Expand Up @@ -184,3 +190,56 @@ def find_banned_terms(text: str, exempt_texts: Iterable[str] = ()) -> list[str]:
continue
hits.append(match.group(0).lower())
return hits


# The template heading that opens the claims-vs-evidence table. Exact line
# match only: do not invent markers or HTML comments.
_CLAIMS_TABLE_HEADING = "## Claims vs evidence"


def _claims_table_region(memo: str) -> tuple[int, int] | None:
"""Character offsets [start, end) of the claims-table region, or None.

Start is the first line that is exactly the template heading. End is
the next line that starts with `## ` (that closer is excluded). A
missing start or a missing closer yields None: claim-quote exemptions
then apply nowhere. Only the first start/closer pair is used.
"""
start: int | None = None
offset = 0
for line in memo.splitlines(keepends=True):
content = line.rstrip("\r\n")
if start is None:
if content == _CLAIMS_TABLE_HEADING:
start = offset
elif content.startswith("## "):
return start, offset
Comment on lines +212 to +216

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Content corrupts table boundaries

When a render-only casefile includes an embedded ## line in a claim-table field, or an exact ## Claims vs evidence line in earlier prose, _claims_table_region treats that content as a structural delimiter. This excludes legitimate claim rows from the exemption region and causes the language backstop to reject the rendered memo.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/coldscreen/language.py
Line: 212-216

Comment:
**Content corrupts table boundaries**

When a render-only casefile includes an embedded `## ` line in a claim-table field, or an exact `## Claims vs evidence` line in earlier prose, `_claims_table_region` treats that content as a structural delimiter. This excludes legitimate claim rows from the exemption region and causes the language backstop to reject the rendered memo.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

offset += len(line)
return None


def find_banned_terms_in_memo(
memo: str,
*,
claim_texts: Iterable[str] = (),
identity_names: Iterable[str] = (),
) -> list[str]:
"""Banned terms in a rendered memo, with claim quotes region-scoped.

The memo is scanned as three pieces. Before and after the claims-table
region, only identity_names are exempt. Inside the region, claim_texts
and identity_names are both exempt. Matching rules are find_banned_terms
unchanged. If the region cannot be bounded, the whole memo is scanned
with identity_names only.
"""
identity = tuple(identity_names)
bounds = _claims_table_region(memo)
if bounds is None:
return find_banned_terms(memo, identity)
start, end = bounds
claims_and_identity = (*claim_texts, *identity)
return (
find_banned_terms(memo[:start], identity)
+ find_banned_terms(memo[start:end], claims_and_identity)
+ find_banned_terms(memo[end:], identity)
)
5 changes: 3 additions & 2 deletions src/coldscreen/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,9 @@ class Claim(BaseModel):
(CLM-001 style), text is the company's own words as extracted from the
deck or site, source names where they appeared ("deck p.4", "site
/about"). Unfalsifiable puffery is kept with checkable False, never
dropped. Because text is quoted data, it is the one kind of string the
language gate exempts (span-level, exact match); see coldscreen.language.
dropped. Because text is quoted data, the whole-memo backstop exempts
it (span-level, exact match) only inside the rendered claims-table
region; see coldscreen.language.
"""

id: str
Expand Down
Loading
Loading