Skip to content

calls_out: keywords, annotations and tcl SQL are not calls (#3359) - #3403

Merged
squid-protocol merged 8 commits into
mainfrom
fix/3359-calls-out-keywords
Sep 23, 2026
Merged

squid-protocol merged 8 commits into
mainfrom
fix/3359-calls-out-keywords

Conversation

@squid-protocol

@squid-protocol squid-protocol commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

Closes #3359

Precision fix for calls_out, per the #3327 contract (docs/calls_out_rule_contract.md, classes K, A and S). A keyword followed by ( is not a call (C2). A metadata annotation is not a call (C1). String content is not a call (C7).

What changed

  • K (keywords): added _calls_out_ignore entries for 24 languages: ada, cpp, csharp, dart, embedded_python, go, groovy, java, kotlin, livecode, lua, perl, php, powershell, python, rust, scala, scheme, solidity, swift, typescript and zig, plus cobol and agc. Only keywords were added. No built-ins were removed, and _CALLS_OUT_GLOBAL_IGNORE is unchanged (#3327 follow-up: built-ins are calls -- trim _CALLS_OUT_GLOBAL_IGNORE to keywords (contract C2) #3361 owns those).
  • Exact-case ignore sets: detector.py now casefolds the per-language set only for languages declared identifier_case: insensitive. Every other language compares names exactly. Without this, go's type keyword swallowed the real call v.Type(), this swallowed C#'s factory.This(), and rust's let swallowed Expr::Let(.
  • A (annotations): a new pattern, CALLS_OUT_C_STYLE_NO_ANNOTATION ((?<!@)), for java, kotlin, swift, dart, groovy and scala. detector.py treats it the same as CALLS_OUT_C_STYLE, so qualifiers are still captured (#3265 step 2: capture the call qualifier (utils.parse, obj.save, Class::method) #3329). Python, TypeScript and JavaScript keep the plain pattern, because a decorator factory like @retry(3) really is a call.
  • S (tcl): the command-position rule now skips upper-case SQL keywords at the start of a line inside a brace-quoted query. Only upper-case forms are skipped, because set and update are real Tcl commands.
  • cobol (rebuilt on top of Engine: COBOL GO TO is a transfer, recorded beside calls (#3362) #3394's new rule): END-PERFORM and END-CALL end in the verb. Because of that, \b treated the next statement's first word (MOVE, IF, END-IF, ...) as the callee, so the anchor is now (?<![\w-]). Inline PERFORM VARYING/UNTIL/WITH TEST names no paragraph, so those words are ignored.
  • agc: TC Q is the return idiom (a transfer through the Q register), not a call.
  • The test_calls_out_contract_3327.py strict xfails for #3327 follow-up: calls_out keyword and annotation captures (contract C1/C2) #3359 (annotation, go func) are replaced by a new pin file, tests/extraction/languages/test_calls_out_keywords_3359.py (33 cases). Each case pairs the dropped non-call with a real call in the same body, so a recall loss would fail the test.

Census (language-crucible v1.4.0 + keyword-rosetta, every function, before vs after)

4,342 names removed and 18 added, across 25 languages. The set of sliced functions did not change.

Largest removals: php 1,000 (foreach 237, isset 210, empty 152, elseif 114, array 111, unset 79, fn/use 44), livecode 893 (private 250, public 99, command 95, ...), go 445 (func 436), ada 316 (is 269), csharp 223, zig 172, rust 163, python 147, scheme 146, solidity 101, scala 86, powershell 80, groovy 77, cobol 77, perl 70, agc 61 (Q), lua 57, typescript 47, dart 42, cpp 35, tcl 35 (SQL), java 34, swift 26, kotlin 6, embedded_python 3.

For every removed name, every occurrence in that function was a keyword, a special form, an annotation (@X() or SQL string text. Ambiguous candidates were checked by reading the crucible source, which is why these are not ignored:

  • php match: $this->match( is a real method call.
  • perl do/then: every crucible hit is $dbh->do( or ->then(. The contract table lists perl do as K, but the crucible shows it is a real call.
  • tcl default: a macports command.
  • perl qx(: it runs a shell command, and the rosetta oracle plants it as a call.
  • go type was dropped from the go set in favour of exact compare, because v.Type() is a common real call.

The 18 added names are real calls that were recovered:

  • perl Warn (17): $self->Warn( in exiftool. It had been filtered because the old casefolded compare matched it against warn.
  • cobol 740-WRITE-CUSTOUT-FILE (1): END-PERFORM PERFORM 740-... used to capture the second PERFORM as the callee.

These are the only recall movements, and both are side effects of the precision fixes.

Golden master

crucible_check.py --update --yes ran against the pinned v1.4.0 corpus (language-crucible-worktrees/v1.4.0-agent), and then rebase_rebless.py --execute ran on top of #3394/#3396/#3399. Both fixtures report no drift, so the bless diff is 0 lines. calls_out_to is not part of the audit JSON, which is why nothing moved there.

Verification

  • Full pytest tests/ (after the #3327 follow-up: nested declarations captured as calls (contract C5) #3360 merge): 10,661 passed, 318 skipped, 10 xfailed, 3 xpassed. The 3 xpasses are pre-existing jcl ones.
  • audit_check.py: ruff, mypy, dead-key and ast-accuracy all clear. ruff format is clean.
  • rosetta_audit.py (51 languages): with the companion rebless, 0 regressions. Against rosetta main, 7 languages move, all in calls_out_to and all intended: csharp nameof, groovy/java SuppressWarnings, kotlin when/Suppress, php isset/unset, scheme guard, and solidity returns (13 cells).
  • check_calls_out_truth.py --all on the companion branch: 51 PASS. The remaining WARNs are pre-existing periphery gaps in languages this PR does not touch.

call-graph-accuracy gate (#3332)

The gate compares engine callees against tree-sitter call nodes. Tree-sitter parses php isset/empty/unset/die/exit and C# nameof as call nodes, but the #3327 contract says they are keywords. So the gate read this PR as a recall drop (php 98.3 -> 92.5, csharp 97.2 -> 95.8).

The tool's own docstring says both sides follow the contract. CONTRACT_NON_CALLS now removes those names from the tree-sitter side, and the baseline was regenerated against v1.4.0. Results:

  • Precision up: go 90.3 -> 99.6 (func), php 88.1 -> 96.3, csharp 89.5 -> 93.8, python 92.1 -> 93.8, java 96.9 -> 99.1, rust 84.9 -> 86.7, cpp 87.4 -> 87.7, typescript 80.7 -> 81.2.
  • Recall unchanged everywhere except php, 98.3 -> 98.2, because the excluded names left the matched set.

Built-ins are deliberately not excluded, since they are calls (#3361).

After merging #3360 the baseline was regenerated again. Current precision: python 96.2, typescript 82.6, ruby 91.3, rust 87.1, csharp 93.9, php 96.3, java 99.1, go 99.6. Recall is unchanged in every language.

Merge with #3360 (nested declarations, C5)

This branch has origin/main merged in, not rebased, so nothing was force-pushed. The only conflict was in detector.py:

The resolution keeps #3360's body and uses the two-pattern condition. That matters: had the merge kept either side's version alone, the #3360 check would have stopped running for java, kotlin, swift, dart, groovy and scala, and nested inner( headers would have come back as callees. test_nested_declaration_check_covers_the_annotation_free_pattern (kotlin, java, scala) pins this, and it fails 3/3 if the condition is narrowed back to CALLS_OUT_C_STYLE alone.

Census against the new main (which includes #3360): 4,341 names removed and 18 added, the same classes as below. Nothing beyond this PR's own keyword/annotation/SQL removals moved, because main already applied the nested-declaration check to those six languages and this merge keeps it.

Golden masters: #3402 (the split layout) has not merged, so the merge took main's monolithic fixtures. crucible_check.py --update --yes against v1.4.0 reports no drift in either mode. A hand-run golden_diff on the zero-dep audit also gives 0 differences.

Deferred (named in the contract doc's new "#3359 resolution" section)

  • scheme let binding lists ((x 1)): the same (( shape is also a cond clause or a curried call, so fixing it needs more than a regex tweak.
  • dotted and use-site annotations (@a.b.C(, kotlin @file:JvmName().
  • rust Fn( trait sugar and #[cfg(not(...))] predicates.
  • tcl lower-case prose inside brace-quoted help text.
  • powershell hashtable-key and enum-member lines in command position.
  • cobol PERFORM <data-name> TIMES.

Cross-repo

  • Companion: rebless: calls_out keywords/annotations are not calls (companion to gitgalaxy#3403) keyword-rosetta#154 (draft), which re-blesses the 7 moved manifests and updates docs/calls_out_truth.json. It drops php isset/unset and scheme guard from the periphery lists and adds the new keyword/annotation traps to forbidden_everywhere.
  • Merge order: this PR first. The rosetta PR stays in draft until this merges, because rosetta CI checks out gitgalaxy main.
  • Re-run after: once this merges, re-run the rosetta PR's CI and then merge it. bias-history.yml regenerates on its own. Label: rosetta:rebless-owed.

🤖 Generated with Claude Code

squid-protocol and others added 5 commits September 23, 2026 14:33
Contract C2/C1/C7 (docs/calls_out_rule_contract.md, classes K/A/S):
- _calls_out_ignore keyword additions for 24 languages (no built-ins
  touched, _CALLS_OUT_GLOBAL_IGNORE untouched)
- CALLS_OUT_C_STYLE_NO_ANNOTATION ((?<!@)) for java/kotlin/swift/dart/
  groovy/scala; python/ts decorator factories stay calls
- per-language ignore compared exactly for case-sensitive languages,
  casefolded only for identifier_case: insensitive ones
- tcl: upper-case SQL keywords excluded from command position
- cobol: END-PERFORM/END-CALL no longer read the next statement as a
  callee; inline PERFORM VARYING/UNTIL/WITH names no paragraph
- agc: TC Q is the return idiom

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…3359)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…3359)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@squid-protocol squid-protocol added the rosetta:rebless-owed Intentionally moves keyword-rosetta counts; audit warns, corpus re-blesses after merge label Sep 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🐦‍⬛ Muninn Security Scan

✅ No security issues found.

🐦‍⬛ Powered by Muninn · Skald Lab

squid-protocol and others added 3 commits September 23, 2026 14:58
… too (#3359)

tree-sitter parses php isset/empty/unset/die/exit and C# nameof as call
nodes; the contract says they are keywords (C2), and the engine now drops
them. Exclude them from the tree-sitter side so the gate does not read
following the contract as lost recall. Baseline regenerated: precision up
in go (90.3 -> 99.6), php (88.1 -> 96.3), csharp (89.5 -> 93.8),
python, rust, java, cpp, typescript; recall unchanged except php
98.3 -> 98.2 (the excluded names left the matched set).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Resolve detector.py: keep #3360's nested-declaration check and run it for
both CALLS_OUT_C_STYLE and CALLS_OUT_C_STYLE_NO_ANNOTATION, so the
java/kotlin/swift/dart/groovy/scala family gets C5 too. Pinned by
test_nested_declaration_check_covers_the_annotation_free_pattern.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Precision up (python 93.8 -> 96.2, typescript 81.2 -> 82.6, ruby 89.8 ->
91.3, rust 86.7 -> 87.1, javascript, csharp, java, php); recall unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@squid-protocol
squid-protocol merged commit 2dfeb83 into main Sep 23, 2026
34 checks passed
@squid-protocol
squid-protocol deleted the fix/3359-calls-out-keywords branch September 23, 2026 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rosetta:rebless-owed Intentionally moves keyword-rosetta counts; audit warns, corpus re-blesses after merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

#3327 follow-up: calls_out keyword and annotation captures (contract C1/C2)

1 participant