Siloscan v1.6 wayfinder map: measured secret detection

[aksOps]

Destination

siloscan's secret detection is measured rather than asserted: a corpus that covers the shapes real repositories carry, parity with gitleaks demonstrated by diff rather than by arithmetic, and the three detection debts deferred through 1.5.x closed with before-and-after numbers. Reaching the end of this map means siloscan can be recommended as a standalone secret-detection control for depth-1 working-tree CI scanning, with its remaining limits named and measured rather than suspected.

Notes

Rust workspace; rules are YAML; the detection corpus lives at crates/siloscan-core/tests/corpus with manifest.tsv and the harness at tests/detection_corpus.rs.

This map carries execution, not only decisions. The architecture is settled; what remains is measured implementation. Every rule change on this map ships with corpus coverage for the shape it targets and a before/after finding count on a large real tree. That rule exists because it was broken once: a corpus rebuild without coverage for C amalgamations and generated bindings took findings from 325 to 1222 on 1.8 GB of registry source, ~870 of them indefensible. A corpus only measures the shapes it contains.

Scope is depth-1 working-tree scanning. Git history scanning is explicitly out (see Out of scope).

Consult the dependencies skill before adding any crate; resolve versions live, never from training data.

Decisions so far

• Corpus expansion - hybrid sourcing (committed synthetic + registry/leaky-repo as external targets), split precision enforcement (CI noise corpus + manual big tree), per-family recall floors with global 1.0 precision. Grilled 2026-08-07.

• gitleaks parity - pin v8.30.1, checksum-pinned binary, scripts/ harness, manual + workflow_dispatch, never PR CI; every delta dispositioned on the ticket. Grilled 2026-08-07.

• Skipped gitleaks rules - glob translation with opt-in case_insensitive path flag; path-only rules as walk-layer findings (fire on binaries); size_limit raised measured, pypi/vault imported, generic-api-key stays excluded for generic.yaml. Moves up the order: it carries the k8s answer. Grilled 2026-08-07.

• Base64 decoding - pattern first via kubernetes-secret-yaml; reopens on any confirmed encoded FN across corpus and external targets, closes otherwise. Now blocked by Corpus expansion: cover the shapes the rules are judged on #49 and Recover the 8 gitleaks rules the converter skips #53. Grilled 2026-08-07.

• Corpus expansion - RESOLVED (PR test: per-family detection corpus and gitleaks parity harness (#49, #50) #54): five families, 551 rows, per-family floors enforced in CI. Baselines: urls 0.4783, xml 0.5926, k8s 0.8333, core 0.9881; registry tree 434 findings; leaky-repo 17/43 seeded files.

• gitleaks parity - RESOLVED (PR test: per-family detection corpus and gitleaks parity harness (#49, #50) #54): harness merged, first run dispositioned. generic-api-key exclusion stands measured; pkcs12 recovery has 9 named acceptance files (Recover the 8 gitleaks rules the converter skips #53); the docker auth-blob miss ARMS the Base64-encoded credentials: decode, or document as out of reach #52 trigger; one fidelity question spun out as age-secret-key discrepancy.

Order after grilling: #49 -> #50 -> #53 -> (#51, #44) -> #52 gate check.

Not yet specified

• Toolchain-free installs, parked 2026-08-07: the ar/binutils build requirement comes solely from tree-sitter's C grammars (cc-rs). Two additive fixes when picked up: cargo-binstall metadata pointing at the release assets, and a default-on ast cargo feature so --no-default-features builds pure Rust (ast rules must then refuse with exit 2, per the cannot-evaluate contract).
• Whether generic-rule precision should be defended by cross-rule deduplication in the engine rather than by allowlist patterns in each rule. Every allowlist debt on this map is a symptom; this may be the cause. Cannot be specified until the corpus can measure it.
• Pre-commit / staged-diff mode. Wanted, but the shape depends on whether the cache can key on staged blobs.
• Whether siloscan.toml should be able to tune generic-rule aggressiveness per repository, or whether that is a baseline concern.

Out of scope

• Git history scanning. Deliberately excluded: the target use case is a depth-1 working-tree scan on every commit, where history adds nothing. gitleaks and betterleaks cover the history case; siloscan does not compete there.
• Secret liveness verification. Ruled out by charter - siloscan is offline by mandate, no network ever. This is a deliberate non-feature, not a gap.
• Peak memory ~6x scanned bytes and vendored-tree descent below the repo root. Accepted and documented in the v1.5.x hardening record; revisit only if a user reports them.