diff --git a/.gitleaksignore b/.gitleaksignore index 9e90786..2b9033b 100644 --- a/.gitleaksignore +++ b/.gitleaksignore @@ -6,3 +6,41 @@ 4f94b88909f5c8951027b11bd862e91edb8bd1b1:DEPLOYMENT.md:curl-auth-user:232 4f94b88909f5c8951027b11bd862e91edb8bd1b1:DEPLOYMENT.md:curl-auth-user:317 4f94b88909f5c8951027b11bd862e91edb8bd1b1:DEPLOYMENT.md:curl-auth-user:320 + +# --- Issue #63: tag-only commits from the pre-history-scrub v0.2.0 / +# v0.2.0-beta tags. actions/checkout@v4's fetch-depth:0 force-updates tags +# (+refs/tags/*), so CI scans these abandoned tag tips, not main. Each of +# the 5 duplicate entries below is byte-identical content to an +# already-ignored main-ancestor commit above (same author/date/diff), just +# under the pre-rewrite hash. Each of the 7 was individually verified live +# or dead before being added here -- see SECRET_SCAN_CI_INVESTIGATION.md +# and SECRET_SCAN_RESOLUTION.md for the full trail. + +# github-fine-grained-pat, .env:15 -- ROTATED/REVOKED. Verified: confirmed +# absent from the current GitHub token list entirely (only one unrelated +# fine-grained PAT exists today, different purpose, different creation +# date). +979ba0c0cb4ced0e233e38ef18436e2c54db71c6:.env:github-fine-grained-pat:15 + +# generic-api-key (RAGBIO_API_KEY), .env:8 -- ROTATED. Verified: direct +# comparison of the leaked value against the current live value confirms +# they differ. +cced81dacacf28eed07a4426fe4b116295573507:.env:generic-api-key:8 + +# jwt, docker-compose.yml:232 -- dead. Verified: JWT payload decoded from +# the leaked token, exp claim (~2026-05-25) has passed. Duplicate content +# of already-ignored 8e35ae534a8 above. +981226817e716889fbbdcd213f7010af17e6d47b:docker-compose.yml:jwt:232 + +# jwt, docker-compose.release.yml:213 -- dead. Verified: JWT payload +# decoded from the leaked token, exp claim (~2026-05-25) has passed. +# Duplicate content of already-ignored 9a12a77d00b above. +11c868fb5436a7aea541120ab4f64962c36706b8:docker-compose.release.yml:jwt:213 + +# curl-auth-user (Grafana/Prometheus basic-auth), DEPLOYMENT.md:232,317,320 +# -- different/safe. Verified: direct comparison confirms the current +# GF_ADMIN_PASSWORD is not the leaked "omnibioai" default. Duplicate +# content of already-ignored 4f94b88909f above. +975460f350e777340dc2d83911f9c565f18c12c8:DEPLOYMENT.md:curl-auth-user:232 +975460f350e777340dc2d83911f9c565f18c12c8:DEPLOYMENT.md:curl-auth-user:317 +975460f350e777340dc2d83911f9c565f18c12c8:DEPLOYMENT.md:curl-auth-user:320 diff --git a/SECRET_SCAN_CI_INVESTIGATION.md b/SECRET_SCAN_CI_INVESTIGATION.md new file mode 100644 index 0000000..3feaf79 --- /dev/null +++ b/SECRET_SCAN_CI_INVESTIGATION.md @@ -0,0 +1,116 @@ +# Secret-Scan CI Investigation + +Status: **Part 1 resolved. Part 2 blocked — needs your decision before any +`.gitleaksignore` change is made. Part 3 not started. Part 4 done separately +(see bottom).** + +## Part 1 — Commit-count discrepancy: RESOLVED + +Reproduced CI's exact result locally: `450 commits scanned` / `leaks found: 7` +(count differs slightly from CI's 437 only because my dev clone still carries +a few extra local-only branches beyond what a fresh CI clone has — see below; +this has no effect on which leaks are found, confirmed). + +**Root cause, with evidence:** + +1. `scripts/secret-scan.sh` doesn't take a ref — it mounts the whole `.git` + directory and runs `gitleaks git /repo`. Gitleaks' git-mode walks **every + ref in the repo** (branches *and* tags), not just the checked-out branch. +2. CI's `actions/checkout@v4` with `fetch-depth: 0` fetches with + `+refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/*` — confirmed + from the actual job log (`gh api .../jobs/99201926572/logs`). The leading + `+` **force-updates every tag**, even a conflicting one. +3. My day-to-day local clone had **stale copies of the `v0.2.0` and + `v0.2.0-beta` tags**. Running the identical fetch CI uses produced: + `t [tag update] v0.2.0 -> v0.2.0` and `v0.2.0-beta -> v0.2.0-beta` — i.e. + my local tags were pointing somewhere else and got force-moved. A plain + `git fetch`/`git pull` never force-moves a conflicting tag, so an ordinary + dev clone silently keeps drifting from what CI actually scans. +4. Once local tags matched origin exactly, the local scan reproduced CI's + `leaks found: 7` immediately. +5. All 7 findings' commits are **unreachable from any branch** (`git branch + --all --contains` = empty for all seven) but **are reachable from the + `v0.2.0` / `v0.2.0-beta` tags** (`git tag --contains` confirms). They are + not part of `main`'s real history. +6. For 5 of the 7, I diffed the tag-only commit against the commit already + named in `.gitleaksignore` for the same file/rule/line — same author date, + same commit message, byte-identical `--stat` (same files, same + insertions/deletions) in every pair checked: + - `981226817e7...` (tag-only) ≡ `8e35ae534a8...` (already in + `.gitleaksignore`, ancestor of `main`) — both "feat: integration tests + - 121 passing, 0 failures", identical 9-file/1670-insertion diff. + - `11c868fb543...` (tag-only) ≡ `9a12a77d00b...` (already ignored) — + identical diff (License gate + UI changes). + - `975460f350e...` (tag-only, 3 findings) ≡ `4f94b88909f...` (already + ignored) — identical diff (Grafana/Prometheus datasource + Workbench + UI changes). + + Conclusion: at some point `main`'s history was rewritten (commit hashes + changed for identical content — the parent chain differs, so every + descendant hash changed too), which is exactly why `main`'s current + history is clean (my very first local scan, before touching tags, found + **0 leaks in 389 commits**). The `v0.2.0`/`v0.2.0-beta` tags were never + moved forward and still point at the pre-rewrite snapshot, so they + permanently retain the old secret-containing blobs. This is *not* related + to the `backup/main-before-pr51-rebase` branch (checked — unrelated, much + later, Aug 16) — it's a separate, earlier rewrite. + +**Net effect:** CI is not scanning `main`. It's (unintentionally) scanning +two abandoned release tags that predate a history-scrub, every single run. + +## Part 2 — Triage of the 7 findings: 5 of 7 have a provenance answer, liveness is NOT verified for any + +| # | File | Rule | Tag-only commit | Status | +|---|------|------|------|--------| +| 1 | `.env` | `github-fine-grained-pat` | `979ba0c0c` (2026-05-28) | **New** — no prior `.gitleaksignore` entry | +| 2 | `docker-compose.yml:232` | `jwt` | `981226817` (2026-05-25) | Duplicate of already-ignored `8e35ae5` (see above) | +| 3-5 | `DEPLOYMENT.md:232,317,320` | `curl-auth-user` | `975460f35` (2026-05-22) | Duplicate of already-ignored `4f94b8890` (Grafana/Prometheus basic-auth) | +| 6 | `.env` (`RAGBIO_API_KEY`) | `generic-api-key` | `cced81dac` (2026-05-22) | **New** — no prior `.gitleaksignore` entry | +| 7 | `docker-compose.release.yml:213` | `jwt` | `11c868fb5` (2026-05-27) | Duplicate of already-ignored `9a12a77d0` | + +Structural checks that came back clean: +- Current `main` no longer hardcodes any of these — `docker-compose.yml`'s + `JWT_SECRET`/`AUTH_SECRET_KEY` is `${AUTH_SECRET_KEY:?...}`, and `.env` is + untracked + gitignored on `main`. +- The existing `.gitleaksignore` comment ("Reviewed historical findings...") + is a **blanket** justification, not five individual ones — it doesn't say + *why* each is safe (rotated? fixture? placeholder?), so I can't treat it as + the individual verification Part 2 requires even for the 5 duplicates. + +**What I could not verify, and why I stopped here rather than guess:** +Whether any of these five values — a GitHub fine-grained PAT, a +`RAGBIO_API_KEY`, a JWT signing secret (x2), and Grafana/Prometheus +basic-auth credentials — are still valid *today* is not something I can +determine from the repository alone, and I did not test any of them against +a live service (I don't have the plaintext — I ran everything with +`--redact=100` — and testing a possibly-real credential against a live +system without your explicit sign-off is exactly the kind of decision your +instructions asked me to bring to you rather than automate). The fact that +current `main` has since parameterized these files is evidence someone +*touched* this area, but it doesn't by itself prove the underlying secret +values were rotated at the service that issued them. + +**I'm stopping here per your instructions** ("if any finding is a +genuinely live... credential: STOP... credential rotation is a decision I +need to make explicitly" and "do not add anything to `.gitleaksignore` you +haven't individually verified"). I have not confirmed any of the 7 as +definitely live, but I also can't confirm any as definitely dead — so I'm +not writing `.gitleaksignore` entries or touching Part 3 until you tell me +whether these five secrets (PAT, RAGBIO key, 2x JWT secret, Grafana/ +Prometheus creds — all dated 2026-05-22 through 2026-05-28) have been +rotated since. If you confirm rotation/dead, I'll write the five individual +justification lines and proceed to Part 3. If any turns out to still be +live, tell me and I'll stop there entirely per your instructions. + +## Part 3 — Not started (blocked on Part 2 above). + +## Part 4 — PR #67 + +Independently verified before merging (didn't just take the task description +on faith): `gh pr view 67 --json files,commits,mergeable` shows a single-file, +7-line, 0-deletion, comment-only addition to `docker-compose.yml` (a `TODO` +above the `rag` service's now-unused `GITHUB_TOKEN`/`RAGBIO_API_KEY` build +args, referencing `omnibioai-rag#25`), `mergeable: MERGEABLE`, diff matches +its own description exactly. No code/behavior change, no secrets touched. + +Merged separately — see below. diff --git a/SECRET_SCAN_RESOLUTION.md b/SECRET_SCAN_RESOLUTION.md new file mode 100644 index 0000000..174a16d --- /dev/null +++ b/SECRET_SCAN_RESOLUTION.md @@ -0,0 +1,77 @@ +# Secret-Scan CI Resolution (Issue #63) + +Follow-up to `SECRET_SCAN_CI_INVESTIGATION.md`, which diagnosed the root +cause (Part 1) and left Part 2 blocked on live-credential determinations +only the repo owner could make. This document records those +determinations and the resulting fix. + +## Root cause (recap) + +`actions/checkout@v4` with `fetch-depth: 0` force-updates every tag +(`+refs/tags/*`). The `v0.2.0` / `v0.2.0-beta` tags were never advanced +past a pre-history-rewrite snapshot, so CI's `gitleaks git /repo` (which +walks every ref, not just `main`) permanently rescans two abandoned tag +tips instead of `main`'s real, clean history. `main` itself has 0 leaks in +389 commits. + +## Live-credential determinations (owner-verified) + +Each of the 7 findings was individually checked — no blanket +justification. Methods used: + +| # | Commit | File:Line | Rule | Status | Verification method | +|---|--------|-----------|------|--------|----------------------| +| 1 | `979ba0c0c` | `.env:15` | github-fine-grained-pat | Rotated/revoked | Absent from current GitHub fine-grained token list entirely | +| 2 | `cced81dac` | `.env:8` | generic-api-key (RAGBIO_API_KEY) | Rotated | Direct value comparison: current live value differs from leaked value | +| 3 | `981226817` | `docker-compose.yml:232` | jwt | Dead | JWT payload decoded from leaked token; `exp` ≈ 2026-05-25, already past | +| 4 | `11c868fb5` | `docker-compose.release.yml:213` | jwt | Dead | JWT payload decoded from leaked token; `exp` ≈ 2026-05-25, already past | +| 5-7 | `975460f35` | `DEPLOYMENT.md:232,317,320` | curl-auth-user (Grafana/Prometheus) | Different/safe | Direct comparison: current `GF_ADMIN_PASSWORD` is not the leaked `omnibioai` default | + +Findings 3-7 are also byte-identical content (same author, date, diff) to +commits already covered by pre-existing `.gitleaksignore` entries +(`8e35ae534a8`, `9a12a77d00b`, `4f94b88909f`) — those were the correct +main-ancestor commits before the history rewrite changed every descendant +hash. The tag-only commits above are the same secrets under their +pre-rewrite hashes, which is why the old entries didn't suppress them. + +Note: the RAGBIO_API_KEY's current live value was independently rotated a +second time during this investigation, for an unrelated reason (accidental +exposure in a separate chat session). That second rotation isn't reflected +in `.gitleaksignore`'s justification comment — it's not something this +specific git-history finding required, and is mentioned here only for the +record. + +## Fix applied + +Added 7 individually-justified entries to `.gitleaksignore` (see that +file's comments for the per-entry rationale and evidence), one per +fingerprint above. Kept the 5 pre-existing entries as-is. + +## Verification + +1. Re-fetched with CI's exact refspec + (`+refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/*`) to rule + out the stale-local-tag false-clean failure mode described in Part 1 of + the investigation doc. Confirmed local `v0.2.0` / `v0.2.0-beta` already + matched `origin` exactly (no drift this time). +2. Ran a fresh, unfiltered `gitleaks git` scan (no ignore file) directly + against the repo to pull real fingerprints/line numbers from the tool + itself rather than transcribing them — all 7 matched the investigation + doc and the table above exactly. +3. Ran `scripts/secret-scan.sh` (the actual CI step) with the updated + `.gitleaksignore`: `450 commits scanned`, **no leaks found**, exit 0. + +## Artifacts + +- `gitleaks-report.json` (untracked, `[]` — a clean-`main`-only scan with + zero findings) deleted; nothing in it needed preserving. +- `SECRET_SCAN_CI_INVESTIGATION.md` and this file committed as the + permanent record of the investigation and resolution. + +## Remaining steps + +- Open a PR on `fix/gitleaks-issue-63-tag-history-secrets` referencing + issue #63, confirm CI goes green there. +- Comment on issue #63 with this resolution once the PR is up. +- Close issue #63 after the PR merges (not before — merge is a human + decision, not automated here).