Skip to content

feat: rebuild local provenance, security, and review workflow (TAB-18–29) - #47

Merged
hudsonaikins merged 15 commits into
mainfrom
codex/tab-18-29-rebuild
Sep 13, 2026
Merged

feat: rebuild local provenance, security, and review workflow (TAB-18–29)#47
hudsonaikins merged 15 commits into
mainfrom
codex/tab-18-29-rebuild

Conversation

@hudsonaikins

@hudsonaikins hudsonaikins commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Rebuild Tabellio’s local provenance workflow: bounded private metadata, local PostgreSQL persistence, exact Git candidate identity, source-attributed fixture readers, deterministic replay, explicit evidence verdicts, safe review packets, separate security checks, and guarded CLI/GitHub status publication. Child PRs #48–55 are integrated.

The sample uses real Git and PostgreSQL, exercises restart and clean-store replay, and covers 11 failure cases. Provider records and GitHub status transport in the sample remain synthetic. Publication approval consumption uses unique compare-and-swap reservations on a verified private GitHub.com control repository. Security scans cover the entire immutable Git tree, including when invoked from a subdirectory; candidate capture rejects ambiguous merge bases.

Buildkite’s hosted macOS queue was incompatible with the organization’s Personal plan. The existing bootstrap and all pipeline jobs now use linux-small. Per-job setup provides supported Git and PostgreSQL; database-backed validation runs as an unprivileged user. All required checks are retained.

Validated candidate: 30e8c3020bb7f818666c29e2d98395c362135d16. Exact local gate, hosted Product and Quality, real sample demo, four bounded security checks over the application’s immutable content, and native review gate passed. Buildkite #178 passed all six steps. All 25 known inline review findings are fixed and resolved.

Plane: TAB-18 through TAB-29. TAB-21 requires fixture normalization; TAB-27 includes matching blocked CLI/GitHub results. This does not claim complete live-provider provenance, release publication, deployment, a Neon migration, or a web dashboard. Release requires a separate human decision.

hudsonaikins-crown added 2 commits September 12, 2026 15:21
@makeplane

makeplane Bot commented Sep 12, 2026

Copy link
Copy Markdown

Plane-Work-Item: TAB-19
Entire-Checkpoint: 95aae876ef91

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d929ab2781

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread scripts/lib/provenance-ledger.mjs Outdated
Comment thread scripts/lib/provenance-ledger.mjs
Comment thread scripts/tabellio-local-store.mjs Outdated
Comment thread scripts/lib/provenance-ledger.mjs Outdated
Comment thread scripts/lib/provenance-record.mjs
…48)

* feat: normalize candidate-bound provenance source snapshots

Plane-Work-Item: TAB-21
Entire-Checkpoint: 95aae876ef91

* fix: select comparison base for manual quality checks

Plane-Work-Item: TAB-21
Entire-Checkpoint: 95aae876ef91

* feat: replay source evidence with safe review and recovery (TAB-22–29) (#49)

* feat: rebuild provenance from pinned source snapshots

Plane-Work-Item: TAB-22
Entire-Checkpoint: 95aae876ef91

* feat: explicit provenance verdicts and safe review (TAB-23–29) (#50)

* feat: preserve safe source failure reasons in review packets

Plane-Work-Item: TAB-23
Entire-Checkpoint: 95aae876ef91

* feat: exact-candidate packets, security, and review (TAB-24–29) (#51)

* fix: enforce complete review packet bounds and candidate acceptance

Plane-Work-Item: TAB-25

Validates TAB-24 candidate movement and unrelated-history rejection.

Entire-Checkpoint: 7098fc8f9b88

* test: validate review packets with the native schema checker

Plane-Work-Item: TAB-25
Entire-Checkpoint: 7098fc8f9b88

* feat: exact security evidence, review, and recovery (TAB-26–29) (#52)

* feat: bind separate security checks to immutable review candidates

Plane-Work-Item: TAB-26
Entire-Checkpoint: 3a4c8131eec1

* ci: require pinned provenance security scanners

Plane-Work-Item: TAB-26
Entire-Checkpoint: 3a4c8131eec1

* feat: exact review results, safe publication, and recovery (TAB-27–29) (#53)

* feat: show actionable provenance review and GitHub statuses

Plane-Work-Item: TAB-27
Entire-Checkpoint: 3a4c8131eec1

* test: demonstrate failure recovery and secured review (TAB-28/TAB-29) (#54)

* test: demonstrate provenance failure and clean-store recovery (TAB-28)

Entire-Checkpoint: 3a4c8131eec1

* fix: review the secured lineage in the release demo (TAB-29) (#55)

* test: generate unsafe token fixture at runtime (TAB-29)

Entire-Checkpoint: 3a4c8131eec1

* fix: demonstrate review of the secured provenance lineage (TAB-29)

Entire-Checkpoint: 3a4c8131eec1

---------

Co-authored-by: hudsonaikins-crown <hudson.aikins@blanoire.co>

---------

Co-authored-by: hudsonaikins-crown <hudson.aikins@blanoire.co>

---------

Co-authored-by: hudsonaikins-crown <hudson.aikins@blanoire.co>

---------

Co-authored-by: hudsonaikins-crown <hudson.aikins@blanoire.co>

---------

Co-authored-by: hudsonaikins-crown <hudson.aikins@blanoire.co>

---------

Co-authored-by: hudsonaikins-crown <hudson.aikins@blanoire.co>

---------

Co-authored-by: hudsonaikins-crown <hudson.aikins@blanoire.co>

---------

Co-authored-by: hudsonaikins-crown <hudson.aikins@blanoire.co>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 88ed14083a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread scripts/lib/provenance-security.mjs Outdated
Comment thread scripts/lib/provenance-sources.mjs
Comment thread scripts/lib/provenance-ledger.mjs Outdated
@hudsonaikins hudsonaikins changed the title TAB-18–20: add local provenance storage and replay feat: rebuild local provenance, security, and review workflow (TAB-18–29) Sep 12, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aefa341a1b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread scripts/lib/provenance-security-scanners.mjs Outdated
Comment thread .tabellio/validators.json
Comment thread scripts/lib/provenance-ledger.mjs
Comment thread scripts/lib/provenance-record.mjs Outdated
Comment thread scripts/lib/provenance-review-publication.mjs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 79f0861097

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread scripts/tabellio-local-store.mjs Outdated
Comment thread scripts/lib/provenance-review-publication.mjs
Comment thread scripts/lib/provenance-security.mjs Outdated
Comment thread scripts/lib/local-provenance-store.mjs Outdated
hudsonaikins-crown added 2 commits September 12, 2026 20:22

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8cb5218755

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread scripts/lib/provenance-review-publication.mjs
Comment thread scripts/lib/provenance-record.mjs Outdated
Comment thread scripts/lib/provenance-security-scanners.mjs Outdated
Comment thread scripts/lib/provenance-security-scanners.mjs
Comment thread scripts/lib/local-provenance-store.mjs
hudsonaikins-crown added 3 commits September 12, 2026 20:55

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 95ad82e45b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread scripts/lib/provenance-review-publication.mjs Outdated
Comment thread scripts/lib/provenance-security-scanners.mjs Outdated
Comment thread scripts/lib/provenance-ledger.mjs Outdated
hudsonaikins-crown added 3 commits September 12, 2026 23:41
Plane-Work-Item: TAB-18
Entire-Checkpoint: ab2c360fdf7c
Plane-Work-Item: TAB-18
Entire-Checkpoint: ab2c360fdf7c
Plane-Work-Item: TAB-18
Entire-Checkpoint: ab2c360fdf7c
@hudsonaikins
hudsonaikins merged commit c7c689e into main Sep 13, 2026
5 checks passed
@hudsonaikins
hudsonaikins deleted the codex/tab-18-29-rebuild branch September 13, 2026 03:56

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 30e8c3020b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

requireFact(pr?.number === context.selection.pullRequestNumber, "record_missing");
requireFact(snapshot.repositoryId === context.candidate.repositoryId, "scope_mismatch");
requireFact(pr.source?.commit === context.candidate.headCommit && pr.target?.commit === context.candidate.baseCommit, "candidate_mismatch");
requireFact(pr.state === "open" && !pr.draft, "review_not_ready");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Require the draft flag to be exactly false

When an imported GitHub snapshot contains a malformed falsy draft value such as null or 0, !pr.draft treats it as proof that the pull request is ready, so an otherwise matching approval produces a passed review observation and can satisfy the gate once security evidence is added. Require pr.draft === false or validate the complete GitHub snapshot contract before accepting it.

Useful? React with 👍 / 👎.

if (files.size >= 4096 || totalBytes > 32 * 1024 * 1024) throw new Error("Candidate exceeds security scan bounds.");
const target = join(directory, path);
await mkdir(dirname(target), { recursive: true });
await writeFile(target, blob.stdout, { mode: 0o600 });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Detect filesystem aliases before materializing scan files

On the supported default case-insensitive macOS filesystem, a Git tree containing distinct paths such as A.js and a.js causes these writes to target the same file; because tree order can write a clean blob after a secret-bearing blob, both scanners inspect only the clean content and can issue a passing receipt. The fresh evidence in this revision is that fatal UTF-8 decoding still materializes valid names directly and therefore does not prevent case or Unicode-normalization aliases; reject colliding paths or scan each blob in an alias-free namespace.

Useful? React with 👍 / 👎.

const review = await readInput(options.input);
const secured = attachSecurityReview({ lineage, review, policyDigest: options.policyDigest, now });
const digest = await store.putLineage(secured);
await writeJsonOutput({ status: review.status, candidate, digest, securityDigest: review.digest }, options.out);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Protect provenance outputs from input aliases

When import-security receives an --out path that directly, through a hard link, or through a symbolic link aliases its --input receipt, this write succeeds after the database import but replaces the full security receipt with the short import summary. The stored lineage retains only the derived security observation, so the original findings can no longer be supplied to review --security-input; validate output paths against every input before writing, as the other evidence CLIs do.

Useful? React with 👍 / 👎.

"artifactCount"
],
"properties": {
"id": {"type": "string", "pattern": "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$"},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Version the Buildkite schema before adding build IDs

When an existing consumer validates new collector output with the previously published buildkite-build-snapshot/v0.1 schema, the newly emitted id field is rejected because that version's build object used additionalProperties: false and did not declare this property. Changing the closed v0.1 contract in place makes producer output incompatible with consumers pinned to the same advertised schema version; preserve v0.1 and introduce a new schema version for snapshots containing build IDs.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant