Enforce resource quotas for untrusted sandboxes

[timwhitez]

Summary

• launch proposer, candidate test/build, Loader, and packed-overlay work in fresh delegated cgroup v2 domains with frozen memory, swap, CPU, PID, and I/O limits plus CPU/file/open-file rlimits
• replace writable host binds and unbounded scratch paths with size/inode-bounded tmpfs mounts, trusted seed/export handling, read-only root and /dev, disabled nested user namespaces, empty target capabilities, and whole-domain teardown
• isolate each untrusted target below a private descendant PID namespace so it cannot see or signal the trusted Node supervisor or reach its control descriptor
• require semantic success receipts at admission, resume, cache, and audit: exact frozen policy/mounts/PID isolation, complete bounded peaks, zero limit events, and COMPLETED with exit 0/no signal
• atomically commit worker/resource/gateway/diagnostic evidence behind a manifest-last marker; fsync the exported child tree and every replacement boundary before that commit
• quarantine incomplete or mismatched cache/execution/tree pairs plus interrupted publication residues, then replay durable gateway ids from immutable input
• derive V011 inventory and semantic replay from one canonical active-action scan; reject symlink, hardlink and special entries while keeping quarantine de-authorized
• verify exact materialization wrappers and CAS bytes on cache/build/audit; publish materialization only by fsynced staging, no-clobber link and directory fsync
• keep stable-proposer durable gateway requests outside its commit directory and quarantine every manifest-less partial bundle before replay
• fsync reservation and completion files/directories around paid dispatch, publish only a fully staged manifest, and revalidate exact stable proposal bundles in audit

Independent review blocker resolution

Earlier reviews found and drove fixes for semantic resource receipts, the same-UID supervisor boundary, proposal publication/CAS/staging crash windows, manifest-less stable bundle recovery, disk durability around gateway and manifest publication, exact gateway schema/route validation, completed-receipt terminal semantics, and replay of every V011 materialization rather than only retained generations.

Exact-head review of aeaea55246415f522cdfccde0bdc0dceea7b6f0c found two crash/evidence gaps: an extra manifest-committed execution without a materialization was outside audit enumeration, and trusted child export returned without fsyncing files/tree/parent replacement boundaries. 4224a080c417265d9abe91cb55639094286cb2c1 added active manifest/materialization inventory, durable child export, installed-tree digest binding and committed execution/tree recovery.

Exact-head review of 4224a080c417265d9abe91cb55639094286cb2c1 then found that semantic replay rescanned quarantined materializations after inventory, active symlink/special entries were silently ignored, and a present or torn materialization cache bypassed recovery. 38eb413ef7b80f0f664d7d037db07611f9cb9fc5 changed inventory and replay to one direct canonical action scan, rejected non-canonical active entries, and added fsynced no-clobber cache publication plus pre-adoption recovery.

Exact-head review of 38eb413ef7b80f0f664d7d037db07611f9cb9fc5 found two remaining fail-open paths: an inaccessible staging cleanup probe was converted to “absent”, and a cache inode with an external hard-link alias could be adopted. f6a6754f0d996ad079f1d9e2ee210688d252ad18 propagates staging inspection/cleanup failures, requires a regular single-link cache, reads it through an O_NOFOLLOW held descriptor with stable inode/metadata checks before and after the read and adoption, and quarantines any multi-link cache with its execution/tree. Multi-link quarantine bytes are copied into fresh fsynced inodes so an external alias cannot mutate retained evidence; durable gateway request state remains active for deterministic replay.

Exact-head review of f6a6754f0d996ad079f1d9e2ee210688d252ad18 then found publication could still resolve after the action directory was renamed/recreated or the final authority name was removed after the first directory fsync. Exact head 8598ade5b0968594500e070b49ef0abf7f175f2d holds the original directory and staging descriptors through cleanup, requires the requested directory to retain the same dev/inode, verifies the final path is the exact stable single-link staged inode with unchanged metadata and bytes, fsyncs the held directory, and repeats those checks before returning. Regressions cover both directory replacement and final-name removal. A fresh exact-head independent review remains required.

Validation

• format, docs, lint, and typecheck pass
• unit: 119 files, 912 passed, 1 platform skip
• no-key E2E: 17 files passed plus 2 credential-gated skips; 44 passed plus 4 skipped
• targeted regressions: V011 proposal execution/cache 16/16; related cache/audit unit 38/38; real crash/export E2E 14/14
• real Harbor ACP, three extract-elf outcomes, offline Loader, crash/replay, V011 admission, memory/fork/CPU/storage and namespace/write-boundary probes pass
• provenance, upstream-clean, byte-equality, and release-readiness pass (431 tracked UTF-8 files, 112 links)

This is a denial-of-service containment/evidence repair and makes no benchmark or promotion claim.

Closes #51

[timwhitez]

INDEPENDENT_REVIEW: REQUEST_CHANGES

Exact head reviewed: d3604d916ee02804282653bc52380838d0676436 (base 6324afdbce7b1fb64ee7efef34d1ad1b1d6d3022). I reviewed the issue contract, AGENTS/specs, ADR-025, the complete diff, production call sites, tests, and exact-head hosted CI.

Hosted CI run 33170905762 is SUCCESS for this exact SHA: 854 unit tests passed; E2E reported 42 passed / 4 credential-gated skipped; documentation, format, lint, typecheck, provenance, upstream-clean, byte-equality, and release-integrity steps all succeeded. The green suite does not cover the blocking success/failure invariant below.

Blocking findings:

1. A positive V011 admission contains a failed packed-overlay resource receipt. After the ACP handshake succeeds, runPackedOverlayProbe destroys stdin and immediately calls sandbox.kill('COMPLETED'). That whole-domain SIGKILL prevents the trusted supervisor from emitting its control record; spawnResourceBoundSandbox.finish correctly converts the missing control record to CONTROL_PROTOCOL_FAILURE. However, admitV011Candidate hashes that failed receipt and still returns admitted: true; it never requires every successful admission stage to have a complete COMPLETED receipt.

   Read-only exact-head reproduction used the same baseline/runtime/overlay fixture as packages/candidate-sdk/tests/v011-admission.e2e.ts:99-170, invoked admitV011Candidate, and printed the returned causes:

   {
     "admitted": true,
     "causes": {
       "candidateTests": "COMPLETED",
       "builds": ["COMPLETED", "COMPLETED"],
       "loaderSolve": "COMPLETED",
       "loaderPropose": "COMPLETED",
       "packedOverlayBoot": "CONTROL_PROTOCOL_FAILURE"
     },
     "packed": {
       "writableStoragePeakBytes": null,
       "writableStoragePeakFiles": null,
       "terminationCause": "CONTROL_PROTOCOL_FAILURE",
       "exitCode": null,
       "signal": "SIGKILL"
     }
   }

   The existing admission E2E still passes because it asserts the ACP/admission fields but never asserts result.packedOverlay.resource.terminationCause or complete storage peaks.

2. Resume/audit validates only a digest-shaped wrapper, not the required resource evidence. readV011StableBuild and v011-audit check only digestV011(resource) and candidateDigest; there is no strict resource-receipt schema/validator and no check of policy id/digest, limits, enforcement, peaks/events, or successful stage termination. The positive unit fixture at packages/dsh-self-evolving-cli/tests/v011-identity.test.ts:35-60 deliberately uses {schemaVersion:1,candidateDigest,fixture:true}—with none of the issue-required evidence—and readV011StableBuild accepts it. The V011 proposer resource is likewise only placed in proposal-diagnostic.json; cached materialization at v011-real-capabilities.ts:720-723 does not require or validate it, and the audit does not consume it.

These behaviors violate issue #51's requirement to retain auditable configured limits, peaks, and termination cause, and spec 06 §10's explicit rule that a missing/corrupt control receipt cannot be interpreted as normal completion and successful publication must content-bind the resource receipt.

Required before approval:

• make successful packed-overlay shutdown yield a complete supervisor receipt (or otherwise model planned teardown without losing the trusted control record), and reject admission unless all success-stage receipts are structurally complete and have the permitted successful termination semantics;
• add a versioned strict schema/runtime validator for resource receipts and enforce it in production, resume, and audit, including policy digest recomputation, frozen limits/enforcement, metrics/events, and stage termination;
• content-bind and require proposer resource evidence on replay/audit;
• add regression tests that (a) assert every positive admission stage is COMPLETED with complete peaks, (b) make lost/malformed supervisor control fail admission, and (c) make resume/audit reject minimal, malformed, policy-mismatched, or failed receipts.

Please do not merge or close #51 on the current head.

[timwhitez]

INDEPENDENT_REVIEW: REQUEST_CHANGES

Exact head reviewed: 7812f2ee8cdfe79f5bfec5fd1e501150f01ec1d4 (PR #246). Hosted CI run 33179507467 is SUCCESS and its headSha is this exact SHA; documentation, format, lint, typecheck, unit, Loader/local E2E, provenance and release-integrity steps all completed successfully.

Review scope: Issue #51 resource-quota contract; the d3604d9 failed packed-overlay receipt plus weak resume/audit/proposer binding; the a88a6f1 same-UID supervisor boundary, proposal execution atomicity, materialization wrapper/CAS audit and baseline/generated staging-claim lifecycle; exact-head production callers, tests and regressions.

Verified improvements:

• local candidate build/test/Loader/packed-overlay and proposer paths now use frozen cgroup-v2 memory/swap, CPU, PID, I/O and bounded-storage controls with semantic receipts;
• packed-overlay success now exits naturally and admission/resume/audit require complete COMPLETED/exit-0/no-signal receipts;
• the target runs below a private descendant PID namespace; the exact E2E observes PID 1/PPID 0 and no supervisor visibility;
• V011 execution evidence is manifest-last, resource-bound, and materialization/cache/audit now cross-check the exact wrapper plus materialization/analysis CAS bytes;
• candidate staging removes and fsyncs its claim before rename, then fsyncs the publication parent.

Independent exact-head validation passed: 6 targeted unit files / 22 tests, and 3 targeted E2E files / 11 tests (resource bombs, PID isolation, V011 admission and proposal sandbox).

Blocking regression:

publishBundle now commits sorted data entries with no-clobber hard links and rejects any pre-existing final entry. That protects a completed bundle, but the production stable-demo realProposal treats an absent manifest as retryable unless proposal.json already exists. Because the sorted publication order starts with gateway-receipts.json and idempotency-key.json, a crash after either early link but before proposal.json/manifest is not quarantined or adopted. Resume re-runs the proposal and the later publication call fails permanently on EEXIST.

A read-only exact-head fault-state reproduction produced:

{"manifestBefore":null,"retryError":"publish: final path already exists: .../gateway-receipts.json"}

The added V011 boundary tests quarantine children/execution residue, but no test covers the other production caller of the modified shared publisher. This is a new crash-recovery regression and conflicts with the repository requirement that uncommitted proposal residue be recoverable without treating it as authority.

Required before approval:

1. Preserve no-clobber protection while making stable proposal pre-manifest states recoverable (for example, quarantine the whole incomplete bundle or safely verify/reuse the exact uncommitted bytes under a defined single-writer protocol).
2. Add production-path crash tests for every sorted data-entry boundary before the manifest, including states where only gateway-receipts.json or the first two files exist.
3. Run exact-head hosted CI and obtain a fresh independent review.

Do not merge PR #246 or close Issue #51 on this head.

[timwhitez]

INDEPENDENT_REVIEW: REQUEST_CHANGES

Exact head reviewed: 06c12184686df4cf86ae4403227b403777b3035e (PR #246, base 6324afdbce7b1fb64ee7efef34d1ad1b1d6d3022). Hosted CI run 33180663954 is SUCCESS on this exact SHA: 117 unit files / 874 tests passed; no-key E2E reported 17 files passed plus 2 credential-gated skips, 42 tests passed plus 4 skipped; provenance, upstream-clean and byte-equality checks also passed.

Independent read-only validation passed:

• 11 targeted unit files / 47 tests: stable/V011 publication, materialization/CAS, resource receipt/audit, build claim, shared publisher and gateway restart idempotency.
• 3 targeted real E2E files / 11 tests: memory, PID, CPU and storage boundaries; descendant PID namespace; V011 admission/Loader/packed overlay; proposal sandbox.
• A direct migration/replay probe covered all four sorted stable bundle boundaries (gateway-receipts.json, idempotency-key.json, proposal.json, sandbox-resource.json). Each legacy durable response replayed after quarantine with paid_calls_each=1.
• git diff --check origin/main...HEAD passed.

The exact-head repair closes the prior process-restart EEXIST regression: manifest-less stable proposal directories are moved aside, legacy gateway state is migrated outside the publication directory, and a normal process restart replays the completed response. The resource/PID namespace, semantic receipt, execution-bundle, materialization/CAS and staging changes also pass their targeted tests.

Three durability/audit blockers remain under the repository contract, which explicitly includes partial disk writes and requires durable intent before side effects:

1. The provider request reservation is not durable before the paid side effect. reserveDurableRequest uses writeFile with exclusive create and immediately returns; completeDurableRequest writes and renames without file or parent-directory fsync. The handler is dispatched immediately after that reservation. A host/power crash can therefore lose the only pending record after a paid request started, allowing restart to dispatch the same request again. The new migration has the same first-use gap: it creates proposal-gateway-requests and incomplete-proposal-publications, but recoverIncompleteStableProposalPublication never fsyncs stateDir, so the newly created parent entries are not crash-durable even though their child directories are synced.

2. The shared commit marker is still not atomically published. publishBundle opens the final publish-manifest.json path and writes into it directly. A crash between open/write/fsync can leave a present truncated marker. loadPublishedBundle returns null only when the marker is absent; a torn marker throws before stable recovery can quarantine it. Thus one manifest-write crash still permanently strands the action.

3. Stable final audit does not replay proposal resource evidence. auditStableRun validates generation build receipts, while its proposal check checks only journal evidence references. It never loads the stable proposal bundle or semantically validates sandbox-resource.json. Missing or self-consistently corrupted proposal resource evidence can therefore leave the stable audit unchanged, contrary to spec 06 section 10 and the earlier review requirement that resume and audit both validate receipt bytes, semantics and binding.

Required before approval:

1. Make gateway reservation/completion and first-use directory creation genuinely durable: file fsync, atomic no-clobber publication where applicable, and fsync every affected parent directory before paid dispatch or successful return.
2. Publish the manifest through a fsynced staging file plus atomic no-clobber commit; recover/quarantine every pre-commit boundary, including a torn final-marker scenario.
3. Make stable audit enumerate the exact completed proposal actions, load each committed bundle, bind its idempotency/proposal identity to the journal event, and run the strict resource validator.
4. Add fault-injection tests at reservation file sync, directory sync, migration rename, manifest publish and audit-tamper boundaries; prove zero duplicate paid dispatch.

Review was independent and read-only. I did not edit files, commit, push, merge, close the issue, or rely on the implementation agent for the conclusion. Do not merge PR #246 or close Issue #51 on this head.

[timwhitez]

INDEPENDENT_REVIEW: REQUEST_CHANGES

Exact head reviewed: fb09740d7aa8d1edea002909355fbf549a4eae34 (PR #246, base 6324afdbce7b1fb64ee7efef34d1ad1b1d6d3022). Hosted CI run 33183672621 is SUCCESS and its headSha is this exact commit. Documentation, format, lint, typecheck, unit, real Loader/local E2E, provenance, upstream-clean and release-integrity steps all succeeded.

Independent read-only validation passed:

• 11 targeted unit files / 52 tests: gateway reservation/completion durability, manifest publication, stable publication recovery/audit, stable/V011 resource receipts, V011 execution/CAS and staging claims.
• 4 real E2E files / 15 tests: memory/PID/CPU/storage containment, descendant PID namespace, V011 admission with real Loader/packed overlay, proposal isolation, and process crash/replay.
• git diff --check origin/main...HEAD passed and the worktree remained clean.

The exact head fixes the prior blockers around cgroup/resource enforcement, trusted-supervisor PID isolation, semantic resource validation, V011 execution/materialization authority, stable pre-manifest quarantine, gateway file/directory fsync ordering, atomic manifest publication, and stable resource-bundle replay.

One blocking audit bypass remains in the new stable gateway-bundle verifier. validGatewayReceipt in packages/dsh-self-evolving-cli/src/audit.ts:42-58 validates only that requestId is a string, the three hashes are syntactically shaped, and attempts is an array. It does not require a non-empty request id, validate any AdapterFetchAttempt entry, enforce success/failure field consistency, require a successful receipt, or bind routeHash to the frozen route in the run config.

A read-only exact-head reproduction published a manifest-valid proposal/resource/idempotency bundle whose only gateway receipt was:

{
  "requestId": "",
  "requestHash": "sha256:4444444444444444444444444444444444444444444444444444444444444444",
  "responseHash": "sha256:5555555555555555555555555555555555555555555555555555555555555555",
  "routeHash": "sha256:6666666666666666666666666666666666666666666666666666666666666666",
  "attempts": [null],
  "error": ""
}

verifyStableProposalPublications(...) returned [] and therefore accepted it. This receipt cannot be emitted by the declared ProposalGatewayReceipt / AdapterFetchAttempt contract and does not prove any successful, frozen-route provider call. A self-consistently corrupted or producer-invalid gateway bundle can therefore pass final audit, contrary to the exact gateway-bundle replay claimed in spec 05 section 5.2, spec 06 section 10, ADR-025, and this PR.

Required before approval:

1. Add one strict runtime gateway-receipt validator: exact keys, non-empty bounded request id, exact hash forms, exact AdapterFetchAttempt entry schema/types/ranges, and coherent success/failure fields.
2. Require at least one successful receipt and bind every receipt routeHash to the frozen provider/endpoint/model/reasoning/max-token route from the validated run config.
3. Add negative stable-audit tests for empty ids, malformed attempt entries, failure-only matrices, wrong route hashes and extra fields; rerun exact-head hosted CI and obtain a fresh independent review.

Review was independent and read-only. I did not edit files, commit, push, merge, or close the issue. Do not merge PR #246 or close Issue #51 on this head.

[timwhitez]

INDEPENDENT_REVIEW: REQUEST_CHANGES

Exact head reviewed: 13d497660babf35c737a08a583cf9ba050c36070 (base 6324afdbce7b1fb64ee7efef34d1ad1b1d6d3022). I reviewed Issue #51, AGENTS/specs 00–07 safety/evidence/acceptance requirements, the complete 50-file PR diff, production callers, tests, and every prior independent blocker.

Hosted CI run 33186141043 is SUCCESS; its headSha is this exact commit. Documentation, format, lint, typecheck, unit, real Loader/local E2E, provenance, upstream-clean, byte-equality and release-integrity steps all completed successfully.

Independent exact-head validation passed:

• pnpm lint and pnpm typecheck.
• 11 targeted unit files / 65 tests: stable/V011 publication and audit, gateway durability/attempt accounting, materialization/CAS, semantic resource receipts, identity and staging paths.
• 4 targeted real E2E files / 15 tests: memory/PID/CPU/storage containment, descendant PID namespace, real Loader/packed overlay admission, proposal isolation, and process crash/replay.
• git diff --check origin/main...HEAD; the worktree remained clean.

The latest patch does close the four concrete fb09740 examples: stable audit now rejects an empty request id, attempts: [null], an empty error, and a foreign route hash. The earlier resource-domain, same-UID/PID isolation, semantic receipt, manifest-last, durable gateway reservation/completion, partial-publication recovery, CAS/materialization and staging-claim blockers also appear repaired on this head.

Two blocking audit gaps remain:

1. Stable completed proposals still accept receipt-level impossible/failure-only matrices. validGatewayAttempt validates each row in isolation, while validGatewayReceipt does not discriminate success from failure or require a successful receipt for a proposal.completed bundle. An independent /tmp-only probe published manifest-valid, resource-valid bundles and got no audit reasons for all three cases:

   failure_only=[]
   success_with_error=[]
   nonretryable_then_success=[]
   

   The cases were respectively: one 503 attempt plus non-empty error; one 200 attempt plus non-empty error; and an impossible 400 non-retryable attempt followed by 200. The trusted producer cannot emit the latter two sequences, and a completed proposal cannot be proven by a failure-only matrix. This is the unclosed part of the prior review requirement to enforce coherent success/failure fields and require at least one success.

2. The V011 final audit still treats gateway evidence as an arbitrary array. loadV011ProposalExecution checks only Array.isArray(gatewayReceipts), and assertV011ProposalExecutionBinding binds only its count. A second independent probe published gatewayReceipts: [null]; load plus materialization/execution binding accepted it (v011_malformed_gateway_accepted=[null]). auditV011Run consumes that path, while auditStableRun invokes the new strict gateway check only for profile === 'stable-demo'. This conflicts with spec 06 §10's unqualified requirement to replay and validate proposal gateway evidence.

Required before approval:

1. Use one shared runtime validator that models receipt success/failure: error absent only with a final 2xx attempt, error present only with a final non-2xx/null attempt, no attempt after a 2xx or non-retryable row, and at least one successful receipt for a completed proposal (preferably a successful terminal row for every logical request id).
2. Apply it to both stable and V011 final audit/binding paths with the frozen route.
3. Add negative tests for failure-only, success+error, 2xx/non-retryable nonterminal sequences, and malformed V011 gateway rows; then rerun exact-head CI and obtain a fresh independent review.

Do not merge PR #246 or close Issue #51 on this head. This review was independent and read-only: I did not edit files, commit, push, merge, or close the issue.

[timwhitez]

INDEPENDENT_REVIEW: REQUEST_CHANGES

Exact head reviewed: e7d1bd8c6680f1706f405c0e71f6c311b89b5fc5 (base 6324afdbce7b1fb64ee7efef34d1ad1b1d6d3022). Local HEAD, the remote branch, and PR head were independently rechecked and all equal this SHA.

Hosted CI run 33188550906 completed SUCCESS with exact head_sha=e7d1bd8c6680f1706f405c0e71f6c311b89b5fc5; every job step, including docs, format, lint, typecheck, unit, real Loader/local E2E, provenance, upstream-clean and release-integrity, succeeded.

Independent read-only validation passed:

• pnpm lint, pnpm typecheck, and git diff --check 6324afdb...HEAD.
• Full unit suite: 119 files, 901 passed + 1 platform skip.
• Full no-key E2E: 17 files passed + 2 credential-gated skipped; 42 passed + 4 skipped, including real Harbor ACP, three extract-elf outcomes, packed/offline Loader, V011 admission, resource attacks and crash/replay.
• Focused resource/publication/gateway set: 13 files / 75 tests; focused terminal-semantics set: 3 files / 24 tests; focused real cgroup/PID/V011/proposal/crash E2E: 4 files / 15 tests.
• Worktree and all three upstream reference checkouts remained clean.

Verified repairs: the producer-owned shared validator rejects failure-only, success-with-error, post-success/non-retryable continuation and malformed receipt rows; stable final audit calls it with the frozen route; V011 execution load and materialization binding now reject [null]. The cgroup-v2, bounded tmpfs, private descendant PID namespace, semantic resource receipt, manifest-last publication, durability and stable audit repairs also pass direct inspection and testing.

One blocking V011 final-audit coverage gap remains. auditStableRun permits 3..9 proposal.completed events but invokes verifyStableProposalPublications only for profile === 'stable-demo' (packages/dsh-self-evolving-cli/src/audit.ts:288-317). In auditV011Run, all materialization.json files are enumerated, but lines 375-386 validate only materialization/CAS authority. The shared gateway/resource execution load and binding at packages/dsh-self-evolving-cli/src/v011-audit.ts:446-472 occur only inside the fixed generation 1..3 loop, after selecting the materialization whose proposalDigest belongs to an admitted build.

That misses a valid production state: the engine records proposal.completed before build (packages/dsh-self-evolving-cli/src/engine.ts:500-555), and a later build rejection continues to another attempt. Such a terminal run can therefore contain an extra completed proposal/materialization that did not become one of the three admitted generations. Its proposal-execution-v1 bundle is never loaded by final audit. Starting from such a run, deleting that rejected attempt's execution directory—or committing [null]/failure-only gateway receipts there—does not add a V011 audit reason, because the action is absent from the three selected built.proposalDigest matches. The journal still claims proposal.completed, yet its resource/gateway evidence is not replayed. This violates the newly stated requirement that the shared validator run in V011 final audit for every completed proposal and leaves the prior malformed-array blocker only partially closed.

Required before approval:

1. Cross-bind every V011 proposal.completed event to exactly one action/materialization and load/bind that action's committed execution through assertCompletedProposalGatewayReceipts with the frozen route, regardless of later build admission/rejection.
2. Reject missing/dangling/duplicate proposal events, materializations, or execution bundles rather than validating only the final three generations.
3. Add final-audit regressions with an extra build-rejected completed proposal whose execution is missing, [null], failure-only, or route-mismatched; rerun exact-head CI and independent review.

This head is not mergeable for Issue #51 closure. Do not merge PR #246 or close Issue #51 yet. Review was independent and read-only; I did not edit files, commit, push, merge, or close anything.

[timwhitez]

INDEPENDENT_REVIEW: REQUEST_CHANGES

Exact head reviewed: aeaea55246415f522cdfccde0bdc0dceea7b6f0c (base 6324afdbce7b1fb64ee7efef34d1ad1b1d6d3022). Local HEAD, origin/fix/51-resource-quotas, git ls-remote, the PR head, and CI headSha all match this exact SHA.

Hosted CI run 33191159992 completed SUCCESS on this exact SHA. Every step, including documentation, format, lint, typecheck, unit, real Loader/local E2E, provenance, upstream-clean, byte-equality, and release-integrity, succeeded.

Independent read-only validation:

• pnpm lint and pnpm typecheck passed.
• Full unit suite passed: 119 files, 902 passed + 1 platform skip.
• Focused resource/publication/gateway/V011 set passed: 13 files / 76 tests.
• Focused real E2E passed: 4 files / 15 tests covering memory/PID/CPU/storage containment, descendant PID namespace, V011 real Loader/packed overlay, proposal isolation, and process crash/replay.
• git diff --check 6324afdb...aeaea552 passed; the repository worktree remained clean.

The e7d1bd8 retained-generation blocker is substantially repaired: every discovered materialization.json is now bound to the exact proposal.completed payload and its manifest-committed worker/resource/frozen-route gateway/usage/diagnostic execution is replayed, including a later build-rejected materialization. Missing or malformed execution for a discovered materialization fails closed.

Two blockers remain.

1. Dangling/extra manifest-committed V011 executions are not enumerated. auditV011Run at packages/dsh-self-evolving-cli/src/v011-audit.ts:367-423 derives its universe only by recursively selecting paths ending in /materialization.json. It checks materialization → event and event → audited-materialization, but never inventories proposal-execution-v1/publish-manifest.json directories. Therefore a manifest-committed execution with no materialization/event is invisible, including malformed gateway/resource evidence.

   Exact-head read-only probe:

   {"materializationSelectorCount":0,"committedExecutionCount":1,"committedExecution":"/tmp/.../v011/actions/proposal-9-9/proposal-execution-v1/publish-manifest.json"}

   The committed bundle used gateway-receipts.json: [null]; because no materialization.json existed, it entered no audit replay path. Adding this tree to an otherwise accepted run leaves the audit's selected materializations/events unchanged. This is the dangling-execution case the prior review explicitly required to reject and means the three sets are not yet exactly equal.

2. The new bounded-tmpfs export installation is not crash durable and can strand a committed execution. materializeExport in packages/dsh-self-evolving/src/proposal/process-sandbox.ts:144-173 writes the staged files, renames the original tree to backup, renames staging into place, and removes the backup without fsyncing file contents or the parent directory. V011 then reads worker-output.json and publishes the fsynced execution manifest (v011-real-capabilities.ts:1038-1053). After a host/power crash, the execution manifest may therefore be durable while the installed child-tree rename/content is absent or reverted. Resume calls loadV011ProposalExecution(...workerOutputPath) first (v011-real-capabilities.ts:832-843); a missing/mismatched worker path throws, while quarantine/rebuild runs only when the loader returns null. The action is permanently stranded instead of converging from immutable parent plus durable request/execution evidence. This regresses the partial-disk-write durability contract.

Required before approval:

1. Inventory canonical V011 action directories and require exact one-to-one set equality among every proposal.completed event, materialization, and manifest-committed execution. Reject dangling/duplicate/unknown action or execution paths, and add final-audit tests for an extra valid and malformed committed execution without a materialization/event.
2. Make exported child installation durable before publishing the execution manifest (fsync written files/directories and the rename parent), or define recovery that safely quarantines/restores a committed execution whose installed tree is missing/mismatched without redispatching the provider. Add fault injection at every export write/rename/remove/fsync boundary.
3. Rerun exact-head hosted CI and obtain a fresh independent review.

Do not merge PR #246 or close Issue #51 on this head. This review was independent and read-only; I did not edit files, commit, push, merge, or close anything.

[timwhitez]

INDEPENDENT_REVIEW: REQUEST_CHANGES

Exact head reviewed: 4224a080c417265d9abe91cb55639094286cb2c1 (base 6324afdbce7b1fb64ee7efef34d1ad1b1d6d3022). Local HEAD, remote PR head, and hosted CI head all match this SHA.

Hosted CI run 33194958685 completed SUCCESS on this exact SHA. Every step, including docs, format, lint, typecheck, unit, real Loader/local E2E, provenance, upstream-clean, byte-equality, and release-integrity, succeeded.

Independent read-only validation passed:

• pnpm lint, pnpm typecheck, and git diff --check origin/main...HEAD.
• 7 focused unit files / 56 tests: V011 execution/materialization/resource binding, stable proposal audit, bundle publication, gateway durability, and gateway receipt semantics.
• 3 real E2E files / 13 tests: memory/PID/CPU/storage cgroup containment, descendant PID isolation, exact V011 Loader/packed-overlay admission, and proposal export durability.
• The worktree remained clean.

The new exporter does fsync every staged file and directory plus each authoritative parent rename/removal boundary before execution publication. Cached adoption and final audit also recompute the installed child tree and bind it to both finishedTreeDigest and the materialization sourceDigest. The ordinary pre-materialization missing/drifted-tree path quarantines execution/children and retains the durable gateway request store.

Two blocking authority/recovery gaps remain:

1. Quarantined history can re-enter the active materialization audit, while malicious filesystem entries can remain invisible. verifyV011ProposalExecutionInventory excludes paths whose second component is incomplete-executions (v011-audit.ts:161-168), but auditV011Run then rebuilds actionFiles from the unfiltered tree (v011-audit.ts:418-436). A complete retained copy under actions/proposal-1-1/incomplete-executions/retained/proposal-1-1/ is excluded from inventory yet its materialization.json is selected again; because the nested basename is still proposal-1-1, lines 444-459 can bind it to the live proposal:1:1:completed journal event. A no-filesystem exact-head probe of the actual exported inventory function returned:

{"inventory":[],"selectedMaterializationCount":1,"selectedActionRoot":"/tmp/review-actions/proposal-1-1/incomplete-executions/retained/proposal-1-1","selectedBasename":"proposal-1-1"}

The recursive walker also silently ignores symlink/special entries (v011-audit.ts:133-143). A symlink named proposal-execution-v1/publish-manifest.json reports isFile=false,isDirectory=false,isSymbolicLink=true and never reaches the inventory, so an extra malicious active-namespace entry is not itself a fail-closed reason.

2. Once materialization.json exists, missing/drifted worker bytes or tree bypass recovery and permanently strand resume. realV011Proposal reads and adopts the cache before calling quarantineIncompleteV011ProposalExecution (v011-real-capabilities.ts:778-830). loadBoundV011ProposalExecution then throws on missing/drifted installed bytes, but that early branch has no quarantine/replay path. The recovery helper also does not move/deauthorize the materialization. In addition, writeExclusive writes directly into the final cache path (v011-real-capabilities.ts:162-177,1151-1154), so a crash after exclusive create but before a complete fsync leaves a present torn cache; resume takes the same early branch and fails at JSON parsing forever. This is exactly the committed-execution/tree-drift class the PR claims to recover.

Required before approval:

1. Derive one canonical active-action inventory and use it for both set equality and semantic materialization replay. Exclude the entire quarantine subtree before either path, require direct canonical action roots, and reject symlink/special entries in the active namespace. Add full auditV011Run regressions for nested quarantined copies and symlinked/unknown manifest paths.
2. Run recovery before cached materialization adoption, quarantine/deauthorize the cache together with the execution/tree when rebuilding is allowed, and atomically publish the materialization through a fsynced staging/no-clobber commit. Add crash/replay tests for torn cache and post-cache missing/drifted worker output/tree while proving the durable gateway request is not dispatched again.
3. Rerun exact-head hosted CI and obtain a fresh independent review.

Do not merge PR #246 or close Issue #51 on this head. Review was independent and read-only: I did not edit files, commit, push, merge, or close anything.

[timwhitez]

INDEPENDENT_REVIEW: REQUEST_CHANGES

Exact PR head reviewed: 38eb413ef7b80f0f664d7d037db07611f9cb9fc5 (base 6324afdbce7b1fb64ee7efef34d1ad1b1d6d3022). Local HEAD, remote branch and PR head match exactly.

Hosted CI run 33198187681 has the exact head SHA but is still IN_PROGRESS at review time, so it cannot supply an approval gate. Independent read-only validation passed pnpm lint, pnpm typecheck, 3 focused unit files / 16 tests, 3 real resource/Loader/export E2E files / 13 tests, and git diff --check. The worktree remained clean.

Verified repairs:

• one canonical direct-action scan feeds both inventory and semantic replay;
• nested incomplete-executions/** materializations/executions stay de-authorized;
• leading-zero/non-canonical actions and active symlink, hardlink, FIFO, socket and special entries produce fail-closed reasons;
• valid execution bundle inventory, installed worker/tree checks, resource/gateway semantics and resource-limit E2E remain green.

Two cache-publication/adoption blockers remain:

1. publishV011MaterializationCache can report success after cleanup/path failure. Its cleanup probe uses lstatOrNull(staging).catch(() => null), converting every lstat error, not only ENOENT, into absence. A read-only exact-head fault probe moved the action directory at the directory-fsynced checkpoint and replaced the original path with a regular file. The function resolved successfully even though the requested authority path was inaccessible:

   {"outcome":"RESOLVED","finalInfo":"ENOTDIR","movedInfo":{"isFile":true}}

   Cleanup uncertainty must never become success. Propagate non-ENOENT probe/cleanup errors and prove the final authority file is still canonical and directory-durable before returning.

2. recoverV011ProposalCache adopts an externally hardlinked cache before audit. Both its initial cache check and quarantineIncompleteV011ProposalExecution accept any regular cache file without requiring nlink === 1; cleanup recognizes only the expected local staging-name residue. With a complete valid execution, installed tree and cache plus one hardlink outside the action, the exact-head probe returned:

   {"adopted":true,"cacheLinks":2,"aliasLinks":2}

   The later final audit would reject this link count, but production has already consumed mutable aliased authority. Recovery must distinguish the known post-link staging crash, remove that residue durably, then re-lstat and require the final cache to have exactly one link before the second validation/adoption. Any unknown hardlink must quarantine cache + execution + children while retaining durable gateway requests.

Add negative tests for non-ENOENT cleanup/probe failure and an out-of-action cache hardlink, rerun exact-head hosted CI, and obtain a fresh independent review. Do not merge PR #246 or close Issue #51 on this head.

Review was independent and read-only; I did not edit files, commit, push, merge, or close anything.

[timwhitez]

INDEPENDENT_REVIEW: REQUEST_CHANGES

Exact PR #246 head reviewed for Issue #51: f6a6754f0d996ad079f1d9e2ee210688d252ad18 (base 6324afdbce7b1fb64ee7efef34d1ad1b1d6d3022). Local HEAD, tracked remote branch, git ls-remote, PR head, and hosted CI headSha all match exactly.

Hosted CI run 33200808537 completed SUCCESS on this exact SHA. Every step succeeded: source closure, docs, format, lint, typecheck, unit, real Loader/local E2E, provenance, upstream-clean, byte-equality, and release integrity.

Independent read-only validation passed:

• format/docs/lint/typecheck and git diff --check origin/main...HEAD;
• full unit: 119 files, 910 passed + 1 platform skip;
• full no-key E2E: 17 files passed + 2 credential-gated skipped; 44 passed + 4 skipped, including real Harbor ACP, three extract-elf outcomes, offline/real Loader, resource attacks and crash/replay;
• focused cache/audit/gateway/resource unit: 12 files / 79 tests;
• focused real cgroup/Loader/export/process-crash E2E: 4 files / 17 tests;
• provenance/upstream/byte-equality/release-readiness passed (431 tracked UTF-8 files, 112 links); deepseek-harness/, harbor/, and tb/ remained clean.

The two 38eb413 blockers are materially repaired. A non-ENOENT staging inspection error now rejects. A cache hard-linked before or during load is never adopted: it returns null, moves cache/execution/children out of authority, preserves the durable gateway request, and retains a fresh single-link quarantine inode. Static inspection also confirms O_NOFOLLOW on the held read descriptor plus dev/ino/nlink/size/mtime/ctime checks before and after read and again after the trusted load/adoption step.

One publication blocker remains, so this head cannot close Issue #51:

publishV011MaterializationCache still never verifies that the requested final authority path and its containing directory are the same durable objects before returning. After the directory-fsynced checkpoint, rename the action directory aside and create a new empty directory at the original path. The staging probe now receives ENOENT, interprets that as normal absence, and the publisher resolves even though the requested final path does not exist:

{"outcome":"RESOLVED","requested":"ENOENT","movedBytes":"{\"ok\":true}\n"}

A second exact-head probe simply unlinked the final path at that checkpoint. Cleanup removed the remaining staging link and again returned success with no authority name:

{"outcome":"RESOLVED","requested":"ENOENT"}

This is the unclosed half of the prior requirement to “prove the final authority file is still canonical and directory-durable before returning.” It is production-relevant: realV011Proposal publishes the cache, then verifies the in-memory cacheValue, so it can return a completed proposal while materialization.json is absent at the requested authority path. Final audit may reject later, but production has already reported success.

Required before approval:

1. Hold/bind the publication directory and staging inode identity; after staging cleanup, require the final path to be the same regular single-link inode with exact stable metadata, require the path directory to match the held directory identity, and fsync that directory before returning.
2. Treat final-path ENOENT, replacement, link-count drift, or directory replacement as failed publication.
3. Add regressions for both action-directory rename + same-path directory recreation and final-path unlink after directory-fsynced, then rerun exact-head CI and obtain a fresh independent review.

Do not merge PR #246 or close Issue #51 on this head. Review was independent and read-only; I did not edit files, commit, push, merge, or close anything.

[timwhitez]

INDEPENDENT_REVIEW: APPROVE

Exact PR head reviewed: 8598ade5b0968594500e070b49ef0abf7f175f2d (base 6324afdbce7b1fb64ee7efef34d1ad1b1d6d3022). Local HEAD, tracked remote, live branch ref, refs/pull/246/head, PR head, and hosted CI head all match this exact SHA; the review worktree remained clean.

Hosted CI run 33203320750 completed SUCCESS on this exact SHA. Source closure, documentation, format, lint, typecheck, unit, real Loader/local E2E, provenance, upstream-clean, byte-equality, and release-integrity steps all succeeded.

Independent read-only validation passed:

• pnpm format:check, docs, lint, typecheck, and git diff --check origin/main...HEAD;
• full unit suite: 119 files, 912 passed + 1 platform skip; focused authority/audit/gateway suite: 6 files / 55 tests;
• 6 real E2E files / 23 tests: cgroup memory/PID/CPU/storage containment, candidate admission with real Loader in solve/propose modes, proposal sandbox/export durability, callback and controller SIGKILL replay, and direct real Loader lifecycle/unload;
• deepseek-harness/, harbor/, and tb/ remained clean at their pinned revisions.

The two f6a6754 blockers are closed. Independent exact-head fault probes after directory-fsynced confirmed:

1. renaming the action directory and recreating an empty directory at the requested path rejects with authority directory changed; requested materialization.json is absent and the moved original retains the exact bytes;
2. unlinking the final authority name rejects with ENOENT; the publisher never reports success without the requested authority.

Static inspection confirms the action-directory and staging descriptors stay open through cleanup; cleanup checks staging inode identity; success requires the requested directory to match the held dev/inode and the final path to match the staged regular single-link inode, stable metadata, and exact bytes. The held directory is fsynced and all authority checks are repeated before return. A 200-publication descriptor probe had FD delta 0.

Earlier blockers remain repaired: non-ENOENT staging inspection errors propagate; hardlinks introduced before or during load are never adopted; cache/execution/children are de-authorized together into fresh single-link retained evidence while durable gateway state remains; active inventory, resource-receipt semantics, cgroup quotas, manifest-last execution, installed-tree binding, and crash recovery all passed their regressions.

No correctness, security, evidence, or Issue #51 closure blocker remains on this exact head. PR #246 is approved for merge; Issue #51 may be closed after the exact approved PR is merged.

Review was independent and read-only. I did not edit files, commit, push, merge, or close the issue.