fix(runtime): land permission switches before the next turn's first tool call (#3349) - #3615
fix(runtime): land permission switches before the next turn's first tool call (#3349)#3615chinawch007 wants to merge 12 commits into
Conversation
…che#3349) runSessionQuiescentMutation bails with session_busy whenever any execution claim exists, so a permission switch can never land while a Goal keeps admitting successor turns back to back. Add runSessionQueuedQuiescentMutation: it reserves a slot on each session's mutation tail synchronously, then runs the operation only once the session is at rest — every claim that predated the reservation has settled, and no run is active. Claims only cover admission (a turn's claim settles once its run is bound), so live turns are observed through hasActiveRuns instead; a run registers on its backend generation before its claim settles, so an in-flight admission is never invisible to both checks. Claims created after the reservation carry it in their admission barrier, so neither they nor runs started through them can appear first; waiting chains run strictly backwards in claim-creation order, which keeps the queue deadlock-free. Wakeups fire from claim settlement and run unregistration. Worst-case delay is one turn. The eager variant keeps its semantics unchanged. Generated-by: ZCode (Z.ai GLM)
…che#3349) setPermissionMode, setExecutionBoundaryKind and transitionSessionConfiguration rejected with session_busy whenever a turn was running or an admission claim existed, so a switch could never land while a Goal keeps admitting successor turns back to back. Route commitExecutionResourceTransition through the kernel's queued quiescent mutation instead: the switch waits out the claims and runs that predate it, commits in the inter-turn gap, and the successor turn — admission-barrier-gated on the reserved slot — observes the new configuration before its first tool call. The eager hasActiveRuns guards come out (quiescence is now the kernel's single authority); the waiting_for_user rejections and relocateSessionWorkspace's fail-fast keep their semantics. Existing assertions expecting the reject behavior are updated to the queued semantics, and a regression test drives a gated turn through an Auto→Bypass switch, asserting the switch commits in the gap and the next turn is rebuilt from the committed mode. Generated-by: ZCode (Z.ai GLM)
…che#3349) ctx.permissionMode was frozen at backend build time from the session header, so even a committed switch left a running turn's approvals on the old mode — the half-applied state behind the issue's mixed symptoms. Derive the mode per tool dispatch from the authoritative boundary (executionBoundaryDisplayMode), with the plan-mode downgrade applied at the same point; an external boundary falls back to the last known header mode. resolveCollaborationPermissionMode moves to core so the composer (build time) and the tool runtime (dispatch time) share one rule. Also close the catalog short-circuit amplifier: session.configuration.update now requires the durable boundary to match the requested mode before treating the update as a committed no-op, so a header/boundary divergence is repaired instead of blessed. executionBoundaryMatchesPermissionMode moves to core next to the display-mode derivation. Legacy 'execute' is audit-safe: compilePermissionProfile and the filesystem worker treat it identically to 'ask', and the subagent snapshot's permissionMode is unused by tool building. Generated-by: ZCode (Z.ai GLM)
…pache#3349) Commits 1-3 rely on every config write disposing the backend before it commits. Keep that convention from staying implicit: each generation now records the boundary revision it was composed against, and ensureActive compares it against the store before reuse. On drift with no active runs the generation is disposed and rebuilt in the same activation; with runs still live it is only marked for invalidation — the existing flush path retires it when they exit and the next activation composes fresh. Tools stay correct during the grace turn: they read the boundary live on every call. An unreadable boundary leaves the guard dormant; a self-heal test drives a stray revision bump through both the idle and the live-run branches. Generated-by: ZCode (Z.ai GLM)
…sweep (apache#3349) Complete the verification plan: the plain-session case the reporter asked for drives an ordinary turn, an idle Auto→Bypass switch, and the next turn, asserting the successor is composed from the committed mode and that a probe tool call — resolving the boundary through the session's own store, the same read a real dispatch performs — sees executionBoundary.kind === 'bypass' and permissionMode === 'bypass' at once. A seeded sweep then alternates ask/bypass (both widening and narrowing, the latter through the shell-run fence) across interleaving classes — idle, mid-turn, racing the turn's release — asserting the invariant that every turn started after a switch resolved observes the committed configuration. Every checkpoint awaits a deterministic event; the seed is fixed, so failures reproduce. Together with the earlier suites this closes the matrix: mid-turn, gap/idle, successor-claim-pending, waiting_for_user, mid-dispatch boundary flips, and stray revision bumps. Generated-by: ZCode (Z.ai GLM)
…le awaiting quiescence (apache#3349) Review finding (P0): the queued quiescent mutation held the session's mutation tail for its entire wait, and a running turn can depend on an admission mutation enqueued on its own session's tail — graph operator provisioning runs on the supervisor's session exactly when the supervisor's yield tool is waiting on a reconciliation milestone that needs it. Tail held + run drained + run waiting on the tail = deadlock. Decouple the two: the mutation now closes a per-session admission gate (new claims capture it in their admission barrier) and waits for quiescence WITHOUT the tail, so admission mutations a running turn depends on still pass. Only after quiescence does the operation join the tail, still serialized with other mutations; claims created after the request stay gated until it completes. Session-manager side, the wait is now scoped to the primary session only — waiting on descendants could deadlock the same way through gated child claims — and descendant activity is rejected at commit time instead, restoring the pre-queue session_busy guard as a truthful failure rather than a hang. Also corrects the falsified invariant in the doc comment: a running turn CAN enqueue a mutation on its own session's tail; the kernel regression test drives exactly that interleaving. Generated-by: ZCode (Z.ai GLM)
…pache#3349) Review finding (P1a): executionBoundaryMatchesPermissionMode judged read-only-ness by profile NAME while the authoritative display mode judges it structurally (apache#1611). A read-only-named profile widened by an approved expansion therefore read as explore to the matcher while presenting as ask — the catalog short-circuit could bless exactly the profile-level divergence this series set out to repair — and a custom structurally-read-only profile forced a transition that silently reset it to the canonical explore profile. Derive the match from executionBoundaryDisplayMode so both answers come from one implementation: a widened read-only no longer matches explore (repaired through the transition instead), custom read-only profiles match and are preserved, legacy 'execute' never matches (forcing the transition is the safe direction), and an external boundary stays unverifiable. Generated-by: ZCode (Z.ai GLM)
…solated sessions (apache#3349) Review finding (P1b): the new boundary-consistency condition in the session.configuration.update short-circuit made an external boundary (always unverifiable) fail the check on every update, so even a no-op re-apply went through transitionSessionConfiguration and hit the store's refusal to move an externally isolated boundary — a regression for sessions whose configuration had matched. Skip the boundary comparison when the boundary is external: the header comparison alone decides the no-op there, as it did before the divergence repair. Generated-by: ZCode (Z.ai GLM)
…action (apache#3349) Review finding (P2): setPermissionMode and setExecutionBoundaryKind only inspected waiting_for_user at request time. A turn that pauses on an approval after the switch was queued never settles until the user answers, so the queued commit waited indefinitely — the D1 'reject while the user holds a pending decision' semantics degraded into an unbounded hang in that window. The quiescence wait now treats an active interaction as busy: it throws SessionQuiescentMutationBusyError, interaction registration wakes the waiters so the rejection is timely, and the session-manager wrapper maps it to the same session_busy outcome transitionSessionConfiguration already produces for the same condition. Request-time checks stay as fast-fail; the regression test drives a switch queued behind a gated turn that then opens a sandbox boundary approval. Generated-by: ZCode (Z.ai GLM)
Document the boundary-revision stamping race (the stamp may trail the revision actually composed against — safe direction, one extra rebuild), the one-read-per-activation cost choice in the revision guard, and the operation_unavailable cliff a kernel without the queued primitive would create. Grow the seeded interleaving sweep from 12 to 100 iterations, closer to the stress volume the plan promised. Generated-by: ZCode (Z.ai GLM)
…target (apache#3349) Main removed 'execute' from the permission-mode vocabulary, so the P1a regression assertion now passes the legacy value through a type cast: the runtime property it protects — a stale persisted mode never matches, so it always routes through a transition — is unchanged. Generated-by: ZCode (Z.ai GLM)
Generated-by: ZCode (Z.ai GLM)
Astro-Han
left a comment
There was a problem hiding this comment.
Thanks for the careful admission-gate work and the unusually thorough race coverage. I found one security-timing gap at this exact head:
[P1] Apply permission reductions before waiting for the current turn to finish
runSessionQueuedQuiescentMutation closes admission for future claims but waits until hasActiveRuns(sessionId) is false before it runs the transition (packages/runtime/src/runtime-kernel.ts:578-587,664-703). The narrower boundary, shell-run termination, and backend disposal therefore do not begin until the current turn has fully ended (packages/runtime/src/session-manager.ts:1727-1789).
For a mid-turn Bypass→Auto/Explore request, the durable boundary remains bypass throughout that wait. ToolRuntime now correctly rereads the boundary per dispatch (packages/runtime/src/tool-runtime.ts:1326-1352), but every later tool call in that same turn still reads the old unrestricted boundary, and background shell authority is not terminated yet. This window is not usefully bounded in wall-clock time: the same dispatch path explicitly supports long-running installs, builds, training, and subagent loops, and a turn can issue multiple more tools.
The tests currently encode this gap: session-manager.test.ts:5164-5185 asserts the switch remains unsettled and the old boundary remains until turn 1 ends; the seeded sweep at :5498-5527 only verifies turns begun after the switch promise resolves. The per-dispatch test manually flips a fake boundary between calls, but the production transition cannot commit that flip while a run is active.
Please split widening from tightening. Delayed widening is a UX tradeoff, but tightening should establish a dispatch fence and revoke promptly—either stop the live turn or atomically install the narrower boundary so its next dispatch uses it, with the chosen contract also fencing already-running shell/subagent resources. An integrated Bypass→Ask/Explore regression should have the current turn attempt another write-capable/Bash dispatch after the user request but before terminal completion, and assert that it sees the narrower authority or that the turn was stopped. The successor-turn assertions should remain as a separate invariant.
zhiiw
left a comment
There was a problem hiding this comment.
Independent second lane at exact head 8ba7c4c907ec01e56035b0ee2e2742f1ec5d7417 (MERGEABLE). I read the mechanism before reading the existing review; converging on the same conclusion independently.
The existing [P1] is mechanically correct — confirmed by my own trace, not by reading it.
The tightening path: setPermissionMode → runSessionQueuedQuiescentMutation([sessionId], …) (session-manager.ts), which closes the admission gate immediately but runs the mutation — the durable boundary write included — only after quiescence (runtime-kernel.ts waitForSessionQuiescence loops on hasActiveRuns). So while the current turn is live, store.readExecutionBoundary still returns the old, wider boundary. ToolRuntime does re-read that store per dispatch (the new executionBoundaryDisplayMode derivation), but what it reads has not changed yet — the narrowing takes effect at the first dispatch after turn end, not after the user's request. The queueing machinery is direction-agnostic, so tightening inherits the same deferral as widening: session-manager.test.ts (Auto → Bypass requested mid-turn … the switch commits in the gap as soon as turn 1 settles) encodes this gap as the expected behavior, with expect((await store.readExecutionBoundary(session.id)).kind).toBe('managed') while the turn is still running.
The expensive parts the quiescence protects (backend disposal, descendant shell fencing) justify waiting — but the boundary write itself is a single store mutation, and the per-dispatch reread this PR added is precisely the mechanism that would let an early write take effect at the next tool call. Splitting "commit the narrower boundary now, dispose the old backend at quiescence" would close the window without destabilizing the running turn. Agree with the existing P1's framing: delayed widening is a UX tradeoff, delayed tightening is an authority window.
Checks note: the test check at this head is failure, but the failure is the ASF license-header gate — runtime-kernel-queued-quiescent-mutation.test.ts and tool-runtime-permission-mode.test.ts are missing headers, the job exits before running suites. So no test evidence exists on this head; the header gate failing early means the red X overstates what's known. npm run write:asf-headers should fix it.
No additional findings beyond the existing P1.
中文说明
独立复读了机制,既有 P1 成立:权限收窄请求进入排队静默 mutation,边界写入推迟到当前 turn 结束后的静默期;期间 ToolRuntime 虽然每次派发都重读边界,但读到的还是旧的宽边界。测试把这个窗口编码成了预期行为。另外这个 head 的 test 红是 ASF license header 门禁(两个新测试文件缺头文件注释),测试套件根本没跑——不是测试失败。
Summary
Fixes #3349
A permission switch (Auto→Bypass) was not observed by the next turn. Under Goal continuation the switch was rejected with
session_busy— the quiescent mutation bailed eagerly whenever any execution claim existed, and claims are near-continuous while a Goal admits successor turns back to back. When a switch did commit mid-turn, tools still acted on apermissionModefrozen at backend build time, so the picker said Bypass while Bash stayed sandboxed and approvals kept prompting. The issue asks for one property in any session, not just Goals: a permission change is observed by the next turn that starts after it, before that turn's first tool call.What changed
Kernel — execution serialization:
runSessionQueuedQuiescentMutation: a config change closes a per-session admission gate and waits for quiescence — every claim that predates the request has settled and no run is active — before its operation joins the mutation tail. The gate (not the tail) is what new claims observe, so admission mutations a running turn depends on — graph operator provisioning — still pass; waiting never holds a resource the waited-on execution needs, which keeps the queue deadlock-free. Claims only cover admission (a turn's claim settles at run bind), so runs are watched throughhasActiveRuns; a run registers before its claim settles, leaving no instant where an in-flight admission is invisible.setPermissionMode,setExecutionBoundaryKindandtransitionSessionConfigurationroute through it. The eagerhasActiveRunsrejections are gone — quiescence is now the kernel's single authority. The wait is scoped to the primary session; descendant activity is rejected at commit time (session_busy) instead of waited on, a truthful failure rather than a potential hang.waiting_for_userstill rejects, now also when it appears mid-queue. Behavior change: a switch during an active turn waits (bounded by one turn) instead of rejecting.Read model — the boundary as the single authority (#1611):
ctx.permissionModeis derived live from the durable boundary at every tool dispatch (executionBoundaryDisplayModeplus the shared plan-mode downgrade), so a committed switch reaches the very next tool call without waiting for a backend rebuild; an external boundary falls back to the last known header mode.resolveCollaborationPermissionModeandexecutionBoundaryMatchesPermissionModemove to core, so the composer (build time) and the tool runtime (dispatch time) share one rule. The matcher now derives structurally: a read-only profile widened by an approved expansion no longer reads as explore, and a custom read-only profile is preserved instead of being reset.ensureActiverebuilds on drift — immediately when idle, or via the invalidation flush once live runs exit. Any write path that skips backend disposal self-heals within one activation.How this meets the issue's stated goals
executionBoundary.kind === 'bypass'andctx.permissionMode === 'bypass'together.session_busy.permissionMode(live derivation),ensureActivereusing a stale generation (revision guard), and the catalog short-circuit that could bless a divergence (consistency check).Verification
Regression tests were written first and verified failing against the pre-change behavior: a gated turn with a mid-turn Auto→Bypass switch (queues, commits in the gap, next turn rebuilt from the committed mode), and the plain no-Goal session from the issue — the successor turn's first tool call sees
executionBoundary.kind === 'bypass'andpermissionMode === 'bypass'together, the reporter's primary case.A seeded interleaving sweep (100 iterations, fixed seed) alternates widening and narrowing switches across idle, mid-turn, and racing-the-release interleavings, asserting every turn started after a switch resolved observes the committed configuration.
External review findings each carry their own regression test: the admission-gate deadlock interleaving, the widened read-only and external-boundary matcher cases, and the interaction-pause rejection.
Not run: the full runtime-host suite and the remaining workspace suites, and manual Desktop verification of the picker.
npm run format:check— clean (8 formatting nits auto-fixed and folded in)npm run lint— 2304 files, no issuesnpm run typechecknpm --workspace @maka/runtime run build— cleannpm --workspace @maka/runtime run test:dist— 3110 tests, 3097 pass, 13 skipRoot cause
Three cooperating defects: the eager bail in
runSessionQuiescentMutation, the build-time freeze ofheader.permissionModeinto the tool context, and a catalog short-circuit that compared only header fields. The fix converges on the boundary as the single read model and moves quiescence authority into the kernel.AI use
Select exactly one:
Tool(s) and scope: ZCode (Z.ai GLM) authored the implementation, tests, and review fixes; the contributor directed the design, reviewed each finding, and made the rebase decisions.
Checklist
Does this PR entail a change in behavior?