feat(tools): default shell timeout that detaches, and reads outside the working directory through the approval ladder (F47, F48) - #452
Open
plombeer31 wants to merge 3 commits into
Conversation
…readScope
USER_CONFIG_VERSION 66 -> 67, one step carrying four fields; 66 joins the
supported input versions and the upgrade is additive for all of them (an
older file has none of the fields and takes the defaults).
`tools.shell.defaultTimeoutMs` (default 600 000) is the wall-clock wait
for an `os.shell.run` call whose `timeoutMs` the model omitted. A command
still running when it elapses is detached as a job the model can `wait`
for or `kill`, with its output so far in the result; `0` keeps the old
unbounded behaviour. An explicit per-call `timeoutMs` (including `0`)
always wins, and kills at its limit. An existing file has no `tools`
block and takes the default; the value is validated as a non-negative
finite integer.
`tools.shell.jobMaxMs` (default 3 600 000) is the absolute ceiling for a
shell job the default timeout detached, counted from its start;
`tools.shell.maxJobs` (default 3) is how many such jobs a session may
have running at once. Both positive integers: a `0` ceiling would mean
"kill at once" and a `0` limit "never detach", so both are refused.
`agent.readScope` (`"working-dir"` | `"unrestricted"`, default
`"working-dir"`): v66 files are accepted on input and take the default;
the parser rejects any other value.
The runtime config carries them as `config.tools.shell.{defaultTimeoutMs,
jobMaxMs, maxJobs}` for the shell tool to read and as `agent.readScope`
for the read scope. The mechanisms that read them land in the F47 and
F48 commits.
…g it — wait, kill, jobs, keep
`os.shell.run` without `timeoutMs` used to run unbounded; a recursive
grep over a home directory ran for twenty minutes until a person killed
it, and `agent.toolTimeoutMs` never applied to the shell. At
`tools.shell.defaultTimeoutMs` (10 min) the call now returns `ok` with
`details.detached: true`, `details.jobId`, `details.pid`, the output so
far, and a first line `still running after 10 min (job 3) — output so
far below; os.shell.run {"wait": 3} keeps waiting (up to another 10 min
per call), {"kill": 3} stops it, pass timeoutMs for a longer first
wait`. A build the operator's default interrupted is not a build the
model wanted stopped, so the process keeps running in its own process
group; its stdout/stderr keep being captured, head + tail, 1 MiB per
stream.
New call forms on the same tool: `{wait: id, timeoutMs?, keep?}` returns
the exit shaped like a fresh result (`details.jobId` kept) or the same
still-running result when the wait — the explicit `timeoutMs`, else the
default — elapses; a cancelled turn returns from a wait at once and
leaves the job running. `{kill: id}` is the group kill, `details.killed:
true`, the tail of the output. `{jobs: true}` lists the session's jobs
with id, command head, start and state. A call that mixes forms is
refused with the forms it named; unknown keys stay refused (F40). An
explicit `timeoutMs` still kills at its limit: the model asked for a
bound. The still-running and killed results show only the last lines
of output, because the result compressor keeps a twelve-line tail and
the notice above the command line has to be inside it.
The explicit-`timeoutMs` kill path — the limit resolution in
shell-timeout.ts (`source: "explicit" | "default"`), `details.timedOut`
and the `stopped after <n> s (timeoutMs) — output so far below` first
line — is the first version of this fix, which killed at the default
too; only the default's outcome changed.
Ownership (`src/tools/os/shell-jobs.ts`, one registry owned by the
bootstrap): a job dies when its turn ends (`executeTurn`'s `finally`,
the choke point every turn passes) unless the call that started it or a
later `wait` carried `keep: true`; every job dies when its session is
deleted, when `finish` completes it, at shutdown, and at
`tools.shell.jobMaxMs` from its start; at most `tools.shell.maxJobs`
run per session — the next detach stops the oldest un-kept job (the
oldest kept one when all are kept) and the result says so. A fusion
worker's jobs die with its turn. A tool built without a registry
(embedders, tests) gets a private one whose jobs die only at the
ceiling.
The primitive is `startCommandJob` (`src/sandbox/command-job.ts`): a
spawn whose lifetime is not tied to one await — `waitFor`, `stop`
(SIGTERM, SIGKILL after 2 s), `kill` — with `CappedOutput` behind it.
`runCommand` keeps its direct tree-kill; the process group (`detached`
on POSIX, `-pid` on timeout so a subshell's `sleep 30 &` cannot hold the
stdio pipes and the tool result) lives in the job, and so do its tests.
shell.ts is split to stay under the file limit: the interpretation
helpers into shell-interpretation.ts, the result rendering into
shell-result.ts, the job forms into shell-job-calls.ts. The descriptor
and the strict JSON schema carry the four forms; nothing is required at
the schema level because each form requires a different key.
Not covered: a runtime killed with SIGKILL cannot stop its jobs — they
run on until they exit or write to the closed pipe; the ceiling timer
died with the process.
… ask through the approval ladder
Until now only fusion workers had their reads confined; every other
session could read anywhere on disk, and a bench run showed a plain
cloud session reading a sibling run's solution, the harness's screen
dumps and the benchmark's own checker from far outside its working
directory. From config v67 (agent.readScope, default "working-dir")
every session's filesystem reads and shell path arguments are scoped to
the working directory plus the absolute / ~-prefixed paths the user
named in their own messages — never in model output; parsed from user
turns, recomputed per step by the step executor and handed down as
ToolContext.readRoots (a named file widens to that file, a named
directory to that directory; quoted paths keep their spaces, sentence
punctuation is stripped). Lexical or canonical containment both count.
src/tools/read-scope/ replaces src/tools/fusion/worker-read-scope.ts;
the worker check is unchanged (its tests moved, byte-for-byte in
meaning) and the worker names stay exported from src/tools/fusion as
aliases.
Covered: os.fs.read/list/grep/glob/diff/hash/watch/read_document/
archive.*, vision.describe and verify.syntax, wrapped at the registry;
and os.shell.run through a narrow token check — a token that is an
absolute path under home or the directory homes live in, and lies
outside every root, or a `..` climb that escapes every root. System
prefixes (/usr /bin /sbin /opt /dev /etc /System /Library
/Applications, C:\Windows, C:\Program Files) are never questioned, and
neither is the OS temp directory: a path under os.tmpdir() (plus /tmp
and /var/tmp on POSIX) is scratch space, always in scope, fs and shell
alike — models routinely write a helper to /tmp and run it, or redirect
into /tmp/x.txt and read it back, and the threat was other users' homes
and a benchmark's sibling trees. Fusion workers keep their narrower
rule (working directory plus the fan-out's write scope, never the
brief) as a hard refusal for reads and shell alike. The opt-out is one
config line, agent.readScope: "unrestricted", read per call, so it
needs no restart.
A read outside that scope is not a one-line refusal at every approval
level (which would make the operator's answer a config edit). It is a
question, on the same ladder as every other gated action:
- New ApprovalCategory "fs_read_outside" (label "read outside the
working directory"), pinned at level 5 like trust_config and other,
grantable. At the asking levels (1-4) a read-class call or a shell
command naming a path outside the working directory, the user-named
paths and the directories approved so far raises the standard prompt
through requireApproval / the gate / the router, on the TUI modal,
the CLI stdin prompt and the Telegram bridge alike. The prompt names
the path, the working directory and what a yes means.
- A directory grant widens the session's read roots: `y` remembers the
directory the call named (or the parent of the file it named) in
ReadScopeGrants on the gate, keyed by session id and dropped by
clearSessionGrants with the category grants, so the next read under
it runs unasked. `[s]` grants the category: read anywhere this
session. A `n` answers the model with the one-line refusal naming
the path, the working directory and the opt-out.
- Auto-allowed at level 5 / --no-approval: nothing asks, nothing is
refused, like the other categories pinned there.
- `agent.readScope: unrestricted` never asks: the scope check is off
for every session, read per call, no restart.
- Fusion workers keep the hard refusal for reads and shell alike: an
ephemeral session has nobody to ask; WORKER_READ_REFUSAL_REASON stays.
Questions are asked one at a time per session (a per-session queue in
ReadOutsideApprover): read tools run in parallel inside a batch, and a
second prompt raised while the first waits would strand one of them on
a surface that shows a single pending request. The second read waits
for the answer and re-checks against the widened roots, so three reads
under one new directory ask once.
Wiring: confineReads takes the ladder's DangerousToolOptions and throws
when given a readScope without it, so the scope cannot be half-wired
into a silent refusal. The public surface of read-scope is confineReads,
findSessionReadOutside (the path, not the refusal), userNamedPaths and
findShellPathOutsideScope; the shell check shares shellCommandLine for
the prompt's preview. The batch planner's per-tool category table does
not list fs_read_outside under os.shell.run / verify.run on purpose: it
would pin every shell batch solo below level 5 for a check that fires
only when a command names an outside path, and a denial there changes
no state.
Known cost: a path the user described but did not name ("my Downloads
folder") is asked about rather than read. That is the intended trade.
plombeer31
force-pushed
the
harness/15-session-boundaries
branch
from
September 16, 2026 22:03
40d52cd to
1ff5148
Compare
plombeer31
force-pushed
the
harness/14-fusion-loop-tuning
branch
from
September 16, 2026 22:03
5e9c3ad to
309ec92
Compare
plombeer31
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two session-boundary defaults from the second verification matrix (bench 2, findings 30–31), in the shape the operator asked for after reviewing the first version of this PR:
os.shell.runhad no timeout unless the model passedtimeoutMs; a cloud session ran a recursivegrepover the home directory for 21 minutes, then another for 20. Now, whentools.shell.defaultTimeoutMs(default 10 minutes,0= none) elapses and the model did not pass its owntimeoutMs, the command is not killed: the tool returns to the model with the output so far and a job id —still running after 10 min (job 3) — output so far below; os.shell.run {"wait": 3} keeps waiting, {"kill": 3} stops it, pass timeoutMs for a longer first wait.{wait},{kill}and{jobs}are new call forms; a job dies when its turn ends unless the call carriedkeep: true, always when the session ends, attools.shell.jobMaxMs(default 60 min), and the oldest un-kept job is evicted pasttools.shell.maxJobs(default 3). An explicittimeoutMsstill kills at its limit — the model asked for a bound. Nothing legitimate is killed at 10 minutes; a runaway command is bounded by the model's own decision, the turn's end, or the ceiling.src/tools/read-scope/and applies to every session: read tools and shell paths outside the working directory and the paths the user named in their own messages raise the new approval categoryfs_read_outside. At the asking levels the standard prompt names the path and the directory a "yes" widens to for the rest of the session; at level 5 /--no-approvalit just runs;agent.readScope: unrestrictednever asks; fusion workers keep the hard refusal (ephemeral sessions have no one to ask). System prefixes and the OS temp directory are never asked about.Config version 66 → 67:
tools.shell.defaultTimeoutMs,tools.shell.jobMaxMs,tools.shell.maxJobs,agent.readScope. Existing files upgrade additively.Why
An agent that can block on an unbounded command or read anything the account can read is one bad step away from a 20-minute stall or from reading what it was never asked to read. Both defaults now put the decision where it belongs — the model for a long command, the user for a read outside the project — using mechanisms that already exist (the tool-result contract, the approval ladder), with the operator's one-line opt-outs kept (
defaultTimeoutMs: 0,readScope: unrestricted).How it was verified
npm run lintcleannpx vitest run src/approval src/tools src/runtime src/config src/agent src/sandbox src/session— 230 files / 2,835 tests green, 3 skippedsend-message-concurrencybaseline (the whole stack rebased ontomainat v0.6.2, 2026-09-17)waitreturns the exit;killends the process group with no surviving child; un-kept jobs die at turn end,keep: truesurvives it;maxJobseviction;jobMaxMsceiling; explicittimeoutMsstill kills; without the change the detach tests fail.unrestrictednever asks; workers still refuse (fs and shell); a shell path asks under the same category with the command; parallel reads under one directory ask once.Stacked on #439 (
harness/14-fusion-loop-tuning); merge in order.