Commit e860c65
UN-4008 [FEAT] Add the unstract CLI to run extractions and API deployments from the terminal (#2)
* feat: CLI scaffold with config, output envelope and poll engine
Wheel skeleton for the `unstract` console script: Click app with the
whisper / docstudio / config groups, and the three cross-cutting layers
every command will sit on.
- config: named profiles resolved flag > env > profile > default, with
`env:` indirection so the file records where a secret lives rather than
the secret, 0600 writes, deployment aliases, and `config doctor`
reporting where each setting resolved from without echoing a value.
- output: one JSON envelope {ok, data, error, meta} on stdout for success
and failure alike, so parsing never depends on TTY detection; table and
raw are opt-in renderings, diagnostics go to stderr.
- errors: the exit-code table as a stable API, retry policy that never
retries a 4xx, redaction, and undeclared statuses reported verbatim
rather than guessed.
- poll: transport-agnostic --wait loop reading terminal state from the
response body rather than the HTTP status, never sleeping past the
deadline, echoing the job handle on timeout so work resumes instead of
being resubmitted, and persisting a one-shot result before the read is
acknowledged.
No transport yet: the clients own HTTP. Tests are offline and need no
credentials.
* feat: derive command flags from the committed API specs
Flags for an operation come from the spec the published client is generated
from, intersected with what that client's signature actually accepts: a spec
parameter the frozen client cannot name would raise TypeError at the call
rather than reach the API, so it is not offered.
Two rules keep the derivation honest. Every option defaults to None, meaning
absent, so an unpassed flag is not sent and the client or server default
applies rather than a value pinned here. And only None is treated as absent:
0, false and "" are choices a caller made and travel to the request.
Help text has three sources in order: the overlay, the spec, and the client
method's own docstring, which is the only one that describes the parameters
today. The overlay carries what a generated spec cannot express -- allowed
values, short flags, wording -- in TOML read with the stdlib.
* feat: the v1 command surface for both products
Thirteen commands: whisper extract/status/retrieve/detail/highlights/usage
and its four webhook commands, plus deployment run and status. Each one holds
only what a spec cannot say -- which parameter is the argument, which the CLI
owns, and how a result is polled for.
The CLI runs the poll loop for both products rather than using the loop one
client ships, so --wait, --interval, --timeout and the handle-returned-on-
timeout behaviour are the same everywhere. Deployment runs are queued
(timeout=0) so a request does not hold a connection open for the length of the
job. Line-highlight scaling is arithmetic on a reply rather than a request, so
it is folded into the command that fetches the metadata.
Failures converge on one envelope: LLMWhisperer raises with a status code, the
deployment client returns one, and both become a CLIError with an exit code and
a hint. A result that can be read only once is written to disk before it is
printed.
* feat: --discover and a live probe for config doctor
--discover answers what --help answers, as JSON, in three tiers: groups names
the products, summary adds their commands, full adds every flag with its type,
choices and default plus the exit-code table -- enough to construct a call
without a second round trip. A caller starts cheap and drills down.
Every tier is read back from Click itself, so a described command cannot drift
from the one the parser accepts, and discovery reads no configuration: it is
how a caller learns what exists, so it has to work before anything is set up.
config doctor --probe adds the second diagnostic question -- does the resolved
key work -- to the one it already answered offline, where it resolves from.
LLMWhisperer is checked against its usage endpoint. A deployment has no
side-effect-free endpoint to call, so its entry reports that the settings
resolve and says plainly that nothing was verified.
* test: pin the parameters no command can reach
The vendored specs and the pinned clients move independently, so a refreshed
spec can declare a parameter the published client has no argument for. Such a
parameter is dropped rather than offered and rejected at the call, and dropping
it silently is the failure this pins: the gap is written down per operation, so
widening it is a decision rather than an accident.
* fix(whisper): read the highlight metadata the service actually returns
Two failures a live call found and no offline test could.
The metadata arrives as a named object carrying the coordinate list under
`raw`, while the client's geometry takes the bare list, so no line was ever
scaled. And a line the service has no geometry for is reported as all zeros,
whose page height is a divisor in that scaling: it raised ZeroDivisionError out
of the client, which the entry point does not catch, so the command printed a
traceback with an empty stdout. Such a line now gets no box.
* feat: connection flags, wider clients, and honest one-shot wording
Three follow-ups to the command surface.
Both client pins move forward, and the six extraction parameters and three
status parameters they gained appear as flags with no line written here --
which is what deriving from the specs was for. The contract test's unreachable
set shrinks to what the clients own rather than lack: the URL-in-body flag and
the execution id read from the endpoint URL.
--base-url, --api-key and (for deployments) --org-id sit on the product group
and fill the flag tier of flag > env > profile > default, which the loader
already supported but nothing populated. A key given on the command line warns:
it lands in shell history and in the process list.
The 406 hint is scoped to deployments. A whisper result read twice comes back
as a 400 whose body says so, and translating on that prose would break the
moment the wording changes -- the service's own message already says what
happened, and it is passed through verbatim.
* Forward the status parameters, and stop the doctor overstating itself
`deployment status` derived --include-metadata, --include-metrics and
--include-extracted-text from the spec, collected them into **params, and never
passed them to the client. The command succeeded and the payload parsed, so a
dropped flag was indistinguishable from a working one. The poll loop behind
`deployment run --wait` had the same hole, which made a waited run return less
than the identical flags returned without --wait.
Both now forward what was asked for, and the parameters the status endpoint does
not accept are filtered out rather than sent. Tests cover each flag in both
polarities, since a flag silently dropped is exactly what the offline suite
missed.
Alongside:
- `config doctor` no longer reports an `org_id` setting for LLMWhisperer, which
has none. It always read as unresolved and there was no way to resolve it.
- The deployment probe reports `ok: null`, not `ok: true`. Nothing is called, so
there is no verdict; `true` beside `checked: false` reads as a live check that
passed. `resolved` carries what is actually known.
- The 406 hint pointed at --save, which does not exist on the command that emits
the hint. It now names the command that has it.
- A 400 carries a hint. The service can answer 400 with an empty error body, in
which case the message was a synthesised fallback and there was nothing else
to go on.
Adds RUNBOOK.md: install, moving the client pins, the live-gate checklist, and
the release steps.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* Report which job a waited result belongs to
Waiting returns the result and nothing else: the extracted text, or the
deployment's structured output. Neither names the job, so a caller who waited had
no handle to correlate against the service, quote in a bug report, or use for a
follow-up call. Without --wait the handle is the entire payload, so the identity
appeared and disappeared depending on a flag.
Both waited paths now carry it in `meta` -- the whisper hash and the execution
id -- leaving `data` exactly as it was.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* Stop losing one-shot results, and stop printing keys
`--save` exists to protect a read the service serves exactly once, and it
was the flag that lost the data: the write ran after the acknowledging
read, raised `OSError` through an entry point that does not catch it, and
left an empty stdout with the extraction gone. The target is now proven
writable before anything destructive runs, the write goes through a
temporary file so a full disk cannot truncate the previous copy, and a
write that fails anyway raises with the payload attached under its own
exit code -- by that point the envelope carries the only copy left.
Also on the one-shot path: a waited extract read the result with a bare
`.get("extraction")` where the sibling command falls back to the whole
payload, so a response shaped any other way printed `ok: true, data: null`
for a document that had been processed and billed. Both now read it the
same way, and a genuinely empty result is a failure rather than a silent
success.
Redaction was an opt-in keyword argument that only the success path
passed, so every error envelope and every stderr summary went out with
the key in it -- four times on stdout in the reproduced case. Credentials
are now registered where they resolve and scrubbed by every emitter, and
`CLIError.details` is redacted structurally rather than at each call site.
Three more places where a failure was reported as a success: the
standalone status commands ignored a finished-and-failed execution inside
an HTTP 200, the poll loop treated an unreadable body as progress and then
blamed the timeout on a job it never confirmed was running, and any status
outside 4xx/5xx mapped to exit 0 while printing `ok: false`.
Verified by mutation -- moving the save after the print, dropping the
registry, dropping the details redaction and dropping the status check
each fail the suite now, and none of them did before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* Print a table by default, and version the JSON
A CLI whose output shape depends on whether a terminal is attached is a CLI
whose scripts break when they move from a shell to CI. This drops the
isatty question entirely: the default is a table, in a terminal and in a
pipe alike, and anything that parses the output asks for `-o json`.
An explicit `-o` is the last word. The environment picks the default and
nothing more, so the same `-o json` invocation renders the same bytes
wherever it runs -- which is the property a caller is actually relying on.
Coding agents are the exception worth making: they set a marker in the
environment, and there the default becomes json rather than making every
call carry a flag. `--agent yes|no` settles it either way.
Every envelope now carries `meta.contract_version`, and `--discover full`
publishes what a consumer has to do to hold up its end: ignore unknown
fields, refuse a version above the one it was written against.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* feat: expose the deployment client's socket timeout
Nothing bounded a stalled connection: the deployment client is untimed and
its api_timeout is an execution mode the backend reads, not a socket
timeout. --transport-timeout sets one. Unset by default, so a run that
would have hung still hangs rather than starting to fail in a way no
existing script expects.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: report an interrupt as an interrupt
Ctrl-C came back as exit 1 with nothing on stdout, which reads to a
supervisor as a failed command worth retrying -- the one thing that must not
happen to a run the user deliberately stopped. It now exits 130, the value
every shell already reads that way, and prints the same envelope as any
other failure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* test: pin which of the three help sources wins
Overlay, spec and client docstring can each describe a flag. No spec
parameter carries a description today, so the order between them is
unexercised until one does, which is exactly when an inversion would ship
unnoticed.
* fix: take allowed values from the spec, not from a copy of them
The vendored LLMWhisperer spec was several revisions behind and now declares
enums the CLI was hand-listing. The two had already diverged: --mode rejected
three modes the service accepts and --output-mode two, and nothing would have
reported it. Read the enum off the spec, keep the overlay for narrowing one on
purpose, and drop the descriptions' own value lists for the same reason their
default sentences are dropped.
`highlights` gains a `mode` query parameter that the published client has no
argument for, so it joins the parameters the CLI cannot reach.
* fix: strip a restated default that contains a period
A sentence-shaped match ends at the first period, so "Defaults to 0.3." was
left in the help beside the default rendered from the signature. Strip each
restated sentence with its own end-anchored pass instead.
* build: move the client pins to the heads the specs were taken from
The pinned clients predated the fix that stops an omitted optional parameter
being sent as the string "None", so a CLI built on them sent it. The derived
surface is byte-identical across the move; neither signature changed.
* docs: trim comments that narrate rather than explain
Each of these restated the line below it, or described a prior state that is no
longer there to check against. Keep the reason, drop the narration.
* feat: add the `clone` command
Copies one organization's resources into another by calling the client's
orchestrator directly. Two endpoints with a key each, which no single profile
describes, so both are flags and both keys come from the environment.
Also moves the client pin forward to pick up the status path-prefix fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: resync the docstudio spec and pin the flags it derives
The vendored copy was several iterations behind the one the pinned client is
generated from, so the CLI's help, its parameter set and what --discover
publishes all described an older service contract.
The flag snapshot is the check that makes a resync safe: every other contract
assertion reads the spec on both sides of its comparison, so a spec that loses
a parameter loses the flag and the expectation with it.
* fix: keep the job handle on any mid-poll failure, and fail a failed status
A transport error was translated into a CLIError outside the poll loop, where
the handle no longer exists, so the caller was left to resubmit a document the
service had already processed and billed. Translating at the call keeps the
loop's own context; the loop attaches the handle itself for anything the caller
did not translate.
`whisper status` reported a failed extraction as a success, its sibling in the
other product having already been fixed: both read the body, not the status
code.
* fix: report a failure as one, and never authenticate against a guess
Four failures the CLI reported as successes or as something vaguer than it knew:
- a server-reported error inside a 2xx got the catch-all exit code, which is
the least informative one for the most interesting failure this API has;
- `config doctor` printed its own findings and exited 0, so a setup script
branching on it read a broken configuration as a working one;
- a deployment alias pointing at an unset environment variable fell back to the
profile's organisation and key, running against a tenant nobody named;
- a webhook's auth token was echoed verbatim.
The restated-default stripper was also greedy to the end of the string, so a
description whose value list came first lost every sentence after it.
* build: move the client pins to the heads carrying the transport fixes
* fix: hold the clone's guards, and say what a clone left behind
The command that writes into a live organisation had none of its own
behaviour pinned. Its table output -- the one a person gets, and the only
output path that did not go through the emitter -- scrubbed by hand and was
run by no test, while the test that claimed a platform key never reaches
stdout passed with the registration deleted. Rendered output now goes out
through the same path as every envelope, and a key planted in a report is
asserted not to survive it.
Also: --on-name-conflict decides what is written into the target and is now
asserted to arrive; skipped documents are counted at the top of the payload,
because skipping is not fatal and a caller reading the exit code alone would
never learn a document did not move; `config doctor` resolves each deployment
alias the way a run does, instead of listing names its docstring implies it
checked; a failed retrieve is pinned to carry the handle; the restated-default
stripper ends at its own sentence rather than at the end of the text; and the
groups tier lists leaf commands apart from groups, which a consumer walks
differently.
* build: move the deployment client pin to the poll-URL fix
The status endpoint's own query parameters are forwarded now, and a
deployment URL that carries no derivable prefix is polled where the service
said rather than at a rebuilt path.
* docs: draft the release notes, and move the client pin to its tip
The notes carry the console-script collision, the behaviours a script would
otherwise discover by being surprised, and the service version a custom page
separator needs. The pin moves to a documentation-only commit.
* docs: shorten the top-level help to what a first run needs
The envelope shape is documented in the README and published by --discover;
greeting every --help with it buries the two things a reader is there for.
* fix: do not let a discovered project config name the host or the key
A .unstract.toml found by upward search comes from whatever checkout the
user happens to be standing in. It may still select a profile, set org_id
and define deployment aliases; api_key and base_url are withheld, with a
warning, and reported as withheld by config doctor. Named explicitly with
--config or $UNSTRACT_CONFIG, the same file is honoured in full.
Also point a first-time user at where keys are minted, from config init,
from doctor and from the README, and ship an on-prem profile shape.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: never write config through a symlink a checkout chose
The discovered config path is written to as well as read from, so a
symlinked .unstract.toml let a repository redirect config set and
config init --force onto any file it named. The upward search now skips
a symlinked candidate, and the write opens with O_NOFOLLOW so a symlink
at the target is a clear error rather than a truncation.
Also: the config group reports the file's warnings instead of dropping
them, doctor answers for a withheld deployment-alias key the way it does
for a product one, trust is derived from the path rather than from how
the loader was called, and the README says plainly that routing stays
repo-controllable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* docs: one key can cover every deployment, and say where it is minted
An organization-wide API deployment key authenticates every deployment in
the org, so the starter config and the README now show one key on the
product block with aliases carrying only api_name; a per-alias key is for
an org whose deployments hold separate keys.
The missing-credential text names the third place a key comes from, and
the 401 hint no longer implies the key is simply wrong: a key that works
elsewhere can be rejected here for covering a different deployment or
another organization, and the responses are indistinguishable.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* test: pin the trust classification and the config group's own warnings
Reverting either left the suite green: the discovered file classified as
project-local however its path is spelled, and the config group reporting
what the loader withheld.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* test: snapshot what each derived flag accepts, not just its name
A spec resync that narrows an enum, changes a type or moves a default
left the gate green while the CLI began rejecting a value it used to
take. The snapshot now carries the whole parameter surface, and the
failure names the flags that moved.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* minor: Edit in discover docstring to clarify intent
* fix: treat an empty config value as unset
`config init` writes a placeholder for every setting only the user can supply.
An empty string satisfied `require`, so a request went out with a hole in it
instead of failing with a message naming the setting.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
* docs: drop the release notes and the runbook
Both restated the README for an audience that has neither shipped nor operated
this yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
* docs: install with uv, and answer what the README left open
Install and dev commands go through uv, matching how the project is built and
tested. The exit-code table says it is this CLI's own convention and names the
enum it copies, and a test now fails when the two disagree. The credential
section says a literal key works and why `env:` is the default. `clone` reads
as the operator command it is, so an agent does not reach for it unasked, and
the connection flags are named as the top tier of the resolution chain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
* docs: cut each comment back to the reason it exists
Every comment that ran to three or more lines narrated the decision rather than
naming it. Each is now one or two lines that hold up without the discussion
they came from.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
* docs: say in the top-level help what the CLI can do
The help named the products but not what they are for, so a first reader (or an
agent) had to run something to find out what was possible. It now says what each
product does and states the json envelope, the exit codes and `--discover` in
one paragraph. The command list under it is printed by Click, so the prose does
not repeat it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BFSunNN6RKRA1xo6kWkztx
* fix: close two local write windows on the config and result paths
Both writes could be steered or observed by another local user with write
access to the containing directory, or read access to an existing config
file.
`persist` wrote through a predictable `<target>.tmp` sibling opened with
plain "w". Anyone able to write the save directory could pre-plant that
name as a symlink and have the write truncate the file it pointed at.
It now writes through an exclusively created, unpredictably named
temporary file in the same directory.
`save_config` passed 0600 to `os.open`, but that mode applies only when
the call creates the file. Rewriting an existing group- or
world-readable config left the old mode in place for the duration of the
write, so a freshly written literal credential was readable by anyone who
could already read the file until the trailing `chmod`. The descriptor is
now narrowed before any content goes through it, which also removes the
path-based `chmod` that followed.
Both are pinned by a test that fails if either guard is removed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* feat: install with one command, without a Python of the right version
uv brings its own interpreter, so the CLI's Python floor stops being the
installer's problem.
* feat: build on the released clients, against the specs they came from
Both clients are on PyPI, so the pins are exact versions rather than
commits: the CLI derives its flags and help from their signatures, so a
client that moves changes the CLI's surface and each release re-pins on
purpose.
The vendored specs are refreshed to the commits those releases were
generated from, and `provenance.json` now records where each one came
from so a copy taken from anywhere else fails a test instead of deriving
flags the released client cannot carry.
What the refresh moves, and why:
* `mode` is gone from five LLMWhisperer operations, so the highlights
command no longer has a parameter it cannot reach.
* `extract` declares the deprecated `page_seperator` spelling, which the
client also accepts. A deprecated parameter is now skipped, or the same
value would have had two flags.
* `--mode pdf_to_images` is gone: that mode is its own operation.
* Several LLMWhisperer defaults are no longer reported, because the
client stopped pinning them and lets the server choose.
* ci: publish a release to PyPI from a dispatch
Mirrors the two Python clients: a manual dispatch bumps the version,
lints, tests, builds, publishes with `uv publish` through a PyPI Trusted
Publisher, and only then commits the bump, tags it and cuts the GitHub
release -- so a failure anywhere before the publish leaves main untouched.
The version now lives only in `__version__`; pyproject reads it through
hatch, so the bump edits one file.
`version_bump: none` publishes what is already in the repo, which is what
a first release needs: the convention here is that the committed version
is the last released one, and nothing has been released yet.
* ci: cut release candidates before a stable release
`pre_release` now publishes `<version>rcN` rather than flagging the
release on GitHub alone: the rc number counts up from the tags already
published for that target version, so repeat dispatches give rc2, rc3,
and the committed `__version__` is left alone because it names the last
stable release, not a candidate for the next one.
Promoting is the same dispatch with `pre_release` off, which bumps and
commits the version, tags it and publishes it stable.
* fix: answer honestly where a live run found the CLI lying
Every one of these is something a real call reached and a test did not.
Discovery published Click's "no default" sentinel as the string
`Sentinel.UNSET` -- fourteen flags on the newest Click the pin allows,
each reading as a value a caller could send back. The marker stopped
being `None` in 8.2 and is not exported, so it is read off a bare option
and tracks whichever version is installed. CI now runs the suite a second
time against the newest resolvable Click, because `uv run` resolves from
the lockfile and an install in the wild does not.
A required option was given `default=None`, which from Click 8.2 counts
as a value the caller supplied, so `required` was never enforced:
`whisper highlights <hash>` with no line range escaped as a raw TypeError
from inside the client. Requiredness now follows the spec rather than the
client signature -- a signature without a default says only that the call
cannot omit the argument, which the command answers by supplying one --
and `--extract-all-lines` stands in for a line range, as the API allows
and the client's positional argument does not.
The CLI read only `LLMWHISPERER_BASE_URL` while the published client
reads `LLMWHISPERER_BASE_URL_V2`, and `UNSTRACT_DEPLOYMENT_KEY` while the
deployment client reads `UNSTRACT_API_DEPLOYMENT_KEY`. An environment set
up for a client therefore left the CLI on its built-in default, which is
production. Both client spellings are now honoured after the CLI's own.
`--no-wait` handed the acknowledgement to the result finisher, so
`--output raw` printed `null` for a started, billed execution and the
handle appeared only on the path that did not need it. An ack now reports
the handle in both places.
Two `retryable` flags said the opposite of the truth: a poll timeout, the
one failure a caller is meant to come back from, said false, while a host
name that does not resolve said true. The unresolvable host is now named
and reported as final; every other connection failure stays retryable.
Also: `--discover` answered in whatever format `-o` asked for, including
a wrapped table, though it is the machine-readable description and its
own contract tells callers to pass `-o json`; it now always answers as
JSON. And it described only leaf commands, omitting the root's `-o` and
the connection flags each product group carries, which is a description
of a call nobody can make.
* fix: never truncate a working config ahead of a write that may fail
`config set` opened the real file with O_TRUNC and tightened its mode
afterwards, so anything that failed between the two -- an fchmod the
filesystem refuses, a full disk, a value that will not serialise -- left
the user with an empty config and no way back to the one they had.
It is now written to a temporary file and renamed into place. The rename
is atomic, so the previous config survives every failure, and `mkstemp`
creates the temporary unpredictably named and 0600, which is the mode the
config lands at: a guessable sibling in a shared directory is a symlink
waiting to be planted, and a mode widened until after the write is a
window in which the new credential is readable.
Replacing a symlink would quietly turn a deliberate one into a regular
file, so that is still refused rather than followed.
* fix: say why a config in an unwritable directory cannot be saved
Replacing the file rather than overwriting it is what keeps a failed
write from destroying a working config, and a rename needs the directory
even when the file itself is writable. Writing in place when the
directory refuses would put the truncate back exactly where recovery is
hardest, so the case is reported instead, naming the directory.
* fix: make raw output and a dead DNS guard answer honestly
`--output raw` printed one declared field, which was wrong for most of
what these commands return. A queued run's acknowledgement carries no
result and names no execution -- the handle is only inside the endpoint
it hands back -- so the field was missing and the whole payload was
dumped instead. A status read on a running job carries the field as
`null`, so raw printed `null` and exited 0, which a caller polling for a
result cannot tell apart from a job that finished with nothing.
Commands now declare what raw prints best-answer-first, and raw prints
the first one the answer actually carries, looking in `meta` too so a
handle the CLI had to derive is still reachable. An answer carrying none
of them fails and says which were looked for, rather than printing
something nobody asked for. `--discover full` publishes the whole list,
so what it advertises covers every shape the command returns.
The unresolvable-host guard could never fire. It read a urllib3
structure, and both clients are httpx-based and re-raise transport
failures carrying only a message, so nothing structural survives at the
top. It now reads the cause chain for the resolver's own error, which is
there whichever transport asked, and takes the host from the request the
error carries. Its test built the urllib3 shape by hand and so could not
fail; the stand-in is now produced by putting a transport error through
the client's own translation, and a second test, skipped unless asked
for, reaches a name no resolver will answer for.
Also:
* Discovery published a different default per Click version for on/off
flags -- `False` on some, nothing on others -- when omitting one sends
nothing at all. Both spellings are read the way Click reports them, and
CI now diffs the whole discovery payload across the two versions it
tests, so the contract cannot vary with the installed Click again.
* A target that is not one of the configured aliases is sent as an API
name, which is a supported way to name a deployment and also what a
misspelt alias looks like. The failure now names the aliases that exist.
* The rejected-key hint blamed a foreign organisation, which cannot
produce it: the resource is resolved within its organisation first, so
that answers 404.
* Highlights on an extraction made without line numbers pointed at the
call that cannot be fixed. The hint now names the extract flag.
* No command mounts a required derived flag, so the fix that keeps Click
enforcing one had nothing live to protect. Pinned directly.
* ci: compare both ends of the click range, not the lockfile's middle
The parity check compared the lockfile's click against the newest one,
and those two agree even with the bug it was added to catch: the answer
only diverges at the floor the pin allows. It now runs the suite and
takes the discovery payload at both ends of that range and diffs those,
which is what the pin promises to support.
* fix: put the new config on the disk before the rename that stands for it
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: refuse a symlinked --save target instead of replacing the link
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: check the save target again at the rename, not only ahead of the read
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: keep one symlink check on the save path, at the point it can still help
Checking again at the rename cannot close the window it aims at, and the
rename replaces the link rather than following it, so the check that fires
before the result is read is the one worth keeping.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* ci: install from the lockfile, on the uv the other packages release with
Resolving the dev extra fresh ignored uv.lock, so neither gate ran the
versions an install resolves to.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* docs: add a skill for bumping the client pins and their specs
A client bump here is three coupled edits -- the exact pin, the vendored spec
re-copied from the commit that client was generated from, and the provenance
sha -- and the coupling is only discoverable by tripping test_specs.py or
test_contract.py and working backwards.
Records the order as a repo-local skill, including the part that is easy to get
wrong: derived_flags.json is refreshed deliberately after reading what moved, a
flag missing from it being a flag the CLI has stopped offering. Also pins the
release shape: publish before tag, `none` for the version already committed, and
pre_release for the rc flow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CSm32dsxD56PkKENgav3Mn
* docs: check all four provenance fields, and hold version_bump across an rc promotion
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CSm32dsxD56PkKENgav3Mn
* Remediate review findings across the CLI scaffold
Four review iterations plus a simplification pass over the CLI, its config
resolution, output envelope and poll engine. The published surface is
unchanged: `--discover full`, the exit-code table, the `{ok, data, error,
meta}` envelope and every existing flag verify byte-identical against the
parent commit.
Security
- Credential-named fields are matched on whole name segments split at
camelCase boundaries, so `accessToken`, `secretAccessKey` and
`authorization_header` are recognised. Case was previously folded before
the split, collapsing those to one unrecognisable word, and only the
trailing segment was tested -- so a server error body echoing them
reached stdout intact.
- A value under a credential-named key is collapsed whole rather than
walked, since nothing beneath such a key is worth the risk of missing one.
- `InvalidHeader` is caught ahead of every other transport failure on both
the shared client path and the clone path. Its message quotes the
offending header value -- the credential -- `repr`-escaped, so the literal
scrub cannot match it either.
- Diagnostics on stderr are scrubbed like stdout; a note can carry
server-authored text, and a credential is no less leaked for arriving on
the other stream.
- A discovered project-local config may not choose which environment
variable is read. `_deref`'s trust boundary has no default, so a caller
has to ask for the permissive branch.
Correctness
- A refused retrieve (408, 429) stays retryable. The blanket un-marking of
the one-shot read also flipped a rate limit, so an agent branching on
`retryable` would discard a paid extraction still on the server.
- A poll spec naming one status as both success and failure is rejected at
construction; `classify` tests failure first, so such a status would be
reported as an error.
- Terminal states are case-folded once at construction rather than on every
poll.
- `to_dict` derives its reserved names from the payload it builds, so a
field added there cannot be forgotten in the guard that stops `extra`
from rewriting it.
- A flag whose spec type the CLI cannot convert is marked `unsupported` in
`--discover`, instead of reading exactly like one it can.
Tests
- 33 added, each verified by removing the fix and confirming the test
fails. Eleven fixes from an earlier round were mutation-green and now
carry tests that bite.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VcqRYgWLM6s8mFKZbryvGt
* fix: make the release workflow parseable, and keep it that way
The release step declared `env:` twice. Read strictly that is a duplicate
mapping key and the file is invalid; read leniently the second block wins
and RELEASE_NOTES is silently never set, so every release takes the
generated-notes branch. The blocks are now one.
Nothing caught it because a workflow is only parsed when it is dispatched,
and this one had not been. The suite now reads every workflow the strict
way at PR time, and checks that a step reading $GITHUB_TOKEN is a step
something sets it on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: let a run name its documents as presigned URLs
FILES was required and checked for existence on disk, so the
--presigned-urls flag the spec derives could be advertised by --discover
but never used: a URL is not a local file, and omitting FILES failed in
Click before the command ran.
FILES is now optional and the two sources are checked together, which is
where the real requirement lives -- a run naming no documents at all. The
check runs before the client is built so a malformed invocation is not
reported as a missing credential.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: give an accepted extraction a raw answer of its own
`whisper extract --no-wait` finished without naming its raw fields, which
falls back to printing the whole acknowledgement -- the one thing raw
output exists not to do.
An accepted job carries a handle and no text, so the declared list is now
the text then the handle, and raw prints whichever the answer has. That is
the same shape a queued deployment run already used.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: write the starter config where it will be honoured
Without --config or $UNSTRACT_CONFIG, `config init` took whatever path
resolution returned -- including a project .unstract.toml found by walking
up from the working directory. Two things went wrong there: the file the
caller never named got created or overwritten, and the starter profiles it
writes are credential indirections a discovered file is not trusted to
supply, so the next command refused the config init had just produced.
Discovery stays a way to read a config, not a way to choose where a new one
goes. A named path is still the target wherever it points.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: read past an empty field when picking a raw answer
The clients answer with an empty string for a field that has no value yet
rather than omitting it, so raw output stopped at the first declared field
every time and printed a blank line for a queued run instead of its handle.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: keep a rescued result whole when a save fails
Where `details` is the only surviving copy of a result the service will not
serve again, redacting it by field name destroys the part of the answer the
caller is being handed it to recover. The literal scrub of every resolved
credential still applies on the way out.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: render a failure in the format the run resolved
Reading `-o` out of argv by hand only recognises the spellings it was written
for, so a clustered `-ojson` succeeded as JSON and failed as a table. The root
callback now fills in a context the entry point holds, leaving the argv scan
for failures that happen before any of it has been parsed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: do not lose a finished report to an unreadable config
Resolving a credential to scrub for goes through the context's config, which
raises a CLI error rather than the config error the handler was written for.
A command that takes its endpoints as flags would then exit on a config file
it never needed, discarding a report of work already done.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: name the execution id when a run times out
The handle a run polls on is a status URL, and the status command takes an
execution id, so a timed-out run told the caller to resume with something no
command accepts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* test: cover building a deployment client
The alias branch, the bare-name fallback and the URL the client reads its
organisation back out of were reached only through tests that stub the builder
out entirely, so nothing exercised the route itself.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* ci: publish only after the revertible release steps have run
A tag, a branch and a release can all be deleted; a version on PyPI cannot.
Publishing first meant any later failure left a released version that no tag
in the repository names, which is the one outcome that cannot be cleaned up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* docs: state how a deployment alias and the connection flags interact
An alias that names its own org or key keeps them: the flags fill in only what
it leaves to the profile, while --base-url is not per-alias and always applies.
Pinned by a test so the documented precedence cannot drift silently.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: bound the poll interval and floor the backoff
Zero seconds between polls is a busy loop against a metered service, and
doubling a zero interval never grows it, so the backoff after a rate limit
answered at the rate that earned it. The flag now refuses it and the loop
floors it for callers that do not come through a flag.
Discovery publishes the bounds as their own keys: Click names a bounded number
"float range", which is not a type a caller can map onto anything.
The sleep seam is resolved on the call rather than captured at import, so
replacing it in a test reaches the loop -- which it previously did not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: preflight the write a save actually performs
The result is written to a temporary sibling and moved over the target, so the
directory is what must be writable. Opening the target itself passed for a
writable file in a read-only directory and failed only after the one-shot read
the flag exists to protect.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: stop reporting a local fault as a retryable server failure
Everything the service raises on purpose is already a CLI error by the time it
reaches the poll loop, so what the bare handler catches is this side's own bug.
Labelling it a retryable server error repeated it until the retry budget ran
out and then blamed the service.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* feat: publish what omitting a spec-derived flag gets you
The CLI leaves these flags without a Click default so that nothing is resent,
which left the value the client or the service applies readable only as a
sentence inside the help text. Discovery now carries it as `server_default`.
The spec states some of those defaults in prose of its own, so a flag could
carry two statements of one default -- and one of them was already wrong.
Stripped, leaving the rendered value as the single statement.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* test: check the token where gh actually reads it
The check looked for `$GITHUB_TOKEN` spelled out in a run block and so matched
nothing: `gh` takes its credential from the environment without naming it. It
now matches the steps that shell out to `gh`, and a guard fails if that stops
matching anything rather than letting the check pass vacuously.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: answer a bare invocation in the format that was asked for
Printing help on stdout and exiting 0 tells a parser the run succeeded and then
hands it a page of prose in place of the envelope. A group invoked with no
command already answers with a usage error; the root now does the same, and a
person still gets the help page.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: put every diagnostic behind --quiet
The config, overlay and credential registries are imported by the output layer
and so cannot import it back; each wrote straight to stderr, which left three
notes that --quiet did not reach. They now go through a sink the run binds.
Notes raised while the command tree is built are held until there is a run to
ask, and anything still held when the entry point returns is written out rather
than dropped for having been early.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* keep the parts of a config file this CLI does not own
`save_config` rebuilt the document from `default_profile` and `profiles`
alone, so writing a profile deleted every other top-level table in the
file. The parsed mapping is kept on `ConfigFile` and the write starts
from it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* say on stderr when a clone left something behind
A skipped, oversize or unsupported file does not fail the run, so the
exit code says nothing about it and only the table renders the counts.
The summary now goes to stderr in every format, under --quiet like any
other diagnostic.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* refuse a config setting the product does not have
`config set` wrote any key under any product, so a typo was stored and
silently never read; `doctor` now names a key already sitting in a
product block that nothing resolves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* name an overlay entry the specs do not declare
An entry for an unknown product, operation or parameter applied nothing
and said nothing, so a short flag or a narrowed value list could be
written and never take effect. The flag snapshot also records what the
overlay resolves, so a narrowing that stops applying moves it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* cover the failure paths that had no test
Timeout translation and the clause order that decides where a connect
timeout lands, the clone command's Platform API, start-up and abort
translations, `whisper webhook update`/`delete` with the token they
carry, and the entry point's closed-pipe and OSError arms.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* drop the header redaction nothing calls
No caller builds or prints a header map, so this redacted nothing while
reading as if headers were covered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: give the deployment client a socket timeout by default
The deployment client sets none of its own, so a stalled connection was
waited on forever unless the flag was passed. Default to the 120s the
LLMWhisperer client applies; `--transport-timeout 0` keeps the old
behaviour for a caller who wants it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* ci: authenticate the release push the way the other repos do
`create-github-app-token@v3` takes the App's client id, and the org's
existing variables are named for it, so this repo can share them rather
than needing an App of its own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* docs: say what the vendored-spec hash does and does not catch
The hash and the file move in the same commit, so the check cannot tell a
deliberate edit from a refresh. It catches a copy that was corrupted or
half-updated, and a provenance entry left behind by its file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* test: tie each vendored spec to the client pin it was synced for
provenance.json now records the exact pin each spec was copied for, and
tests/test_specs.py compares it with the pin in pyproject.toml. A client
bumped without its spec re-synced fails locally, with no network, instead of
deriving flags the released client cannot carry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: a completed run with a failed document exits VALIDATION
The service reports a batch as COMPLETED even when a document inside it
failed, with the failure carried per file in extraction_result. The CLI read
only the execution status, so a caller branching on the exit code was told
the batch succeeded with a document's output missing. Both `deployment run`
and `deployment status` now walk the per-file results and fail with the
failed files named; the full payload is kept verbatim in error.details,
since the status read is one-shot and the successful documents survive
nowhere else, and --save still writes it before the error is raised.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: raw prints an empty extraction rather than the field after it
Reading an empty string as "no answer yet" was true only of the deployment
client, which spells a pending result that way. The LLMWhisperer client
returns empty text for a document with nothing on it, so `-o raw` printed
the hash in its place, or failed and spent the one-shot read. Empty now
means pending for the one field that spells it so, and is the answer
everywhere else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: the resume command a timed-out run suggests keeps --save
The hint is paste-ready, and the status read it names is one-shot: a caller
who asked for --save and pasted it as given spent the read with nothing
written to disk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* test: the rescued partial-failure payload is checked against redaction
The fixture carried no field the redactor matches, so the assertion that
the successful documents survive verbatim passed with or without it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* test: a per-file entry with an error and no status counts as failed
The rule existed and nothing exercised it: the fixture carried a status on
every entry, so dropping the branch left the suite green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* ci: keep the release a draft until the package is on PyPI
Publishing last means a failed publish leaves a public release naming a
version nobody can install, and the tag guard stops the rerun that would
fix it. The release is created as a draft and made public only after the
publish succeeds, so a failure at any point leaves nothing public.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* docs: drop the import claim from warn()'s docstring
It named an import relationship that does not exist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* docs: say what the recorded pin check compares
The test compares two recorded strings; it cannot tell whether the spec
was re-synced for the pin it names.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* test: the broken-pipe redirect is asserted to happen
Under a captured sys.stdout the descriptor lookup raises before the
redirect runs, and the guard around it swallowed that too, so the test
passed with the redirect deleted. It now runs against a real descriptor
and checks the redirect targets stdout.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* docs: name the per-file status spellings the check folds
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: a directory sync that fails after a config write is reported
The suppress existed because Windows cannot open a directory to sync it,
but it also hid a real failure on every other platform and reported the
write as fully durable. The sync is now skipped only on Windows; elsewhere
a failure is warned about, since the config has already been renamed into
place and the exit code stays what the write earned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* chore: trim comments that narrate rather than state a constraint
The long ones restated what the next line does, carried detail that goes stale
(library versions, exact counts) or explained a decision at more length than
the code it sits above. Each is now one or two lines saying why, readable
without knowing how the code got there.
Behaviour is untouched: every changed file parses to the same AST once
docstrings are stripped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* fix: sync the directory entry a saved result is renamed into
`persist` fsynced the temporary file and renamed it over the target, but never
synced the directory the rename wrote to. A crash between the two can leave the
bytes on disk with nothing pointing at them -- and a retrieval that can only be
read once has no second copy to recover from.
The directory is synced after the rename, and a failure to sync it is raised
like any other save failure, with the payload on `details`: the file may be
there, but nothing can promise it is, and the caller has to be handed the
result either way.
`save_config` already did this, and warns rather than raises because a config
can simply be written again; that difference is deliberate and left as it is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* chore: drop comments that argue the design rather than guard an edit
A comment earns its place by stopping a plausible edit from being wrong: an
ordering requirement, a scrub bypass, a one-shot read, a trap in a library.
Comments that only justify a decision the code already makes are removed, and
constraint statements padded with argument are cut to the constraint.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* ci: test the floor of the click range and the built wheel, and fence the release to main
The release workflow checks out whatever ref dispatched it. A candidate or a
`none` bump reaches PyPI with no commit to review, so the job now refuses to
run off main, and the App credentials are checked before the checkout rather
than failing as an empty token further down.
`click~=8.1.0` installs the newest 8.1.x, which is not the floor the package
declares; the floor is now installed exactly. A new job installs the built
wheel into a clean environment and exercises both entry points, `--discover
full`, the usage-error envelope and its exit code, and `config init` -- none of
which a test run inside the source tree can fail on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
* feat: say in the envelope when a config write is not yet known to survive a crash
`save_config` warns on stderr when the directory entry could not be synced
after the rename; a JSON consumer reading only stdout saw a plain success.
`config init` and `config set` now add `meta.durability: "unconfirmed"` in
that case. Nothing changes when the sync succeeds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent 13cb3f7 commit e860c65
46 files changed
Lines changed: 15528 additions & 0 deletions
File tree
- .claude/skills/bump-client-pins
- .github/workflows
- src/unstract_cli
- commands
- core
- specs
- tests
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
0 commit comments