fix(llm): provider errors told apart, size rejections repacked, server chat templates, argument hints, literal grep, write-time warnings (F29–F34, F24) - #435
Open
plombeer31 wants to merge 8 commits into
Conversation
A tool's argument error named the key it wanted and nothing else, so a model that had sent `patternes`, `"path"` (quotes included) or a key fused with prompt markup retried the same call blind. - batch-executor: an error a tool throws on its arguments now ends with `received keys: …; expected: …` (expected from the tool's args schema) and `did you mean X instead of Y?` for any received key within two edits of an accepted one. Keys only, never values. Runtime failures (ENOENT, timeouts) get no key report. - coerce-tool-args: before dispatch, a quoted key (`"path"`, `\"path\"`) and a fused label fragment (`<label>…</label>,limit`) are renamed to the schema key they were meant to be — only when the clean key is in the schema and the model did not also send it.
`os.fs.grep` always ran the pattern as a regex, so a code fragment such as `.add(` failed with a raw ripgrep "regex parse error: unclosed group" and the model had to guess at escaping. - `literal?: boolean` maps to `rg -F` (fixed-string search); added to the descriptor's argsSchema and the native-tools JSON schema. - A ripgrep regex parse error now ends with a hint naming `literal: true` (also in `details.hint`); nothing is added when literal was already set or the failure is something else.
…double-escaping Patch 4's parse check knew `.js`/`.json` only. A local model then wrote an `index.html` with 16,627 bytes of serialized tool-call text after `</html>`, and a `game.js` whose newlines were all literal `\n`; both were found many steps later. New `fs-content-check.ts`, warn-only like the parse check, composed with it through one entry point (`checkChangedFile`) that `os.fs.write`, `os.fs.edit` and `os.fs.patch` now call: - `.html`/`.htm`: every inline classic `<script>` block is parsed with the JS checker (module, JSON and `src=` scripts are skipped) and the error line is the line in the file; non-whitespace content after `</html>` is reported with its size and start line. - any text: a line starting with `assistant_tool_call:`, `tool_result[`, `<|channel|>` or `<|turn>`; more literal `\n` sequences than real newlines (four or more, so a one-line JSON string is not an alarm). An edit or patch reports only the problems it introduced or made worse; a write reports everything.
…ited, generation ids kept A 429 carrying `credit_balance_exhausted` was parked and retried as rate limiting (42 times per worker); an OpenRouter 402 `in_flight_budget_exhausted` with a "retry in 120 s" hint ended a run as final; a 504 after 10,528 streamed tokens left the billed generation invisible. The status said one thing, the body another. - `parseProviderErrorBody` (pure) reads the OpenAI and OpenRouter shapes, OpenRouter's upstream `metadata.raw`, and textual cooldowns; `readProviderErrorReason` turns them into `credit_exhausted` / `retry_after`. `OpenAiHttpError` carries the parsed `body`. - `readProviderErrorVerdict` (reliability) walks the cause chain from the loop's TransportError to the HTTP error and names the provider. - The HTTP client no longer spends its three retries on a 429 whose body says the credit is gone. - Agent loop: `credit_exhausted` stops the turn where it is, resumable (`stopCause: "credit_exhausted"`, `task_stopped:credit_exhausted`, a synthetic reply naming the provider, a `credit_exhausted` event the TUI renders as one yellow line). A cooldown hint is waited for as asked, clipped to 180 s, inside the existing outage budget — a 402 with a hint is waitable now; a plain 429 keeps the backoff park. - Fusion workers report a credit stop as `failed` with the quota hint. - SSE generation ids: every chunk's `id` reaches `CompletionResult.generationId` and the `llm_completion` trace row; a mid-stream `error` event throws a typed error and a body that died after output carries the id, which the trace `error` row records.
…try once
A model without a catalog entry is assumed to have 128K; a 400/413
about the context length ended the turn with no repack, and a learned
window was forgotten the moment a later completion was larger.
- `planSizeRejectionRepack` (agent): on `isRequestSizeRejection` from a
native-tool link, learn the window from the body's own number
("maximum context length is N tokens", "N-token context window", …)
or, failing that, 0.8 × the loop's estimate of the refused prompt;
observe it through the existing learned-window path, retry the same
step once with a `### notice` saying history was trimmed to fit, and
emit `prompt_repacked` (TUI line, trace row). A refusal that names
only the reply cap, the loop's own raised-cap retry, or a second
refusal on the step still fail as before.
- `readContextLengthFromRejection` / `requestSizeRejectionNamesContext`
on the size-rejection classifier.
- Learned windows persist and only move towards what the server showed:
`LearnedContextWindows` (extracted from bootstrap) lowers on an
observation and raises — no longer forgets — when a completion held
more. AGENTS.md invariant 6 updated accordingly.
- `resolveModelPricingFor` falls back to OpenRouter's cached live model
list (context_length, prices) for a model neither configured nor
bundled, so the assumed 128K only applies when nothing is known.
…y use The claude and codex CLI providers announced `native_tools`, so the prompt's persona said "never write tool-call JSON as text" and the instructions pointed at a `tools` payload that is never forwarded to the CLI — while Codex's own steering text asked for the JSON array. - `SubscriptionCliProvider` reports `toolTransport: "grammar"` (the JSON-array text transport in this runtime; there is no separate "text" value) with no tool-call adapter. The stable prefix now renders the JSON-array persona, rules and emission instructions for it, and the step executor parses the call out of `content`. A format drift costs the parser's one-shot retry (a second CLI invocation) instead of the native recovery ladder. - `llm-link-attempt` forwards `responseFormat` on grammar links too, so the CLI's `--json-schema` staging keeps serving the memory sub-calls; llama-server ignores the field in favour of the grammar.
Only Gemma and Qwen had hand-built prompt profiles; every other GGUF
ran as plain-instruct with no turn markers, no reasoning handling and
no way to switch thinking off on /completion.
- `localModels.useServerTemplate: auto|on|off` (default auto = on for
every family without a hand-built profile, i.e. plain-instruct) and
`localModels.thinking: auto|on|off`, parsed with defaults so older
files need no change (the version bump is its own commit).
- `resolveServerTemplatePolicy` decides once per step from config and
the active profile. When on, the step executor builds a framing-free
prompt (no prefill, no turn framing — as for a chat transport) and
hands the grammar link `chat: { system: <stable prefix>, user:
<tail>, prefixHash, enableThinking? }` next to the raw text.
- `LlamaServerClient.applyTemplate` posts `POST /apply-template` with
`messages` and `chat_template_kwargs`; `ServerTemplateRenderer` (in
the llama-server provider) renders the prefix once per stable-prefix
hash, model and thinking setting with a sentinel in the user slot and
splices each step's tail in, so the rendered head stays byte-stable
for the KV cache. A 404 disables the path for the process; any other
failure or a template that drops the sentinel sends the raw prompt.
The GBNF grammar is kept.
- `ModelProfile.supportsThinkingSwitch` marks templates that read
`enable_thinking`; the thinking setting only applies there.
- `ProviderFactoryContext.getModelId` keys the renderer cache on the
live model so a hot swap never reuses another template's framing.
…nking v66: the two F31 fields, both defaulting to auto; v65 files are accepted and inherit the defaults. Everything else in fix/errors is independent of the exact number, so the integrator can squash this with other bumps.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Before: a
429 credit_balance_exhaustedwas retried as rate limiting (168 requests), a402 in_flight_budget_exhaustedwith a retry hint ended a run, and a 504 after 10K streamed tokens lost its billed cost; a context-length 400 ended the turn with no repack and a learned window was forgotten; GGUFs without a hand-built profile ran as plain-instruct with no turn markers and no thinking switch; the claude/codex CLI providers announced native tools they never forwarded; argument errors said "pathmust be a non-empty string" to a model that had sent"\"path\""orpatternes;os.fs.grephad no literal mode and.add(failed with a raw regex error.After: HTTP error bodies are read — exhausted credit stops the turn resumable and names the provider, a retry hint is honoured up to 180 s, a stream cut after output records its generation id; a size rejection learns the window from the body (else 0.8 × the estimate), repacks and retries once with a notice, learned windows persist and only grow, and
context_lengthcomes from OpenRouter's live catalog when no entry exists;localModels.useServerTemplate(auto/on/off) renders local prompts through llama-server's/apply-templatefor families without a profile, withlocalModels.thinkingmapped tochat_template_kwargs; subscription CLIs declare the text transport; argument errors append the keys received and expected with the nearest match, and quoted or fused keys are normalised before dispatch;os.fs.grep { literal: true }maps torg -Fand the parse error suggests it; writes warn on inline-script parse errors and content after</html>, transcript markup at line start, and double-escaped content. Config bump for the new fields.Why
Each of these turned a one-line fix into a blind retry loop or a dead run. Error wording is provider-specific, so a small table with a generic fallback; retry hints are bounded so a provider cannot park a run; write-time checks warn only, never block.
How it was verified
npm run lintcleannpx vitest run --minWorkers=1 --maxWorkers=3 src/agent src/cli src/config src/llm src/llm/provider src/llm/provider/llama-server src/llm/provider/openai src/llm/provider/registry src/llm/provider/subscription-cli src/llm/reliability src/prompt src/runtime/bootstrap.test.ts src/runtime/learned-context-windows.test.ts src/runtime/llm-link-attempt.test.ts src/runtime/resolve-model-pricing.test.ts src/tools src/tools/fusion src/tools/os src/tracing/trace src/tui/agent-event-reducer.test.ts— 308 files / 4118 tests green, 3 skipped (baseline failures: none)Stacked on #434; merge in order.