feat(llm): native messages, prompt caching and per-model params for cloud providers (F26, F27, F28)

AGENTS.md

@@ -569,6 +569,8 @@ Text completion, vision, embeddings, and sub-calls route through plugin-register
 - **Name escape** — qualified tool names use `__` for dots (`os.fs.read` → `os__fs__read`) in [openai-tool-call-adapter.ts](src/llm/provider/openai/openai-tool-call-adapter.ts). `reply` / `finish` are synthetic OpenAI functions alongside registry tools.
 - **Vendor presets** ([src/tui/providers/provider-presets.ts](src/tui/providers/provider-presets.ts)) — 19 named cloud/local endpoints (Anthropic, Groq, Moonshot, Perplexity, Qwen/DashScope, SambaNova, …) that all resolve to the existing `openai-compatible` kind with `baseUrl` prefilled. Adding a vendor is a preset entry, not a provider kind. The one documented exception is `subscription-cli` — a subprocess backend has no baseUrl, no key and no HTTP path, so a preset cannot express it; within that kind the preset philosophy re-applies one level down (a new vendor CLI is a descriptor entry, never a new kind). Vendors that do not authenticate with `Authorization: Bearer` set `apiKeyHeader` (Anthropic: `x-api-key`) plus any mandatory static `headers` (Anthropic: `anthropic-version`); both are copied onto the saved config entry by [providers-wizard-build-entry.ts](src/tui/providers/providers-wizard-build-entry.ts) and applied to **both** request paths by the single [openai-auth-headers.ts](src/llm/provider/openai/openai-auth-headers.ts) builder, so discovery and chat cannot disagree. The bar for a new entry: probe `<baseUrl>/v1/models` **with the headers the preset will actually send** and get either 200 with a `data` array, or a 401/403 that rejects the *credential* — a 401 whose body names a header the preset does not send (`x-api-key header is required`, `Invalid bearer token` for what is an API key) is a **failing** probe, not a passing one. Either way the same host must answer 404 for a bogus sibling path; a gateway that rejects everything before routing proves nothing.
 - **OpenRouter provider routing** — `llm.providers[].providerPreferences` is sent verbatim as the body's `provider` object by `buildOpenAiChatBody` (turns and sub-calls, streaming and unary) and `describeImageViaOpenAi` (vision). Only the `openrouter` factory forwards it: no other kind documents a `provider` field. It is set *before* the `extraBody` merge, so an explicit `extraBody.provider` — the old workaround — still wins. Deliberately **not** sent by `verifyProviderKey` (it probes the cheapest paid model, which a host pinned for the operator's model may not serve, and would misreport a good key as `model_unavailable`), the contract probe (built from wizard state, which carries no entry passthroughs — `extraBody` is absent there too), the catalog fetch (`GET /models`), or OpenRouter embeddings (a pin chosen for a chat model's hosts would strand an embedding model). Pinned by [openrouter-provider-routing.test.ts](src/llm/provider/openrouter/openrouter-provider-routing.test.ts) and [register-built-in-providers.test.ts](src/llm/provider/registry/register-built-in-providers.test.ts).
+- **Prompt caching (F27)** — Anthropic caches nothing without `cache_control` markers. For `anthropic/…` / `claude…` models and for Anthropic's own host, [prompt-cache-control.ts](src/llm/provider/openai/prompt-cache-control.ts) places two ephemeral breakpoints on the native layout: the system message and the last history message before the tail (never inside the changing tail, which would pay the cache-write premium for nothing; never on a `tool_calls`-only assistant message, which has no text part). `llm.providers[].promptCache` decides: `off` sends none, `explicit-markers` always sends them, `auto` (default) sends them for Anthropic only. Google models on OpenRouter cached 83 % of input on Google AI Studio and 1–4 % on Vertex, so an `openrouter` entry with no `providerPreferences` of its own sends `{order: ["Google AI Studio", "Google"], allow_fallbacks: true}` for `google/…` models, behind `llm.openrouter.preferCacheRoutes` (default on). `prompt_tokens_details.cached_tokens` is read on both paths into `usage.cachedTokens` and `cacheHitTokens` (the trace's field); the cost accumulator and the turn usage meter price cached tokens at the model's `cacheRead` rate ([usage-cost.ts](src/llm/provider/usage-cost.ts)). Pinned by [prompt-cache-control.test.ts](src/llm/provider/openai/prompt-cache-control.test.ts), the cache-route cases in [openrouter-provider-routing.test.ts](src/llm/provider/openrouter/openrouter-provider-routing.test.ts) and [usage-cost.test.ts](src/llm/provider/usage-cost.test.ts).
+- **Reasoning fields, tagged calls and per-model parameters (F28)** — the provider's `reasoningFormat` defaults to `auto`, which reads `reasoning`, `reasoning_content` or `thinking` on both the streamed and the unary path; `userModels[].reasoningFormat` pins one field. A reply that is nothing but `<tool_call>` blocks — Qwen's `<function=…>` form or the Hermes `{"name", "arguments"}` form — is decoded as tool calls on every OpenAI-compatible kind ([qwen-tagged-tool-response-adapter.ts](src/llm/provider/openai/qwen-tagged-tool-response-adapter.ts)); prose around a block keeps it a reply, and only the Qwen kind also reads the reasoning channel (#105). A streamed chunk carrying both text and a tool-call delta keeps its text. OpenAI reasoning models (`o1`…`o9`, `gpt-5`, with or without a vendor prefix) get no `temperature` and `max_completion_tokens` instead of `max_tokens` ([model-params.ts](src/llm/provider/openai/model-params.ts)); `userModels[].params` merges per-model wire parameters over the body and `extraBody` (reserved keys still win); `CompletionRequest.reasoningEffort` is spelled `reasoning: {effort}` on OpenRouter, `reasoning_effort` on OpenAI-compatible kinds and omitted elsewhere.
 - **Bundled catalogs** — `OPENROUTER_MODELS_CATALOG` (split across `openrouter-frontier-chat-models.ts` / `openrouter-open-weight-chat-models.ts`) and `AIMLAPI_MODELS_CATALOG` are offline snapshots regenerated from each vendor's public `/models` endpoint; the shared row builders live in [model-catalog-entry.ts](src/llm/provider/model-catalog-entry.ts). Refresh = re-pull the endpoint, remap (`context_length`, `input_modalities` → vision, `supported_parameters` → tools, price × 1e6 → USD/1M) and update the date in each file header. `scoreChat` in the OpenRouter fetcher **ranks** vendors; it must not gate them — the Anthropic/Gemini exclusions it used to carry hid ~40 served models from the picker.
 - **Model search** ([src/llm/provider/model-search.ts](src/llm/provider/model-search.ts)) — one ranked, multi-term scorer over model ids plus catalog metadata (vendor, `vision`/`text`, `tools`, `cache`, context shorthand like `1m`, `free`/`cheap`/`routed`). Tag matching is exact equality, so a context window is tagged three ways — as displayed (`1.0m`), floored to the whole unit (`1m`, the bucket a window falls in rather than a `>=` filter: 1_310_720 answers to both `1m` and `1.3m`, a 2M window only to `2m`), and, when the window is an exact multiple of 1024, in binary (131_072 answers to `128k`). Add a tag rather than changing [format-model-details.ts](src/llm/provider/format-model-details.ts): the display string is what the rows render. Terms are ANDed, matches are ranked (exact id > id prefix > vendor > word start > substring > subsequence) and equal ranks keep input order so the picker does not jitter per keystroke. Used by `filterModelIds` (TUI modal picker + Cloud pane) and by `atomic-agent models search`. Row rendering is shared through [format-model-details.ts](src/llm/provider/format-model-details.ts) — do not re-implement the price/context/capability strings in a frontend.
 
@@ -606,7 +608,7 @@ Pinned by [find-strict-schema-violations.test.ts](src/llm/provider/openai/find-s
 ### Locked invariants
 
 1. **Local llama-server path unchanged when no cloud provider is active.** Grammar, slots, and GBNF tests remain the reference behaviour.
-2. **Stable prefix untouched.** Cloud providers receive the same monolithic prompt string; no chat-message refactor in v1.
+2. **Stable prefix untouched; native-tools links get native messages (D5).** The stable prefix is still built once, byte for byte, and `buildPrompt` still returns the monolithic `text` every grammar link (llama-server, its GBNF prelude and KV layout) and every subscription CLI sends. Beside it, `BuiltPrompt.messages` exposes the same prompt as structure — `system` (the prefix), the packed turns with tool-result bodies capped exactly as the text form caps them, and the tail without `### conversation` — and the step executor puts it on `LlmStreamParams.messages` for a native-tools primary only. `llm-link-attempt` forwards it to a native link alone; `buildOpenAiChatBody` lays it out as `system` + assistant `tool_calls` (ids `call_<row index>`) answered by `tool` messages + one final `user` message ([openai-native-messages.ts](src/llm/provider/openai/openai-native-messages.ts)). Why: every cloud request used to be one `user` message holding the history as `assistant_tool_call:` / `tool_result[…]` text, and Gemini Flash kept continuing that text instead of calling tools (84,931 characters in one completion). `llm.providers[].messageShape: "flat"` restores the single message for a server that rejects tool-role messages, and a 400 about roles / `tool_call_id` / `messages` flips a session to flat and resends once (logged). The fabricated-transcript detector and stream watcher stay as the net for both shapes. Pinned by [openai-native-messages.test.ts](src/llm/provider/openai/openai-native-messages.test.ts), the message-shape cases in [openai-provider.test.ts](src/llm/provider/openai/openai-provider.test.ts) and [openai-build-body.test.ts](src/llm/provider/openai/openai-build-body.test.ts), [llm-link-attempt.test.ts](src/runtime/llm-link-attempt.test.ts) and the structured-prompt cases in [build-prompt.test.ts](src/prompt/build-prompt.test.ts) / [step-executor.test.ts](src/agent/step-executor.test.ts).
 3. **One inference per step survives.** `toolCallsToBatch` produces the same `ToolCallBatch` shape as `parseToolCalls`.
 4. **`atomic-agent serve` never proxies upstream.** HTTP `/v1/chat/completions` always funnels into `runtime.runTurn`.
 5. **Plugin registration only.** New provider kinds call `registerProviderKind`; no central switch statements.

src/agent/step-executor.test.ts

@@ -4548,3 +4548,90 @@ describe("executeStep tool roles (F18)", () => {
     expect(b.params.grammar).toBe(b.baseGrammar);
   });
 });
+
+describe("executeStep — the structured prompt on a native-tools link", () => {
+  const completion = (toolCalls?: CompletionResult["toolCalls"]): CompletionResult => ({
+    content: "",
+    reasoningContent: toolCalls ? "" : "thinking only",
+    stop: true,
+    truncated: false,
+    timing: { promptMs: 1, predictedMs: 1, promptTokens: 20, predictedTokens: 5 },
+    cacheHitTokens: 0,
+    slotId: -1,
+    modelId: "openai/gpt-5.5",
+    ...(toolCalls ? { toolCalls } : {}),
+  });
+  const replyCall: CompletionResult["toolCalls"] = [
+    { id: "c", type: "function", function: { name: "reply", arguments: JSON.stringify({ text: "ok" }) } },
+  ];
+
+  async function run(toolTransport: ToolCallTransport, answers: CompletionResult[]) {
+    const registry = new ToolRegistry();
+    registry.register(replyTool);
+    const base = createEmptySessionState({ id: `s-messages-${toolTransport}`, workingDir: "/w" });
+    const session = { ...base, turns: [{ kind: "user" as const, text: "hi", at: 1 }] };
+    const seen: Array<Parameters<NonNullable<Parameters<typeof executeStep>[1]["llmComplete"]>>[0]> = [];
+    let call = 0;
+    const grammar =
+      toolTransport === "grammar"
+        ? await buildGrammar(PLAIN_INSTRUCT_PROFILE, join(process.cwd(), "grammars"))
+        : "";
+    await executeStep(
+      {
+        session,
+        toolDescriptors: DEFAULT_TOOL_DESCRIPTORS,
+        capabilities: CAPS,
+        skillCatalog: SKILLS,
+        stepIndex: 0,
+        signal: new AbortController().signal,
+        userMessage: "hi",
+      },
+      {
+        registry,
+        slotManager: new SlotManager(2),
+        async llmComplete(params) {
+          seen.push(params);
+          const answer = answers[Math.min(call, answers.length - 1)]!;
+          call += 1;
+          return answer;
+        },
+        grammar,
+        profile: PLAIN_INSTRUCT_PROFILE,
+        toolTransport,
+        toolCallAdapter: null,
+        supportsSlotAffinity: toolTransport === "grammar",
+      },
+    );
+    return seen;
+  }
+
+  it("carries `messages` beside the flat prompt, built from the same packed conversation", async () => {
+    const [params] = await run("native_tools", [completion(replyCall)]);
+    expect(params?.messages).toBeDefined();
+    expect(params?.messages?.system).toBe(params?.prompt.slice(0, params.messages.system.length));
+    expect(params?.messages?.turns).toEqual([{ kind: "user", text: "hi" }]);
+    expect(params?.messages?.tail).not.toContain("### conversation");
+    expect(params?.prompt).toContain("### conversation\nuser: hi");
+  });
+
+  it("re-shapes the structured tail for the one-shot repair, notice included", async () => {
+    const seen = await run("native_tools", [completion(), completion(replyCall)]);
+    expect(seen).toHaveLength(2);
+    const repair = seen[1]!;
+    expect(repair.prompt).toContain("### tool-call-repair");
+    expect(repair.messages?.tail).toContain("### tool-call-repair");
+    expect(repair.messages?.tail).toContain("native function-calling interface");
+    expect(repair.messages?.system).toBe(seen[0]!.messages?.system);
+    expect(repair.messages?.turns).toEqual(seen[0]!.messages?.turns);
+  });
+
+  it("sends none on the grammar transport", async () => {
+    const grammarAnswer: CompletionResult = {
+      ...completion(),
+      reasoningContent: "",
+      content: JSON.stringify([{ tool: "reply", args: { text: "ok" } }]),
+    };
+    const [params] = await run("grammar", [grammarAnswer]);
+    expect(params).not.toHaveProperty("messages");
+  });
+});

src/agent/step-executor.ts

@@ -99,6 +99,7 @@ import {
   type ModelProfile,
 } from "../llm/model-profile.js";
 import type {
+  PromptMessages,
   ResponseFormatJsonSchema,
   ToolCallTransport,
 } from "../llm/provider/completion-types.js";
@@ -115,6 +116,16 @@ export type { PromptCapturedTokens, StepEvent } from "./step-events.js";
 
 export interface LlmStreamParams {
   prompt: string;
+  /**
+   * The same prompt as structure — stable prefix, packed turns, tail —
+   * for a native-tools link that lays history out as real chat messages
+   * instead of one user message of transcript text (which Gemini Flash
+   * kept continuing as text instead of calling tools). Set only when the
+   * primary transport is `native_tools`; the seam forwards it on that
+   * transport alone, so a grammar fallback link still gets `prompt` /
+   * `grammarPrompt`.
+   */
+  messages?: PromptMessages;
   /**
    * Lazy grammar-transport variant of `prompt`. Set when `prompt` was
    * built prefill-suppressed for a native-tools primary while the
@@ -588,6 +599,7 @@ async function executeStepInner(
   const llmParams: LlmStreamParams = {
     ...buildLlmStreamParams({
       promptText: prompt.text,
+      promptMessages: prompt.messages,
       deps,
       grammar: stepGrammar,
       slotId: slot.slotId,
@@ -975,6 +987,24 @@ async function executeStepInner(
         deps.toolTransport,
         promptCarriesPrefill,
       ),
+      // The structured prompt must be repair-shaped too, or a native
+      // link would replay the stale tail without the notice. The notice
+      // lands at the end of the final user message; the chat form never
+      // carried a prefill, so there is nothing to strip.
+      ...(llmParams.messages
+        ? {
+            messages: {
+              ...llmParams.messages,
+              tail: buildToolCallRepairPrompt(
+                llmParams.messages.tail,
+                repairError,
+                deps.profile,
+                deps.toolTransport,
+                false,
+              ),
+            },
+          }
+        : {}),
       // The grammar-link variant must be repair-shaped too — spreading
       // `llmParams` alone would hand a grammar fallback link the STALE
       // base prompt without the repair notice. It is repair-shaped for
@@ -1956,6 +1986,7 @@ function resolveStepGrammar(
 
 function buildLlmStreamParams(args: {
   promptText: string;
+  promptMessages?: PromptMessages;
   deps: Pick<
     StepDependencies,
     | "toolTransport"
@@ -1990,6 +2021,9 @@ function buildLlmStreamParams(args: {
   });
   return {
     ...base,
+    // The structured prompt rides only on the native path; the seam
+    // forwards it only to a native link (`llm-link-attempt.ts`).
+    ...(args.promptMessages ? { messages: args.promptMessages } : {}),
     // Keep `grammar` populated (not blanked) even on the native path: the
     // provider fallback chain may hand this request to a grammar-only
     // llama-server link, which needs the GBNF. Native (cloud) providers

src/analytics/turn-usage-meter.test.ts

@@ -211,3 +211,35 @@ describe("TurnUsageMeter", () => {
     });
   });
 });
+
+describe("TurnUsageMeter — cached prompt tokens", () => {
+  it("sums cachedTokens across calls and prices them at cacheRead", () => {
+    const meter = new TurnUsageMeter();
+    meter.begin(S);
+    const model: ResolvedModel = {
+      ...pricedModel(3, 15),
+      pricing: { input: 3, output: 15, cacheRead: 0.3 },
+    };
+    meter.record({
+      sessionId: S,
+      usage: { ...usage(1_000_000, 0), cachedTokens: 800_000 },
+      model,
+    });
+    meter.record({
+      sessionId: S,
+      usage: { ...usage(1_000_000, 0), cachedTokens: 200_000 },
+      model,
+    });
+    const snap = meter.snapshot(S);
+    expect(snap.cachedTokens).toBe(1_000_000);
+    // (200k × 3 + 800k × 0.3) + (800k × 3 + 200k × 0.3), per million.
+    expect(snap.costUsd).toBeCloseTo(0.84 + 2.46, 9);
+  });
+
+  it("omits cachedTokens when no call reported the figure", () => {
+    const meter = new TurnUsageMeter();
+    meter.begin(S);
+    meter.record({ sessionId: S, usage: usage(10, 5), model: pricedModel(3, 15) });
+    expect(meter.snapshot(S)).not.toHaveProperty("cachedTokens");
+  });
+});