CL-6204: Ollama adapter overrides via the inference adapter manifest - #236
Merged
Merged
Conversation
Red tests for a custom Interchange inference adapter package: typed context-window/max-output-token/reasoning-effort override resolution (per-model beats general default, unset stays unset), and the built request body actually carrying configured num_ctx/max-tokens/ reasoning_effort while staying byte-equivalent to the built-in OpenAI adapter's output when unconfigured.
Interchange's loadAdapterRegistry manifest is the supported extension point for per-provider request overrides (buildRequest returns the body as a string, so a custom adapter has full control over it) — no vendoring needed. This wraps the built-in OpenAI adapter unchanged (SSE parsing, retry/pacing headers, message marshaling) and adds one buildRequest pass applying operator-configured overrides: num_ctx (Ollama's openai-compatible endpoint has no OpenAI-shaped field for context window; it only takes it through the options passthrough object), max output tokens, and reasoning_effort, general or per model.
Documents the SIDECAR_ADAPTER_MANIFEST entry that activates the adapter for the ollama provider key and the quirks (InferenceSource overrides) shape: default/perModel numCtx, maxOutputTokens, and reasoningEffort.
readSidecarConfig hardcoded SIDECAR_ADAPTER_MANIFEST to "[]" for every workflow-process child and passed loadAdapterRegistry([]) for its own adapters — the child-side parsing/wiring (parseAdapterManifest, workflow-substrate-factory) already reached loadAdapterRegistry, but no operator-facing env var ever fed it anything but empty. These tests cover the new optional SIDECAR_ADAPTER_MANIFEST env var: a valid manifest parses through, a malformed one or one missing a required field fails boot naming the variable.
Closes the gap between the child-side manifest wiring (already correct: parseAdapterManifest -> loadAdapterRegistry in workflow-substrate-factory) and the boot edge, which never read an env var and always threaded an empty manifest to both its own loadAdapterRegistry call and every child's substrate env. The manifest is now parsed and validated once at boot (failing loud on a malformed value, matching every other substrate-config knob) and threaded verbatim to both.
Names the contract and shows activating @corbits/ollama-adapter for the ollama provider key.
…pter # Conflicts: # bun.lock
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The owner asked for maximum flexibility on context window, max output tokens, and reasoning options for local inference — without vendoring
@intx/inference(OpenAIQuirksis closed, no body passthrough). Interchange already ships the supported extension point for exactly this:loadAdapterRegistry(manifest)merges an operator-configured custom adapter over a built-in sharing its provider key, andbuildRequestreturns the body as a string, so a custom adapter has total control over it.packages/ollama-adapter— new@corbits/ollama-adapterpackage.createOllamaAdapterwraps the built-in OpenAI adapter (SSE parsing, retry/pacing headers, message marshaling all reused unchanged) and adds onebuildRequestpass applying typed, arktype-validated overrides —numCtx(Ollama's openai-compatible endpoint has no OpenAI-shaped context-window field; it only takesnum_ctxthrough the endpoint'soptionspassthrough object, same as native/api/chat),maxOutputTokens, andreasoningEffort("low"|"medium"|"high", the field Ollama already recognizes forgpt-ossmodels on this endpoint) — general (default) or per model (perModel), per-model winning field-by-field.apps/sidecar/src/workflow-substrate-factory) already parsedSIDECAR_ADAPTER_MANIFESTand fed it toloadAdapterRegistrycorrectly. But the boot edge (apps/sidecar/src/index.ts) hardcodedloadAdapterRegistry([])for its own registry andSIDECAR_ADAPTER_MANIFEST: JSON.stringify([])in every child's substrate env — there was no env var an operator could actually set. AddedSIDECAR_ADAPTER_MANIFESTas an optional, arktype-validated env var inapps/sidecar/src/config.ts, threaded to both call sites..env.example.What actually works end-to-end
All three override classes verified via the built request body (package tests):
numCtx->options.num_ctx,maxOutputTokens-> overrides whichever max-tokens field the built-in adapter set,reasoningEffort->reasoning_effort. Per-model beats general default; no override configured leaves the body byte-equivalent to the built-in adapter's.Checks run (memory-constrained machine — see note below)
Per the owner's active memory constraint,
bun run check/ repo-root lint / repo-root typecheck were not run. Ran only, one at a time:bun install(once, cache-backed, ~12s) to resolve the new packagepackages/ollama-adapter:bun test(14 pass),bun run typecheck(clean)apps/sidecar:bun run typecheck(clean),bun test(full suite, 164 pass — confirms the config.ts/index.ts wiring change didn't regress anything)Not run: repo-root lint/prettier,
check:packages,check:licenses,check:tool-package-pins— CI should run these.check:tool-package-pinsshouldn't apply here: nothing pins@corbits/ollama-adaptervia a{ name, version }tool-package literal (it's an npm-style workspace dependency, not a tool package).Test plan
bun run check(lint/typecheck/test across the workspace)check:packages(packed-consumption check for the new package's exports)check:licenses(LGPL-2.1-or-later + canonical LICENSE text)