fix: support generic OpenAI-compatible local model servers (Ollama) in the guided install - #92
Merged
Conversation
prepareLmStudioModels aborts the local install when the model server answers /v1 but 404s on LM Studio's native management API (/api/v1/models). Generic OpenAI-compatible servers such as Ollama load models on demand, so a 404 there now skips the pre-load step instead of failing stage 2/6. Model availability is still validated against /v1/models earlier in resolveModelSetup. Verified against a real Ollama 0.20.2 server on WSL2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
Temporarily closing and reopening to retrigger CI after correcting the SHA-pinned Actions allowlist; no code state is being changed. |
5 tasks
This was referenced Aug 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
create-answer-engineaborts the local (lmstudio-provider) install at stage 2/6 when the model server is a generic OpenAI-compatible server rather than LM Studio itself.prepareLmStudioModelsprobes LM Studio's native management API (GET <origin>/api/v1/models,POST <origin>/api/v1/models/load); servers like Ollama serve/v1/*(chat, embeddings, models) but 404 on the native endpoints, so the install dies withCould not inspect LM Studio model state: HTTP 404.This PR treats a 404 from the native inspect endpoint as "no management API — models load on demand" and skips the pre-load step. Model availability is still validated against
/v1/modelsearlier inresolveModelSetup, and a reachable LM Studio keeps the exact current behavior (inspect + explicit load).Why
Found while doing the v1.1.1 guided install (INSTALL_AGENT.md) on Windows 11 WSL2 with Ollama as the local model runtime. Everything else in the local route already works with any OpenAI-compatible server:
--lm-studio-url http://127.0.0.1:11434/v1+--models chat=...,embedding=...validates fine against Ollama's/v1/modelscontainerBaseUrlcorrectly rewrites localhost →host.docker.internal(verified reachable from containers)OpenAiCompatibleProviderchat + embeddings calls work against Ollama (verified from inside a container: chat completion withqwen3:30b-a3b-instruct-2507-q4_K_M, embeddings withnomic-embed-textat 768 dims)This one function was the only hard blocker.
Testing
packages/create: 149/149 tests pass,type-checkandbuildcleanprepareLmStudioModels+listLmStudioModelsagainst a real Ollama 0.20.2 server: lists models, gracefully skips pre-loadExtra setup notes from the same install attempt (not part of this diff)
Installer release assetsworkflow has never been dispatched. Inputs that line up for it:tag=v1.1.1,source_commit=a346a8c6a7833596c088ccdebadd69e3608c386b,runtime_image=ghcr.io/the-answerai/answer-engine@sha256:418c93dac3753e0850f5ba6627581f888acdd65f3b91b655bd898cee56f10701. Worth confirming the built bootstrap matches the checksum pinned in the tagged INSTALL_AGENT.md (ea7e4632…) before publishing.ollama pull nomic-embed-text, 768 dims) — Ollama typically ships with chat models only.MODEL_RUNTIMEcheck probes thelmsCLI only, so it warns "LM Studio was not detected" even when an OpenAI-compatible server is up (non-blocking; probing the configured/v1/modelsURL could fix it in a follow-up)./v1/modelsexactly — for Ollama that includes the tag (e.g.nomic-embed-text:latest).🤖 Generated with Claude Code
Closes #93