Skip to content

CL-6477: fix embeddinggemma escaping the embedding-model filter - #209

Merged
TheGreatAxios merged 3 commits into
mainfrom
cl-6477-model-capability
Aug 21, 2026
Merged

TheGreatAxios merged 3 commits into
mainfrom
cl-6477-model-capability

Conversation

@TheGreatAxios

Copy link
Copy Markdown
Contributor

Summary

  • EMBEDDING_MODEL_NAME_PATTERN in packages/hub-client/src/model-capability.ts required a trailing delimiter after every family it matched, including embed(ding)?. Google's embeddinggemma:300m has none between "embedding" and "gemma", so it survived the filter, then won the alphabetical default-model tiebreak — every Myra-created agent on a box with this model pulled answered every chat turn with "embeddinggemma:300m" does not support chat.
  • Fixed by no longer requiring a trailing delimiter for embed(ding)? specifically (the short abbreviations minilm/bge/gte/e5/arctic-embed keep the stricter, delimiter-bound match — they're prone to false positives as bare substrings). Capability data, when a deployment carries any, already takes priority over this name check in isCompletionCapable — this pattern is only ever the belt-and-braces fallback for the uncataloged case (an unprobed local Ollama pull).
  • Separately: added gpt-oss:20b as the new curated Ollama default in packages/hub-client/src/catalog-seed-data.ts — an overnight run found it correct, tool-obedient, and 12-25x faster than the prior default qwen3.8:27b (8s vs 100-216s for the same task). Kept in its own commit so it can be reverted independently.

What I verified

  • New tests in packages/hub-client/src/model-capability.test.ts cover: embeddinggemma:300m is never selected as a chat default even with no delimiter after "embedding"; the other embedding names (nomic-embed-text, all-minilm, bge-m3, qwen3-embedding) still get filtered (no regression); a chat-capable model is selected when both kinds of candidate are present; and probed capability data (when present) still wins over the name fallback.
  • bun test green across packages/hub-client, packages/chat, packages/inference-settings, packages/onboarding, packages/inference-catalog, packages/provider-pricing (1046 pass, 0 fail) — updated one existing onboarding test (complete-credential.test.ts) that hard-coded the old curated-default name.
  • tsc --noEmit clean for packages/hub-client and packages/onboarding.
  • bun run lint from the repo root: 0 errors (12 pre-existing warnings in unrelated files, not introduced by this change).

What I did not verify

  • Did not boot a real stack or an actual Ollama instance against this change — verified via unit tests only, per the no-stack-boot instruction for this lane.
  • Did not run the full monorepo bun run check (typecheck across every workspace) — scoped to the affected packages plus repo-root lint, given the timebox.
  • Found, but deliberately left out of scope: OLLAMA_CAPABILITY_MAP in packages/hub-client/src/credential-test.ts has no entry for Ollama's own "embedding" capability (only completion/tools/vision are translated), so a genuinely-probed embedding-only Ollama model whose name doesn't match the name-filter at all would still collapse to an empty capability list — indistinguishable from "never probed" — and fall through to the (now-fixed) name heuristic. Closing that gap for real would need a new signal that Interchange's own Capability enum (@intx/types, never to be modified here) has no room for, likely a schema/DB-level change — bigger than this hotfix's scope. Flagging for a follow-up ticket rather than expanding this PR.

Not merging

Per instructions for this lane, not merging — a peer reviewer should merge after checking the above.

CL-6477: embeddinggemma:300m has no delimiter between "embedding" and
the "gemma" family name, so the current EMBEDDING_MODEL_NAME_PATTERN
(which requires a trailing delimiter) never catches it. It then wins
the alphabetical default-model tiebreak and answers every chat turn
with "does not support chat".
…imiter

The embedding-model name filter required a delimiter after every
family it recognized, including embed(ding)?. That's correct for the
short abbreviations (minilm, bge, gte, e5, arctic-embed), which are
prone to false positives as bare substrings, but embeddinggemma:300m
has none between "embedding" and "gemma" and slipped through.

Capability data (when a deployment carries any) still takes priority
in isCompletionCapable; this pattern is only the belt-and-braces
fallback for the uncataloged case.
An overnight run across the candidate Ollama models found gpt-oss:20b
correct and tool-obedient, and 12-25x faster than the prior curated
default qwen3.8:27b (8s vs 100-216s for the same task). Leads the
curated list now; qwen3.8:27b and qwen3.5:9b-mlx stay as fallbacks.

Kept as its own commit so it can be reverted independently of the
embedding-model filter fix.
@TheGreatAxios
TheGreatAxios merged commit 3a2fcc1 into main Aug 21, 2026
0 of 2 checks passed
@TheGreatAxios
TheGreatAxios deleted the cl-6477-model-capability branch August 25, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant