[LLM] add system-one endpoint for TypeSafe AI jev - #32796
Open
pmilliotte wants to merge 3 commits into
Open
pmilliotte wants to merge 3 commits into
pmilliotte wants to merge 3 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a third endpoint kind,
system_one, alongsidestreamandbatch, with one endpoint on it:TypeSafe AI's
jev-latest. It answers named yes/no, choice and score questions about a piece ofstate in a single round trip, so it fits neither
StreamEndpoint(no token stream) norBatchEndpoint(no job lifecycle) —SystemOneEndpointextendsClientdirectly and wraps@typesafe-ai/sdk.Two knock-on changes come from adding a
Lab, both forced by the typechecker:PassthroughLabis defined byExclude, so a new lab opts into provider passthrough bydefault.
typesafe_aijoins the exclusion list: system one is single-shot, so there is noassistant turn to replay verbatim.
LAB_TO_PROVIDER_IDis a totalRecord<Lab, ModelProviderIdType>andtypesafe_aihas nolegacy provider id. The key is narrowed and reads go through
labToProviderId, which assertsinstead of persisting a stand-in id on a usage row.
Nothing calls the endpoint yet, so the dust layer (
lib/llms/system_one/), a registry, and anySTATIC_MODEL_IDS/ pricing / tier entries are deliberately out of scope —jevis not a chatmodel and does not belong in the picker.
contextSize,maxOutputTokensandtokenPricingcarryTODOs naming the missing source rather than invented numbers.
Tests
Nine unit tests covering the answer mapping, usage normalization, request shape, and the error
table (401, 500, refused connection, client-side rejection). They drive the SDK through an injected
fetch, so no network. Two of them pin the new security contract: the API key comes from thepassed
Credentialsand never from an ambientTYPESAFE_API_KEY— the SDK silently falls back tothat env var when
apiKeyisundefined, which would serve a BYOK workspace on a Dust key.Risk
Low. The only change to an existing path is
labToProviderId, whose assert fires on a lab thatcannot reach a stream or batch transition.
Deploy Plan
mainonfrontTYPESAFE_AI_API_KEYis read from the environment but is not set anywhere yet; nothing calls theendpoint, so it can be provisioned before the first caller lands