teploy-gateway

The OSS AI gateway: one project key, every model, tracked — on hardware you own. Provider API keys live only in the gateway's config; projects hold scoped, revocable gateway tokens.

ANTHROPIC_API_KEY=... OPENAI_API_KEY=... teploy-gateway serve
teploy-gateway key create --project fylun

Then a project needs exactly two values:

import { createAnthropic } from "@neutron-build/ai/anthropic";
const anthropic = createAnthropic({
  baseURL: process.env.AI_GATEWAY_URL,
  apiKey: process.env.AI_GATEWAY_KEY,     // the minted project token
});
const model = anthropic("anthropic/claude-sonnet-5"); // or openai/…, groq/…

No SDK changes, either wire: the gateway serves /v1/chat/completions (OpenAI shape — also Groq, DeepSeek, xAI, anything compatible) and /v1/messages (Anthropic shape), accepts both auth header styles, rewrites provider/model to the upstream name, swaps in the real key, and streams responses through byte-for-byte.

What you get per project

• All configured models by provider/model id (bare names use the endpoint's default provider).
• Spend tracking — requests and token counts aggregated per project/provider/model (extracted from both wires, streaming included), persisted in the data dir, queryable via teploy-gateway usage --project x. A Nucleus-backed sink slots in behind the same interface.
• Failover — same-wire fallback chains ("fallbacks": {"openai/gpt-4o": ["groq/llama-3.3-70b"]}) walked on 429/5xx/unreachable; cross-wire chains are rejected at config load.
• Kill-switch — teploy-gateway key revoke <id> deadens a token instantly. Only SHA-256 hashes are stored; tokens are shown once.

Config

--config config.json (optional — built-in providers work with just env keys: ANTHROPIC_API_KEY, OPENAI_API_KEY, GROQ_API_KEY, DEEPSEEK_API_KEY, XAI_API_KEY):

{
  "providers": {
    "local": { "baseURL": "http://127.0.0.1:11434", "apiKey": "x", "wire": "openai" }
  },
  "fallbacks": { "openai/gpt-4o": ["groq/llama-3.3-70b"] }
}

Admin API (/admin/v1/keys, /admin/v1/usage) is guarded by a token minted 0600 to the data dir at first start. Don't bind the gateway publicly unless you put auth-terminating ingress in front of it.

Teploy Cloud's managed tier is this same binary operated for you, with zero-BYOK credits billed to your existing account.