runpod-coder

Run a Hugging Face coding model on a RunPod GPU and use it from an opencode session. Pick a model, pick a GPU, start it as a warm vLLM pod (serverless endpoints are also supported), then wire it into opencode with one command.

The default flow is a pod: always warm, no cold-start waits, natural start/stop. It is billed hourly while RUNNING; down stops billing (disk persists, up resumes it).

Install

pip install -e .        # provides the `runpod-coder` command

Needs a RunPod API key: export RUNPOD_API_KEY=... or runpodctl config --apiKey 'rpa_...' (stored in ~/.runpod/config.toml). The key never lives in this repo.

Usage

runpod-coder models                     # curated catalog with quant variants
runpod-coder models --search deepseek   # search Hugging Face for anything
runpod-coder gpus --min-vram 48         # RunPod GPU types with >= 48 GB

runpod-coder up --model qwen3.8-27b     # auto-picks a GPU; fp8 by default
runpod-coder up --model Qwen/Qwen3.8-27B-FP8 --gpu "NVIDIA RTX A6000"
runpod-coder up --model <repo> --target serverless    # scale-to-zero instead

runpod-coder status                     # pod/endpoint state + vLLM readiness
runpod-coder down                       # stop pod (resumable); no hourly charge
runpod-coder down --terminate           # terminate pod / delete endpoint

runpod-coder use                        # write the opencode provider config
runpod-coder use --agent build          # point a specific agent at the model
runpod-coder remove                     # strip the provider from opencode

After use, restart opencode and pick the model via the model switcher or the configured default. The provider id is RunPodCoder, model id RunPodCoder/<repo>.

What use writes

use merges provider.RunPodCoder into ~/.config/opencode/opencode.json (--scope project targets ./opencode.json) using the @ai-sdk/openai-compatible provider, and sets the default model:

{
  "provider": {
    "RunPodCoder": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Qwen3.8-27B (fp8)",
      "options": {
        "baseURL": "http://<pod-ip>:8000/v1",
        "apiKey": "<pod-scoped vLLM key>",
        "timeout": 600000,
        "headerTimeout": 300000
      },
      "models": {
        "Qwen/Qwen3.8-27B-FP8": {
          "limit": { "context": 262144, "output": 131072 },
          "reasoning": true,
          "interleaved": "reasoning_content",
          "tool_call": true
        }
      }
    }
  },
  "model": "RunPodCoder/Qwen/Qwen3.8-27B-FP8"
}

For serverless deployments apiKey is {env:RUNPOD_API_KEY} instead.

Catalog

slug	model	quants	weights	VRAM
qwen3.8-27b	Qwen3.8-27B	fp8 (default), bf16	28 / 55 GB	48 / 80 GB
qwen3-32b	Qwen3-32B	bf16	65 GB	80 GB

Any HF repo works via up --model <repo-id> (generic spec: 32k context, auto-pick a GPU with --min-vram/--gpu).

Notes

• Pod state (pod id, vLLM API key, base URL) lives in ~/.runpod-coder/state.json — home dir, never the repo.
• The pod's vLLM port is public but protected by a generated API key.
• The first up downloads the weights (~30 GB) before serving; wait for the readiness probe to finish.