Deqio

Decisions in. Probabilities out.

Deqio runs fast typed AI decision models behind one consistent local API and a lightweight browser UI. It supports multiple decision engines and lets you switch between installed models without changing your application code.

Public decision endpoints stay the same regardless of the active model:

POST /v1/noul
POST /v1/choice
POST /v1/shared

Benchmark snapshot

Apple Silicon macOS · 16.0 GiB unified memory
deqio-basic-150 · 150 requests · 250 scored decisions · 50 Noul / 50 Choice / 50 Shared
Results captured on 2026-09-26. Higher accuracy is better; lower latency is better.

Highlights from this run

• Highest overall accuracy: Decider 4B / MPS — 95.3% case accuracy, 97.2% decision accuracy.
• Lowest median latency: Laya Multilingual / MLX — 13.2 ms.
• Perfect Noul + Choice accuracy: Kev 4B / MLX — 100.0% / 100.0%.

Model	Backend	Accuracy	Decision accuracy	Median	P95
Decider 4B	MPS	95.3%	97.2%	435.7 ms	1,358.6 ms
Kev 4B	MLX	92.7%	95.2%	317.6 ms	507.9 ms
Decider 2B	MPS	90.7%	94.0%	185.3 ms	524.5 ms
SemIf / Qwen3.5 4B	MLX	90.7%	93.6%	552.3 ms	997.0 ms
Decider 0.8B	MPS	84.0%	88.8%	292.2 ms	376.8 ms
Kev 0.8B	MLX	77.3%	84.4%	60.2 ms	96.9 ms
Von	MPS	64.0%	72.8%	61.0 ms	104.5 ms
Laya Typed Decisions 421M	MLX	62.0%	72.4%	34.2 ms	74.6 ms
Laya English 421M	MLX	54.7%	66.8%	34.5 ms	74.9 ms
Laya Multilingual 322M	MLX	44.7%	56.4%	13.2 ms	27.1 ms

Sorted by overall case accuracy; ties are ordered by decision accuracy. A Shared case passes only when every expected decision in that request is correct, while decision accuracy scores each decision independently.

Per-type case accuracy

Model	Backend	Noul	Choice	Shared
Decider 4B	MPS	100.0%	98.0%	88.0%
Kev 4B	MLX	100.0%	100.0%	78.0%
Decider 2B	MPS	94.0%	94.0%	84.0%
SemIf / Qwen3.5 4B	MLX	96.0%	96.0%	80.0%
Decider 0.8B	MPS	88.0%	94.0%	70.0%
Kev 0.8B	MLX	86.0%	86.0%	60.0%
Von	MPS	86.0%	58.0%	48.0%
Laya Typed Decisions 421M	MLX	74.0%	78.0%	34.0%
Laya English 421M	MLX	68.0%	66.0%	30.0%
Laya Multilingual 322M	MLX	62.0%	48.0%	24.0%

These are machine-specific results from one local run, not universal model rankings. Results can change with hardware, runtime versions, model revisions, and benchmark changes.

Supported models

Model	MLX	MPS	CUDA
SemIf / Qwen3.5 4B	✓	✓	✓
Kev 0.8B	✓	✓	✓
Kev 4B	✓	✓	✓
Kev 9B	✓	✓	✓
Kev 27B	—	—	✓
JevK5 4B	—	—	✓
JevK5 9B	—	—	✓
Open-Jev 2B	—	—	✓
Open-Jev 9B	—	—	✓
Open-Jev 27B v1.1	—	—	✓
CLM 8B	—	—	✓
Decider 0.8B	—	✓	✓
Decider 2B	—	✓	✓
Decider 4B	—	✓	✓
Laya English 421M	✓	✓	✓
Laya Multilingual 322M	✓	✓	✓
Laya Typed Decisions 421M	✓	✓	✓
Von	—	✓	✓
Bespoke Nimble 9B v2	✓	—	✓

1. Installation

The recommended installation is from PyPI with uv tool. Deqio keeps models, isolated runtimes, configuration, and benchmark results in the directory where you use it; the Python package itself stays small.

macOS — Apple Silicon

Deqio supports both MLX and MPS on Apple Silicon.

1. Install uv if you do not already have it:

curl -LsSf https://astral.sh/uv/install.sh | sh

2. Install Deqio from PyPI:

uv tool install deqio

3. Create a workspace and enter it:

mkdir -p deqio-work
cd deqio-work

4. Choose and install a backend/model:

deqio models setup

On a Mac, the installer offers:

• mlx — recommended for models with native MLX support
• mps — PyTorch on Apple Silicon, required by models such as Decider

models setup detects host memory and hides profiles that do not meet the catalogued minimum for the selected backend. It installs the isolated runtime, downloads/prepares the model weights, starts the model once, and requires a real typed-decision readiness probe to pass before the profile is registered as installed.

On first setup Deqio creates editable config.json, models.json, and benchmarks/basic.json files in this workspace. Model runtimes and weights are kept outside the PyPI package.

5. Start Deqio:

deqio serve

After setup succeeds, normal serve and benchmark starts use the local Hugging Face cache in offline mode. They do not intentionally contact Hugging Face or download missing weights; if cached artifacts are missing, reinstall/update the profile instead.

---

Linux — NVIDIA GPU

Deqio uses the CUDA backend on Linux.

1. Make sure the NVIDIA driver is working:

nvidia-smi

2. Install uv and Deqio:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv tool install deqio

3. Create a workspace:

mkdir -p deqio-work
cd deqio-work

4. Choose and install a CUDA-compatible model:

deqio models setup

5. Start Deqio:

deqio serve

---

Windows — NVIDIA GPU

Deqio uses the CUDA backend on Windows.

1. Make sure the NVIDIA driver is working in PowerShell:

nvidia-smi

2. Install uv and Deqio:

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
uv tool install deqio

3. Create a workspace:

New-Item -ItemType Directory -Force deqio-work
Set-Location deqio-work

4. Choose and install a CUDA-compatible model:

deqio models setup

5. Start Deqio:

deqio serve

---

Install from source

For development, clone the repository instead of installing the PyPI tool:

git clone https://github.com/ILuce/deqio.git
cd deqio
uv sync --extra dev
uv run deqio models setup
uv run deqio serve

When running from a source checkout, use uv run deqio ...; when installed from PyPI with uv tool install deqio, use deqio ... directly.

Model lifecycle

Show models that are compatible with the current host:

deqio models list --compatible

For machine-readable host/memory information (useful for agents):

deqio models list --compatible --json

List locally installed profiles:

deqio models installed

Choose another installed profile:

deqio models use

Delete an installed profile interactively:

deqio models delete

Or explicitly:

deqio models delete kev-4b --backend mlx --yes

Deqio removes workspace-owned model/runtime artifacts when they are no longer shared by another installed profile. The global Hugging Face cache is kept by default because other applications may use it. Add --purge-cache if you explicitly want Deqio to remove the profile's primary unshared Hub repository from that global cache.

Show the current selection:

deqio status

Memory values in the catalog are conservative compatibility guardrails, not exact peak-memory guarantees. models install ... --force can bypass the guard for advanced users.

Model startup has two separate readiness deadlines: the sidecar process must open its local API first (default 300 seconds), then the model must answer a typed readiness probe (new-workspace default 1800 seconds). Override them when needed with DEQIO_SIDECAR_PROCESS_READY_SECONDS and DEQIO_SIDECAR_STARTUP_SECONDS. DEQIO_HF_OFFLINE_RUNTIME=false can temporarily disable normal offline runtime mode for diagnostics; installation/update already enables network access automatically.

Stop a running deqio serve process before deleting model/runtime files from the same workspace.

2. Using Deqio from the UI

Start the server:

deqio serve

Open:

http://127.0.0.1:8787

The browser redirects to the Deqio UI.

Select a model

At the top of the UI, choose one of the models already installed on the machine and click Activate. Deqio unloads the previous runtime, loads the selected model, runs a warmup, and keeps the public API on the same address.

Noul — yes/no decision

Use Noul when the result should be a binary decision.

Fill in:

• State — the context
• Question — the yes/no question

Click Send to see the selected answer, probabilities, latency, token count, and cache information.

Choice — choose between options

Use Choice when the model should select one option from a list.

Fill in:

• State
• Question
• one or more options with an ID and description

Add or remove options directly in the UI and click Send.

Shared — several decisions over one state

Use Shared when several decisions should be evaluated against the same context.

Enter the shared State, add the decisions and their options, then send them together. This is preferable to several separate requests when an engine can reuse the common state efficiently.

Useful links

UI:        http://127.0.0.1:8787/ui
API docs:  http://127.0.0.1:8787/docs
Health:    http://127.0.0.1:8787/health

Stop the server with Ctrl+C.

3. Using Deqio as an API

Start Deqio once:

deqio serve

Base URL:

http://127.0.0.1:8787

The UI is not involved when your application calls the API directly.

Noul

curl -s \
  -X POST http://127.0.0.1:8787/v1/noul \
  -H 'Content-Type: application/json' \
  -d '{
    "state": "The patch changed source code and no tests have been run yet.",
    "question": "Should tests be run before considering the task complete?"
  }'

Typical response:

{
  "decision": "yes",
  "probabilities": {
    "yes": 0.97,
    "no": 0.03
  }
}

Choice

curl -s \
  -X POST http://127.0.0.1:8787/v1/choice \
  -H 'Content-Type: application/json' \
  -d '{
    "state": "A customer was charged twice for the same subscription renewal.",
    "question": "Which team should handle this request?",
    "options": [
      {"id": "billing", "description": "Payments, refunds, invoices, and duplicate charges."},
      {"id": "access", "description": "Login and account access problems."},
      {"id": "technical", "description": "Product defects and service failures."}
    ]
  }'

Shared

curl -s \
  -X POST http://127.0.0.1:8787/v1/shared \
  -H 'Content-Type: application/json' \
  -d '{
    "state": "A patch changed an authentication module. Unit tests passed, but integration tests have not been run.",
    "decisions": [
      {
        "id": "validation",
        "question": "What should happen next?",
        "options": [
          {"id": "run_integration_tests", "description": "Run integration tests before proceeding."},
          {"id": "finish", "description": "Finish without more validation."}
        ]
      },
      {
        "id": "release",
        "question": "Is the change ready to release?",
        "options": [
          {"id": "yes", "description": "The change is ready to release."},
          {"id": "no", "description": "More validation is required."}
        ]
      }
    ]
  }'

Check and switch the active model through the API

List installed model profiles:

curl -s http://127.0.0.1:8787/v1/models/installed

Activate an already installed profile:

curl -s \
  -X POST http://127.0.0.1:8787/v1/models/activate \
  -H 'Content-Type: application/json' \
  -d '{
    "model_id": "decider-0.8b",
    "backend": "mps"
  }'

The decision API URL does not change when the active model changes.

Benchmarking installed models

Deqio includes an editable starter suite in benchmarks/basic.json: 50 Noul, 50 Choice, and 50 Shared requests. Stop deqio serve before benchmarking so the benchmark can load each model with the machine's memory available.

Run it interactively and choose all installed models or selected profiles:

deqio benchmark

Run every installed model compatible with the current machine:

deqio benchmark --all

Or select profiles explicitly:

deqio benchmark \
  --model decider-0.8b:mps \
  --model laya-typed-decisions:mlx

The console shows live PASS/FAIL and latency for every request. Full results.jsonl and summary.json files are written under .deqio/benchmarks/<timestamp>/. Add or edit cases in benchmarks/basic.json as the benchmark grows.

Nimble note: Bespoke Nimble 9B v2 follows the upstream MLX/CUDA workflow. Its first installation downloads the adapter and pinned Qwen3.5-9B base, then prepares merged local weights, so it needs substantially more disk/RAM than the smaller models.

CUDA-only model note: JevK5, Open-Jev and CLM use their upstream Linux/NVIDIA serving paths. CLM starts a local vLLM pooling encoder plus clm-serve inside one Deqio-managed sidecar. Kev 27B requires an 80 GB-class NVIDIA GPU.

For the complete request and response schemas, open:

http://127.0.0.1:8787/docs

License

MIT. See LICENSE.