docs: list Kimi K3 in Supported Models (#316)

[evanclan]

Summary

#316 shipped Kimi K3 support and updated two places in the README — the headline paragraph and a new [2026/07] Updates entry. The Supported Models section further down was not part of that pass, so it still describes the pre-K3 world. This finishes the sweep in the one section a user actually scrolls to when picking a model.

Docs only, no code touched.

Problem

Three things in README.md disagree with what main ships:

1. The family list omits Kimi, even though AirLLMKimiK3 is a dedicated class registered in ARCH_OVERRIDES (air_llm/airllm/auto_model.py:24) and exported from air_llm/airllm/__init__.py:

   **Llama** (2 / 3 / 3.1 / 3.3 / 4) · **Qwen** (...) · **DeepSeek** (V2 / V3 / R1) · **Mistral & Mixtral** · ...
   

2. The VRAM table tops out at DeepSeek-V3 671B / ~12 GB (README.md:288), directly contradicting the headline eight screens above it, which claims 2.8T on under 4GB. A reader who trusts the table concludes 671B is the ceiling.

3. The explanation above the table is now incomplete. It says AirLLM "only ever keeps one layer on the GPU at a time" — but that is exactly what K3 cannot do. Per air_llm/airllm/airllm_kimi_k3.py:14-16, a K3 layer is ~55GB expanded, so experts stream individually. Without that sentence, a 2.8T row sitting below a 671B/12 GB row just reads as a typo.

Relatedly, Same one line of code for all of them — no special setup. is no longer true: K3 needs compressed-tensors for its MXFP4 weights (noted as an optional dep in air_llm/setup.py:40-41) and flash-attn, since its own model code mandates flash attention regardless of what you request.

Solution

• Added **Kimi** (K3) to the family list, after DeepSeek — the other large MoE family.
• Added a Kimi K3 | 2.8T | ~3.7 GB row. The number is the 3.72GB peak generation VRAM Support Kimi K3 (2.8T) — runs on a single card in 3.72GB #316 measured end to end on one RTX 6000 Ada, rounded to match the table's existing precision.
• Extended the "one layer at a time" sentence with the per-expert fallback, so the non-monotonic 2.8T row reads as the point rather than an error.
• Qualified the "no special setup" line with K3's two extra pip installs and the CUDA 12 torch constraint, pointing back to the [2026/07] entry for the full reasoning rather than restating it.

Wording, VRAM figure, and the flash-attn/CUDA 12 constraint are all taken from #316's own README entry and commit 2decbfd — nothing new is claimed here.

Out of scope

Noticed while reading, deliberately not touched:

• README_ja.md is a translation of the old Anima README, from before the project was renamed to AirLLM. It still opens with # Anima, documents QLoRA 33B training, and links to README_en.md, which does not exist in the repo (404). Fixing it means retranslating the current README, which is a much bigger change than this one and belongs in its own PR.
• README.md:183 has a broken markdown link — Example [python notebook] (https://...) — where the space between ] and ( makes it render as literal text. Unrelated to K3, left alone.
• The Quickstart snippet still says "initialize AirLLMLlama2" while the code below it uses AutoModel.

Happy to open follow-ups for any of these if you want them.

Test plan

• Rendered the diff locally to confirm the table row and the two prose changes render correctly.
• Verified the [2026/07 update](#updates) anchor resolves to the existing ## Updates heading.
• Cross-checked every claim against auto_model.py, airllm_kimi_k3.py, setup.py, and Support Kimi K3 (2.8T) — runs on a single card in 3.72GB #316's description — no numbers invented.
• CI to confirm the rest.

Related

• Support Kimi K3 (2.8T) — runs on a single card in 3.72GB #316 — Kimi K3 support; added the headline and Updates entry this PR completes
• 2decbfd, 75866c4 — the commits that introduced the K3 docs changes

Made with Cursor