multi-token-prediction

llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% throughput).

PyTorch implementations of modern open-source LLM architectures (Llama, Qwen, DeepSeek, Gemma, GPT-OSS, Kimi, and more) — written from scratch for readability and learning, based on Sebastian Raschka's LLM Architecture Gallery.

DMax: Aggressive Parallel Decoding for dLLMs

Qwen4-Exp (Qwen3.8-Flash-Next) SWA + MTP + PLE-on-iswa — bounded deep-context decode w/ long-range recall, TBQ4 4.25 bpv KV, DSpark + RotorQuant. Main (master) = Qwen4-Exp build; the Qwen35 SWA hybrid lives on the qwen35 branch. RTX 4090.

Serving Qwen3.8-27B-FP8 on a single DGX Spark (GB10): 7.88 to 58.5 tok/s single-stream from decode strategy alone, weights untouched. Speculative decoding and prefix caching benchmarked, plus DFlash 2 — the only Qwen3.8-27B build that can serve it under vLLM.

Curated collection of research on the limitations of next-token prediction and methods that go beyond it.

Run Hermes Agent + Claude Code locally on llama.cpp — zero API costs. A 4h / 7M-token session that would have cost $94 on Claude Opus 4.7

Deploy Qwen3.6-35B-A3B (Q4_K_XL) + MTP speculative decoding on a single NVIDIA L4 24GB — GCP g2-standard-8 — via the official llama.cpp Docker image. Decode-optimized to ~91–99 tok/s (min ~91 chat, max ~99 math), lossless (full GPU residency + ECC-off).

Faithful from-scratch reimplementation of DeepSeek-V3 (MLA + aux-loss-free MoE + MTP + speculative decoding), ~412M params, Chinchilla-optimal 8.4B-token training on a single A100 80GB.

Preregistered, DOI-archived llama.cpp study: Qwen3.8-27B on one RTX 3090 (sm_86). draft-mtp at --spec-draft-n-max 2 is +59.8% [+57.0, +62.8] server-reported decode over 25 purposive prompts; draft-dflash (PR #27342) +51.9%, not separated from it. Decode energy -37.1%, uncalibrated. 23-25 of 25 prompts diverge from serial greedy at 1600 tokens.

From-scratch PyTorch: frontier LLM techniques as of 2026-Q1 — the Muon optimizer and Multi-Token Prediction, plus a base BPE tokenizer. Self-contained, self-checking modules.

统一 C6 对称性 AI 基础设施工具包 - 路由+MTP+量化+六角注意力+纠错+流水线 | TaiChi Matrix M1-M6 Infrastructure

MTP draft-head research harness for Qwen3.5-9B. Verdict: function cuts (pruning, low-rank, distillation) pay a 10-19pt off-distribution tax fidelity cuts do not — the zero-training trimmed-vocab NVFP4 recipe wins at 1.8-2.7x end-to-end in memra. Negative-result ledgers included.

Some benchmarks on running speculative decoding with Qwen-3.6-35B-A3B with EAGLE3, DFlash and MTP.

A controlled 180-run study of DeepSeek-inspired MLA, sparse MoE routing, V3-style load balancing, and multi-token prediction under constrained compute.

Fork of DeepSeek-V3 with MTP speculative-decoding improvements, a cache-aware mask fix, CPU test suite, and docs cleanup.

Benchmarks Multi-Token Prediction in llama.cpp against standard autoregressive baselines. Measures tokens per second, perplexity delta, and speculative acceptance rate across model sizes.

Recover the native MTP predictor missing from the 8-bit MLX Qwen3.8-27B-Uncensored package, build a BF16 sidecar, and reproduce a 15.59 → 48.75 tok/s controlled M4 Max result with MTPLX.

六头深度调度多令牌预测引擎 - 太极矩阵 M2 模块 | Six-Head Depth Scheduled MTP Engine

Multi-Token Prediction benchmarks for Gemma 4 on Apple Silicon — LiteRT-LM, transformers, and llama.cpp at batch=1 on a MacBook M4 Pro. ~2× speedup reproducible in one specific runtime.