Add support for Talkie (talkie-1930-13b) - #1492
Merged
Merged
Conversation
2 tasks
zcbenz
approved these changes
Sep 3, 2026
This was referenced Sep 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the Talkie architecture — talkie-lm's 13B models trained entirely on pre-1931 text (model card). llama.cpp already supports it as
LLM_ARCH_TALKIE. The official repos are raw PyTorch checkpoints, but the transformers port atlewtun/talkie-1930-13b-it-hfconverts directly withmlx_lm.convertand ships a normaltokenizer.json+ chat template, so this is just a model file and a test entry. No tokenizer changes, no new dependencies.Mostly a standard 40-layer MHA decoder (40 heads x 128, SwiGLU, vocab 65540), with a few quirks:
mx.fast.ropeand negated freqs, same trick asnanochat.py; checked against the PyTorch reference (max diff ~5e-7). llama.cpp gets the same effect by folding the sign flip into the q/k weights at conversion time.lm_headis a bare tensor scaled by a learned scalar (what the GGUFs export aslogit_scale)The offset goes straight from the cache into
mx.fast.rope(int or per-sequence array) and there's no custommake_cache, sobatch_generateand the server's continuous batching work as-is.Re: the two earlier talkie PRs (#1220, #1231) — both build RoPE from a precomputed cos/sin table indexed by a scalar
cache[0].offsetand pinmake_cachetoKVCache, so they don't work withBatchKVCache. #1220 also targets the original checkpoint layout rather than the -hf one, and adds a tiktoken-based tokenizer adapter plus atiktokendependency that turns out to be unnecessary: the -hf repo'stokenizer.jsontokenizes identically to llama.cpp's GGUF tokenizer on everything I threw at it, special tokens included.Testing:
lewtun/talkie-1930-13b-it-hfto bf16 and q8 (group size 64); both load strictly with no missing/unexpected keystalkieentry totest_all_models; fulltests/test_models.pypasses and pre-commit is cleanTo try it:
mlx_lm.convert --hf-path lewtun/talkie-1930-13b-it-hf --mlx-path talkie-q8 \ -q --q-bits 8 --q-group-size 64 mlx_lm.generate --model talkie-q8 --temp 0 \ --prompt "What is the proper way to address a letter to a duchess?"which answers: "To the Duchess of —, Belgrave Square, London, W."