Continuous-state reasoning experiments with Gemma and MLX. A learned bridge feeds hidden states back into the transformer, replacing part of a text reasoning trace. The model then generates the remaining reasoning and answer.
Author and maintainer: Larkooo.
Gemma discussion: Gemma 4 hybrid continuous-state reasoning.
Experimental continuous-state baseline, not an explicit mathematically structured reasoning system. The September 2026 feasibility checkpoint records two simple base-inference outputs and one finite optimizer update with checkpoint reload equality. It establishes runtime feasibility only, not improved reasoning, historical-adapter reproduction, or a completed controlled experiment.
The historical Gemma 4 E2B experiment uses two latent positions followed by a fixed text transition. On 600 held-out diagnostic arithmetic and link-traversal questions:
| Method | Accuracy | Median answer latency |
|---|---|---|
| Text reasoning | 598/600 (99.67%) | 1.031 s |
| Hybrid latent/text reasoning | 583/600 (97.17%) | 0.922 s |
The hybrid reduced median completed-answer latency by 10.6%, with a 2.5 percentage-point accuracy loss. Two repeats per method and question produced 2400 timed requests, with randomized, counterbalanced order on an Apple M5 with 32 GB memory. The speed ratio was 1.118, with a paired-question bootstrap 95% interval of [1.069, 1.152]. The recipe was fixed before this test. Mean completed-answer latency was 1.195 s versus 0.994 s, a 16.8% reduction.
The earlier 100-question validation pilot scored 96/100 versus 99/100 with 17.6% lower latency. A separately trained control using the same shortened text targets without latent positions scored 66/100 on that validation sample. Results use one training seed and two procedural task families. The harder OOD evaluation is pending; broader reasoning gains and lower total computation have not been established.
Independent test report · Experiment results · All reports
Python 3.12+ and Apple Silicon are required.
uv venv --python 3.12
uv pip install -e '.[dev]'
.venv/bin/python -m pytest -qThe reproduction guide covers the pinned model, training, combined accuracy/latency benchmark, controls, and public benchmark data.
prompt → K continuous feedback steps → remaining reasoning → answer
Latent steps use continuous embeddings directly, without token sampling or a vocabulary projection. LoRA adapts the backbone. Gemma 4 latent positions retain the continuous per-layer embedding branch and use zero token-table contribution.
Current performance results use Gemma 4 E2B. The backend also supports Gemma 3 text experiments. See the method and evaluation protocol for cache handling, training, measurement scope, and limitations.
src/latent_gemma/— model wrapper, training, inference, and evaluation.scripts/— model/data preparation, paired benchmarks, and result utilities.tests/— model semantics, gradients, decoding, scoring, and data checks.reports/— frozen predictions, timing traces, source snapshots, and analyses.
For Gemma integration, see the continuous-input proposal. The approach builds on Coconut.
Code is available under the MIT license. Model weights are obtained separately and retain their upstream terms.