Per-layer activation-weight difficulty migration for int8 (W8A8) linear layers, with an automatic per-layer migration-strength search.
When you quantize a transformer to 8-bit weights and 8-bit activations (W8A8), the weights quantize cleanly but the activations do not. Transformer activations carry outliers: a small number of input channels hold magnitudes tens of times larger than the rest (documented in Dettmers et al. 2022, "LLM.int8()", and Xiao et al. 2022, "SmoothQuant"). Per-tensor activation quantization has to stretch one scale across that whole dynamic range, so the common small-magnitude channels lose most of their precision and the layer output drifts.
SmoothQuant fixes this by migrating magnitude out of the activations and into the weights along each input channel, controlled by a strength alpha. The catch: SmoothQuant uses a single fixed alpha (0.5) for every layer. The right amount of migration actually depends on how severe a given layer's outliers are, so one global value leaves error on the table.
qmigrate implements the difficulty-migration transform in pure numpy and adds a per-layer search that picks the migration strength which directly minimizes each layer's measured output quantization error.
For a linear layer Y = X W^T, migration rescales each input channel j:
X'[:, j] = X[:, j] / s[j]
W'[:, j] = W[:, j] * s[j]
so the float product X' W'^T equals X W^T exactly, while
s[j] = max_i |X[i, j]| ** alpha / max_k |W[k, j]| ** (1 - alpha)
controls how much magnitude moves from activations to weights. qmigrate sweeps alpha on a grid per layer and keeps the value with the lowest int8 output error. Because 0.5 is always on the grid, the search can never do worse than fixed SmoothQuant.
- It turns a hand-set hyperparameter (alpha) into a measured, per-layer quantity at zero training cost. The search is a handful of int8 error evaluations per layer.
- The improvement is guaranteed monotone against the standard baseline: per layer, qmigrate is at least as good as fixed alpha 0.5, because that point is searched.
- It exposes the structure that a global alpha hides. On the layer stack here the optimal alpha ranges from 0.60 to 0.80, tracking outlier severity. No single value is right for all of them.
FigJam board: https://www.figma.com/board/PMAOJuHs3LskfeptWFOm4c
The diagram shows the flow: per-channel activation and weight statistics feed the migration scale, migration splits difficulty between per-tensor activation quant and per-channel weight quant, the quantized output is compared to the float output, and the per-layer alpha grid search closes the loop before writing metrics and the chart.
graph TD
A["Layer X, W"] --> B["Channel stats"]
B --> C["Migration scale s(alpha)"]
C --> D["X' = X / s, W' = W * s"]
D --> E["Per-tensor int8 activations"]
D --> F["Per-channel int8 weights"]
E --> G["Quantized output"]
F --> G
A --> H["Float output"]
G --> I["Relative error"]
H --> I
I --> J["Grid search alpha per layer"]
J --> C
J --> K["Best alpha, min error"]
The metric is mean relative output error (Frobenius) of the int8 layer output against the float output, averaged over a stack of eight synthetic linear layers whose per-channel outlier severity spans the range real transformers show. Every number is measured by actually quantizing the tensors and comparing to the float output, not asserted.
- Baseline, naive W8A8 with no migration: 7.47 percent mean error. It blows up on the severe-outlier layers, reaching 11.31 percent on the lm_head.
- SmoothQuant with fixed alpha 0.5: 1.84 percent mean error.
- qmigrate with per-layer alpha search: 1.59 percent mean error.
qmigrate cuts mean output error 78.7 percent versus naive W8A8 and 13.5 percent versus fixed alpha 0.5, at no training cost. The right panel of the chart shows why: the naive baseline climbs with outlier severity, while both migration methods stay flat and qmigrate sits at or below fixed alpha on every layer.
The activation distributions are synthetic but structurally faithful (a few heavy channels over a Gaussian bulk); the quantization error itself is real int8 rounding error. Optimal alpha per layer, from the run:
| layer | naive | fixed 0.5 | qmigrate | alpha* |
|---|---|---|---|---|
| attn.q_proj | 4.12% | 1.87% | 1.49% | 0.80 |
| attn.k_proj | 5.33% | 2.08% | 1.71% | 0.70 |
| attn.v_proj | 7.07% | 2.10% | 1.82% | 0.70 |
| attn.o_proj | 8.41% | 2.14% | 1.92% | 0.60 |
| mlp.gate | 7.61% | 1.92% | 1.71% | 0.60 |
| mlp.up | 7.20% | 1.43% | 1.32% | 0.60 |
| mlp.down | 8.75% | 1.53% | 1.35% | 0.60 |
| lm_head | 11.31% | 1.62% | 1.39% | 0.60 |
pip install -r requirements.txt
# run the experiment: prints per-layer results, writes docs/metrics.json
python demo.py
# render docs/before_after.png from the metrics
python make_chart.py
# invariant tests (float preservation, monotone vs fixed alpha, etc.)
python tests/test_qmigrate.py
To use the migration on your own layer, pass activation and weight matrices:
import numpy as np
from qmigrate import quantized_error, search_alpha, default_grid
X = np.random.randn(256, 512) # [tokens, in_features]
W = np.random.randn(512, 512) # [out_features, in_features]
err_naive = quantized_error(X, W, alpha=None) # no migration
err_fixed = quantized_error(X, W, alpha=0.5) # SmoothQuant
alpha, err_auto, curve = search_alpha(X, W, default_grid(11))
print(alpha, err_naive, err_fixed, err_auto)attn.q_proj naive= 4.12% fixed0.5= 1.87% auto= 1.49% alpha*=0.80
attn.k_proj naive= 5.33% fixed0.5= 2.08% auto= 1.71% alpha*=0.70
attn.v_proj naive= 7.07% fixed0.5= 2.10% auto= 1.82% alpha*=0.70
attn.o_proj naive= 8.41% fixed0.5= 2.14% auto= 1.92% alpha*=0.60
mlp.gate naive= 7.61% fixed0.5= 1.92% auto= 1.71% alpha*=0.60
mlp.up naive= 7.20% fixed0.5= 1.43% auto= 1.32% alpha*=0.60
mlp.down naive= 8.75% fixed0.5= 1.53% auto= 1.35% alpha*=0.60
lm_head naive= 11.31% fixed0.5= 1.62% auto= 1.39% alpha*=0.60
mean error naive 7.47%
mean error smoothquant 1.84%
mean error qmigrate 1.59%
qmigrate cuts error 78.7% vs naive, 13.5% vs fixed alpha 0.5
- Xiao et al., SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models, 2022.
- Dettmers et al., LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale, 2022.