docs: clarify compression performance tradeoffs

[aryansk]

Fixes #330. Summary: remove stale guaranteed speedup numbers from the README and API documentation, explain that compression can reduce storage while adding decompression work, and direct users to benchmark on their own hardware. Validation: python3 -m py_compile air_llm/airllm/utils.py; git diff --check. The change is documentation-only and does not alter runtime behavior.

[Hotragn]

Good direction — dropping an unconditional speedup number is right, and "benchmark on your own hardware" is honest.

One thing that might make the docs more useful than "it depends": the direction isn't hardware-random, it's predictable from what compression actually does. I dug into this on #330 and there are two concrete costs, both in source:

1. Compression force-disables prefetching — airllm_base.py:172-174:

   if self.compression is not None and self.prefetching:
       print("prefetching is not supported together with compression for now; disabling prefetching.")
       self.prefetching = False

   So you lose the disk/compute overlap entirely.

2. Every layer is dequantized on the GPU, per layer per token — load_layer (utils.py:142) → uncompress_layer_state_dict (utils.py:86-114): a .cuda() upload of the quantized tensor plus a dequant kernel, ~64 layers × every token. 4-bit's dequantize_nf4 + QuantState.from_dict is heavier than 8-bit's dequantize_blockwise, which is why TyrelCB measured 8bit landing between none and 4bit rather than the two compressed modes being similar.

So the rule users can actually apply is: compression trades disk bytes for GPU work — it wins when disk read is your bottleneck, and loses when it isn't. That's why the original "3x" held on 2023-era commodity disks and inverts on a GB10 with fast unified memory. Same for the peak-VRAM surprise in that table (8bit 3.16GB vs none 1.60GB): dequantize materialises the fp16 tensor alongside the quantized source and its quant state.

Might be worth a sentence to that effect so people can predict which side they're on before spending a 62GB split. Entirely your call on wording — just offering the mechanism since I'd already traced it.

(Caveat: I verified the mechanism by reading the code, not by reproducing the benchmark — no GPU here. The magnitudes are TyrelCB's.)

[aryansk]

Thanks for tracing the tradeoff through the source and for the concrete explanation. I added a sentence to the README in 4883a6f covering the current prefetching limitation and per-layer GPU dequantization, while keeping the performance claim qualified and not repeating benchmark magnitudes I could not reproduce without a GPU. git diff --check passes; I appreciate the pointer.

[aryansk]

Thanks for tracing the mechanism so clearly. I’ve incorporated that explanation into the branch: the README now calls out prefetching being disabled, per-layer GPU dequantization, the disk-I/O tradeoff, and benchmarking on target hardware; the API documentation was updated too. Runtime behavior is unchanged.