MLA

The efficiency lineage · 2024

The idea

Multi-head Latent Attention (DeepSeek). The end of the shrink branch: compress what you keep, but you still keep something per token.

The trick: compress the full hidden vector into one small latent, then expand it back out per head. Only the compact latent and a slim rotated slice (marked teal) are what get cached across steps — not a full key and value per head. That is where the cache-size savings onMLA's own page actually come from.

Verdict

What it fixedK and V compressed into one small latent - a large cache cut at matched quality.

What it still costsStill grows with N; the slope is smaller, the wall remains.

The story, animated

Sixty-four numbers per token, one latent - watch it fold, then unfold back exact.

Watch

Episode 1, beat 9 - the payoff · 146s, from the film series.

Where it sits

Worth having first: GQA

Unlocks: The hybrid