GQA
The efficiency lineage · 2023
The idea
Section titled “The idea”Grouped-Query Attention: the compromise position between one K/V and one per head.
PaperGQA: Training Generalized Multi-Query Transformer Models from Multi-Head CheckpointsarXiv:2305.13245 →
The paper’s own headline result: an existing multi-head checkpoint can be uptrained into GQA with just 5% of its original pretraining compute - which is why it shipped everywhere, fast, instead of requiring anyone to train from scratch.
Verdict
Section titled “Verdict”What it fixedGroups of heads share K/V - most of MQA's saving, most of the quality back.
What it still costsThe cache still grows with context.
The story, animated
Section titled “The story, animated”The dial between one-each and one-shared - every setting’s bill computed live.
Where it sits
Section titled “Where it sits”Worth having first: MQA
Unlocks: MLA