LoRA
Post-training & alignment · 2021
The idea
Section titled “The idea”Full fine-tuning updates every one of a model’s weights - one full copy per task. LoRA freezes the original weight matrix and learns a much smaller correction: two thin matrices whose product approximates the update, added back in at inference for zero extra latency.
Verdict
Section titled “Verdict”What it fixedFreeze the pretrained weights; train two thin low-rank matrices instead - full fine-tuning quality from a fraction of the trainable parameters.
What it still costsOnly approximates full fine-tuning; picking the right rank and which layers to adapt is still trial and error.
Where it sits
Section titled “Where it sits”Worth having first: Instruction tuning & RLHF
Unlocks: QLoRA