Delta rule
The efficiency lineage · 2021
The idea
Section titled “The idea”The repair of blind accumulation: recall, compare, correct. The scrubber below shows the PILE the delta rule exists to fix - plain accumulation, tile on tile.
Verdict
Section titled “Verdict”What it fixedOverwrite by error: recall what the state already says, write only the difference.
What it still costsNothing ever fades - old associations sit there forever.
The story, animated
Section titled “The story, animated”The key reads the slot first. Subtract what’s already there, and write only the correction.
Try it
Section titled “Try it”Where it sits
Section titled “Where it sits”Worth having first: Linear attention
Unlocks: Gated DeltaNet
Episode 2 · The Linear Bet