KDA
The efficiency lineage · 2025
The idea
Section titled “The idea”Kimi Delta Attention: a one-symbol edit of the equation the lineage built - alpha becomes Diag(alpha) - and suddenly memory management is per-channel.
Verdict
Section titled “Verdict”What it fixedA gate per key channel - Diag(alpha). Each row of the state gets its own lifetime.
What it still costsExact retrieval is still not guaranteed - the originals are compressed away.
The story, animated
Section titled “The story, animated”One symbol splits: alpha becomes Diag(alpha). Each row of the box gets its own dial.
Where it sits
Section titled “Where it sits”Worth having first: Gated DeltaNet
Unlocks: The hybrid