The hybrid

The efficiency lineage · 2025-26

The idea

Kimi Linear, then K3: the two branches of the lineage finally meet. Neither side won - they were composed.

Verdict

What it fixedAlternate: three KDA layers per one full-attention (MLA) layer. Exact recall where it counts, most of the cache gone.

What it still costsTwo mechanisms to tune, and the full layers are still quadratic.

The story, animated

Three compressed-memory layers, then one exact recall. Watch a token walk that foursome, then see it repeat — and the same 3:1 with a different fourth layer.

The other 3:1

This foursome is a family, not a Moonshot invention. Qwen3-Next shipped a 3:1 Gated-DeltaNet hybrid in September 2025, six weeks before Kimi Linear. Qwen3.8-Flash-Next keeps the same rhythm — three Gated DeltaNet layers, then one Qwen Sparse Attention layer (the sparse-attention branch) — twelve times: 36 compressed-memory layers and 12 sparse-attention layers in a 48-layer stack.

QSA is not MLA. It picks important micro-blocks instead of compressing every token into a latent. Native context is 262,144 tokens, extensible to 1,000,000. The 3:1 is the shared idea; the exact-recall layer is the fork.

Watch

Episode 2, beat 9 - the verdict · 62s, from the film series.

From the full film - assembling K3, macrocycles + MLA · 154s, from the film series.

Where it sits

Worth having first: KDA · MLA

Episode 3 · The Hybrid Era