LeVJEPA

World models · 2026

The idea

If video is how the world actually arrives - motion, causality, object permanence - why has learning from it been so much more expensive than learning from still images?

Two definitions before the move:

• DINO (Meta, 2021) - self- DI stillation with NO labels. A student network learns to match a teacher (an EMA copy of itself) across two crops of the same image. No labels, no negative pairs.
• SIGReg - a regularizer that prevents embedding collapse by forcing the embeddings toward an isotropic Gaussian, checked with a normality test (Epps-Pulley) on many random 1-D projections instead of in full dimension. Comes from LeJEPA (2025).

A clip has an order of magnitude more tokens than a photo. On top of that cost, V-JEPA and V-JEPA 2 pile on scaffolding whose only job is to stop collapse: an EMA teacher encoder, a stop-gradient, and a capacity-limited predictor. Or they dodge collapse by reconstructing masked pixels, which needs a decoder.

LeVJEPA throws the scaffolding out. One shared encoder plus a small projector looks at a global crop and several local crops of the same 16-frame clip. An invariance loss pulls those embeddings together; SIGReg keeps the cloud from flattening to a point. No target network, no predictor, no stop-gradient. One loss weight (λ = 0.02), never tuned. Open code, weights, and checkpoints.

The “P” in JEPA is the predictor. Here it is gone.

What SIGReg is doing

It is tempting to think of this as “summarizing” the clip, the way an RNN summarizes a sequence. That misses it. Nothing is being compressed into a running total. Two views of the same clip are being pushed together, and a separate term is keeping the cloud of embeddings from flattening.

The Gaussian term is the collapse-prevention. Under the LeJEPA argument, an isotropic Gaussian is the embedding distribution that minimizes worst-case downstream probing risk; a collapsed solution (zero variance on some axis) is as far from that as you can get. Checking the full high-dimensional cloud is expensive, so SIGReg samples many random 1-D directions and runs a normality test (Epps-Pulley) on each projection. If every slice looks like a 1-D Gaussian, the cloud is Gaussian. That is a teaching picture of a statistical regularizer - not a protractor, and the “provable” part lives in the LeJEPA assumptions, not in a demo.

What dropped with the scaffolding

Because both branches share one encoder, two cheap tricks become legal:

• Drop 95% of the patch tokens , uniformly at random. ImageNet frozen accuracy rises as more tokens are thrown away: 33.9% when every token is kept, 47.6% at a 95% drop. The drop is regularization, not just a speed hack. (Motion-centric Something-Something-v2 does get worse at high drop unless you train longer - the paper is honest about that residual.)
• Block-causal attention : bidirectional inside a frame, causal across frames. Frame t only sees frames ≤ t . Bidirectional V-JEPA cannot do this without a second temporal model after pretraining. Causal encodings are what an autoregressive world model actually needs - extend the video one frame at a time, without re-encoding the past.

It is not a robot

Threads called this a new world model. The paper is more precise, and the visual should follow the paper. LeVJEPA is evaluated as a frozen representation learner: an attentive probe on ImageNet-1K, Kinetics-400, and Something-Something-v2. No robot. The action-conditioned planner in this lineage is V-JEPA 2-AC - a separate predictor, fitted after pretraining, on robot data. LeVJEPA puts the temporal constraint in the encoder and stops there.

Against DINO on the same-video compute-matched framing: it loses on static appearance (ImageNet) and wins on motion (Something-Something-v2). The interesting reading is not “beats DINO” - it is temporal and motion understanding per compute dollar.

Verdict

What it fixedVideo pretraining without the teacher-student scaffolding: one shared encoder, one projector, one loss weight. At matched epochs on identical data, 5.6x to 20.8x less compute than V-JEPA 2.

What it still costsA frozen representation learner, not a planner. Evaluated by probing - not on a robot. The action-conditioned world-model stage is a different paper.

The story, animated

One encoder survives, the other is struck out - then the compute savings, and the token-drop that raises accuracy.

The numbers (paper, not the thread)

All of these are from Kuhn et al., LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics (arXiv 2608.27395), under frozen attentive probing. Baselines were retrained on the same 20% K710 subsample unless noted.

• Matched epochs, identical data: 5.6× less total pretraining compute than V-JEPA 2 at ViT-L, 20.8× at ViT-S. ViT-L also beats V-JEPA 2 by 1.9 ImageNet points at that 5.6× saving.
• Matched total FLOPs, ViT-B: +7.6 ImageNet-1K vs the strongest video baseline; highest Kinetics-400; 3.2 points behind the best video baseline on Something-Something-v2 (40.4% vs 43.6%).
• Compute-matched DINOv2 trained on frames of the same videos: DINOv2 wins ImageNet by 3.1 points (appearance); LeVJEPA nearly doubles Something-Something-v2 (30.4% vs 16.9% - motion).
• Same 16 GB GPU: LeVJEPA trains at batch 128 in under 8 GB; a V-JEPA encoder of the same size saturates at batch 28.

Where it sits

Worth having first: LeJEPA · V-JEPA 2

Unlocks: LeWorldModel - same SIGReg family, but a planner from pixels rather than a frozen video encoder.

Source-checked against Kuhn, Maes, Serra, Le Lidec, LeCun, Balestriero, Buettner, arXiv:2608.27395 (2026) - team includes @ylecun, @lukaskuhn77, @lucasmaes_, @quentinlldc, @randall_balestr among others. Thread that flagged it: LeoKharon, 31 Aug 2026. Code / weights: MLO-lab/LeVJEPA.