The drone and the world model
World models · 2026 · What is a world model
If you can predict what your actions will do, you can plan instead of practising. This page builds the maths of world models step by step, from landing a drone on exact physics to learning a model of a world that fights back, with a lab for each idea.
World models · worked through
The Drone and the World Model
NASA can hit an asteroid on the first try. A robot needs thousands of attempts to learn to fold a towel. What does NASA have that the robot doesn't? A world model it can trust. If you can predict what your actions will do, you can plan in your head instead of learning by crashing.
Each equation is built up in plain words, and you can try each idea in a lab. Wherever a shortcut version of an equation is commonly written, it's marked COMMON SLIP and the correct form is shown.
Before we start
Seven symbols carry the whole story
Each one keeps its colour everywhere on this page. Tap any coloured symbol in an equation to see what it means in plain words.
1 · The problem
How much do you learn from one more try?
Two numbers decide how far AI still has to go. Intelligence per watt is how much ability you get for the energy you spend. Intelligence per sample is how much smarter one more training example makes you. Humans are very good at the second: a new card game takes a few rounds. Today's models need thousands of examples for the same jump.
One influential definition says intelligence is the ability to acquire new skills efficiently, not a large store of skills you already have. The ARC-AGI puzzles are built to measure exactly that. Most people solve them with a little thought. Frontier models still struggle.
Now push the idea to its limit. The most sample-efficient learner needs zero tries. That sounds impossible, but engineers do it all the time. Newton's laws predict a spacecraft's path so well that a trajectory can be planned years ahead. Nobody crashes a million rockets first.
Careful: zero samples, with conditions
A perfect world model removes the need to sample the dynamics. You still need a goal, enough compute to plan, and feedback while you fly. Newtonian mechanics is excellent, not exact. Real missions measure where they are and re-plan, which is exactly the method in section 2.
Imagined practice works, but treat the numbers with care
A much-quoted 1967 study had one group practise basketball layups for an hour and improve about 24%, while a blindfolded group only imagined the shots and improved about 23%. The 1967 papers on mental practice are real. These exact percentages, though, circulate in popular writing and could not be traced to the original article. The finding that holds up is that mental practice helps, but less than real practice. For a world model the takeaway is the same: you can improve a policy using rollouts you only imagine.
Careful: the brain timeline
Some neuroscientists argue that the neocortex grew in order to model the world. A popular version dates a “great cortical expansion” to 10 million years ago, which doesn't match the fossil record. Hominin brain growth spans about 7 million years, and most of it is recent: brain size roughly tripled in the last 3 million years. That it grew for world modelling is a hypothesis, not established fact.
2 · When the world model is perfect
Land a drone without a single practice run
A drone has to land on a pad. Gravity pulls it down, and the only thing it controls is its thrust. Everything about the drone right now fits in one list of numbers, the state. Its position along each axis comes first, then its velocity along each axis:
tap a term
The world model answers one question: if I'm in this state and fire this thrust, where am I one tick later? For a drone we don't have to learn the answer, because it's high-school physics. Newton's second law gives the acceleration, and two lines of kinematics turn that into the next state:
tap a term · simplification: we treat the drone as a point; a real quadcopter also has an orientation, which makes the state 12 numbers
Now turn “land softly” into a score you can minimise. Add up, over the next few seconds, how far you are from the pad, how fast you're still moving, and how much thrust you burn. Then find the thrust sequence with the lowest total that the physics allows. This method is called model predictive control (MPC), and versions of it land SpaceX boosters. Quick sketches of it often drop three pieces; here is the complete problem:
tap a term
Lab 1 · Fly it
2-D: x across, z up, so the state has 4 numbers instead of 6state st
thrust ut
‖u‖ vs limit
cost of the current plan
Dashed blue: what the world model predicts the plan will do, its imagination. Solid: what actually happened. Every 0.1 s the controller solves the problem above from the measured state, keeps only the first thrust, and plans again. Hit Gust of wind mid-flight: the world model didn't predict it, so prediction and reality split, and re-planning pulls the drone back. Mass 1 kg, so hovering needs 9.81 N. Push umax near that and watch it struggle. To keep it simple we leave out the ground constraint z ≥ 0; a real lander includes it (it is linear, so the problem stays convex). The solver is projected gradient descent, run live in your browser.
Why a computer can solve this exactly
The problem is convex, a bowl with one bottom, so any downhill method finds the true best plan, and tools like CVXPY solve it in milliseconds. Bowl-shaped cost terms alone aren't enough; convexity needs three things together:
- every cost term is a convex bowl (squared distances, squared thrust);
- the world model is linear in position, velocity and thrust, so its constraints are flat planes. This is the condition people most often forget. Give the drone nonlinear aerodynamics and the bowl is gone;
- the thrust limit ‖u‖ ≤ umax is a round, convex region.
Real rocket landing is harder: an engine can't throttle below a minimum thrust, and that constraint isn't convex. Researchers showed how to relax it into an equivalent convex problem (lossless convexification), and that is what made Falcon 9 landings computable on board.
A common claim is that a non-differentiable problem leaves you with nothing you can do. That isn't true. Subgradient methods, sampling-based planners (random shooting, CEM, MPPI) and dynamic programming all work without gradients, and reinforcement learning is one such family. This problem is solved by an algorithm, not a formula: typically interior-point methods or gradient descent on the Lagrangian. Stochastic methods like Robbins–Monro are built for noisy gradients and aren't needed for this deterministic problem.
3 · When the world fights back
Add a killer drone that wants you to crash
Now a killer drone shares the sky and tries to ram you. The next state depends on its move too, and you can't see inside its controller:
The physics is still Newton's. What changed is that part of the future is now unknown. That's the real reason the tidy convex solve stops working. (Lack of gradients is a symptom, and robust or game-theoretic control can handle some of these cases.) The general fix is to learn the pieces from experience:
From rewards we build the value of a state: how good the future looks from here if you keep following policy π. Think of a rocket launch: inside the safe corridor the value is high; drifting over houses, the value is terrible, even before anything has gone wrong yet. A common shorthand is Vπ(st) = Σt γt rt, which is missing the most important symbol:
Lab 2 · Why value needs an average
each step: 70% the drone gets 1 closer, 30% a gust pushes it 1 backThe drone starts 3 steps from the pad. Every step costs −1 (fuel) and landing pays +20, so the landing step nets +19. One sampled future gives one total, Σγkr: the bar heights show how much each step's reward still counts. Totals vary a lot from run to run. Their average (solid green line) settles on the exact value V(s) (dashed line), which is computed directly from the Bellman equation. That average is the E[·] in the formula.
4 · Two models, or one
What I do, times what the world does back
The question model-based RL asks: from here, what's the chance I take action a and the world ends up in s′? The chain rule of probability splits it into the two pieces we already have:
Lab 3 · A robot, a cup, a table edge
every number below is computed from the slidersLearning the world from video, then adding the controls
A video model learns T(st+1 | st): how frames follow frames, without knowing who did what. Strictly, it learns the bottom row of Lab 3, the world's behaviour averaged over whatever the people in the video happened to do. That's why video alone can't tell “what happens” from “what I can make happen”. The modern trick is to pretrain on huge amounts of unlabeled video, then add action conditioning with a small amount of data recorded with actions. That gives the model embodiment: it learns what its own moves change.
Careful: not every Dreamer works this way
The Dreamer series is often credited as a whole with adding action conditioning later. Only Dreamer 4 works that way. V1–V3 learn their world model from the agent's own action-labelled play from the start. What gets added is a small input adapter for actions, not a whole new model.
World Models. A compressed model of car racing and Doom; a controller trained entirely inside the model's dream still works in the real game.
PlaNet. A learned latent world model with MPC on top. It is sometimes mislabelled as the first Dreamer.
Dreamer: learns a policy and value inside imagined rollouts instead of planning at every step.
DreamerV3: one setting for 150+ tasks; first to collect diamonds in Minecraft from scratch, with no human data.
Dreamer 4: world model learned mostly from unlabeled video, with actions added from a small labelled slice; first to get Minecraft diamonds purely from offline data.
DreamZero, NVIDIA: “World Action Models are Zero-shot Policies.” A 14B video diffusion model (Wan2.1 backbone) that predicts future frames and actions together, at about 7 Hz on a real robot.
Why merge the two models
Keeping πθ and Tψ separate causes two problems. Training them on different data makes their training dynamics fragile. And planning ahead means calling them in turn, policy then world, policy then world, step after step, which is slow. A World Action Model (written Pθ(at, st+1 | st), with one set of weights) outputs the action and the predicted next state in a single call. That halves the calls. It isn't cheap, though: each call still generates video, and DreamZero manages about 7 actions per second.
5 · Sizing up a problem
STAR: four questions to ask before you try RL
Lab 4 · Chess, counted properly
|A| = 20 at the start, about 35 on averageEach of 8 pawns can step 1 or 2 squares (16 moves). Each knight has 2 jumps (4 moves).
|S|: how many chess positions? (log scale)
A tempting guess is 3264, from “32 pieces, 64 squares”. A fair rough bound asks what each square can hold: empty or one of 12 piece types, so 1364. Most of those positions can't occur in a real game; careful counts put the legal ones at about 4.8 × 1044.
|A| is what really explodes: look ahead d moves
Chess's T: the rules are perfectly deterministic. The uncertainty comes only from the opponent. Its R is a +1 / 0 / −1 at the end, or a material count as a shortcut. With an action space this small, a search engine can explore the tree, and that is why chess and Go fell first. Driving and robotics have huge action spaces, and that is where world models have to carry the load.
Based on the first 20 minutes of the YC Decoded video “World Models: An intuitive introduction” (July 2026). The equations were checked against the video's whiteboard and corrected where needed.
Sources. World Models (2018) · PlaNet (2018) · Dreamer (2019) · DreamerV3 (2023) · Dreamer 4 (2025) · DreamZero / World Action Models (2026) · Convex optimisation for trajectory generation (survey) · Chess branching factor · Hominin brain evolution (eLife 2019)
Next: When search runs out