World models.
Predict, then act.

Predict how an environment may respond.

In robotics and agent research, a world model is a learned representation or simulator of how an environment changes. Given what it has observed—and, in an action-conditioned system, a candidate action—it estimates a possible next state. A planner can compare those estimates before choosing what to do.

The word is used across different research traditions, so it does not name one architecture. Some systems predict future pixels or video frames. Others predict a compact latent representation: features that preserve information needed for a task without reconstructing every visible detail.

The loop is the point. A predicted image that looks convincing is not enough if the system cannot use it to choose an action and then check the result.

World models do not all draw the future.

A video-generating model may render a visible future. That can make the simulation easy for people to inspect, but visual detail alone does not establish that an object will move, collide, or stay put as the real environment would.

A latent world model predicts in an internal feature space instead. It may track task-relevant changes—such as the relation between a gripper and an object—without producing a polished frame for every imagined step. The representation is useful only to the extent that it keeps the information a planner needs.

Prediction is becoming part of the action loop.

Google DeepMind’s Genie 3 generates interactive environments from text and responds to navigation actions. Researchers have used generated worlds to observe how an agent behaves in a simulated setting. DeepMind also lists current limits: the available action space is constrained, accurate real-world locations are not guaranteed, and continuous interactions last minutes rather than hours.

Meta’s V-JEPA 2 illustrates a different approach. Its action-conditioned V-JEPA 2-AC predicts future representations rather than relying on full-resolution video generation at each planning step. The paper reports robot planning after post-training on fewer than 62 hours of unlabeled robot video, then zero-shot tests on Franka arms in two labs without task-specific training or new data from those test environments. That is a result in a specific research setup, not a general guarantee for arbitrary robots or tasks.

A preprint posted in October 2026, Video Prediction Policy 2, reports that tested world-action models can make incorrect motion predictions in open-ended manipulation tasks. Its authors report an 18.5 percentage-point success-rate improvement over their strongest baseline on their real-world zero-shot ALOHA tasks. These are the authors’ results on a defined setup; they do not establish performance across all robots or environments.

Together, these examples show why “the model can generate a world” and “the agent can act reliably in the world” are separate claims. One concerns simulation; the other requires action-level evidence.

Realism is not the same as a reliable forecast.

Model predictions are estimates. A system that acts in the physical world still needs independent limits on speed, force, reach, access, and stop conditions. When an observation conflicts with the forecast, it should be able to pause and reassess instead of forcing the planned sequence through.

Score the prediction and the decision it supports.

1. Name the state that matters. For a robot, that might include object position, gripper state, contact, or whether a goal was reached—not only visual similarity.
2. Test multiple candidate actions. Check whether the predicted differences between actions match the observed differences after execution.
3. Measure error over time. Test one-step predictions and longer rollouts; report how errors accumulate and when the model stops being useful.
4. Use held-out conditions. Vary object placement, appearance, camera position, lighting, and the starting state. Keep a clear boundary between training and test cases.
5. Evaluate in a closed loop. Measure task success, recovery after a bad prediction, and how often a person or safety controller must intervene.
6. Check transfer separately. If a simulator is involved, compare simulated results with controlled physical trials before relying on it for real actions.

For agent systems, include the full policy and runtime in the test: the model, action interface, controller, safety limits, and recovery behavior all affect the result. A world model should not be evaluated as though it acts alone.

Connect prediction to the rest of the system.

Start with how an agent turns a model into action ↗, then explore how physical AI connects models to real-world movement ↗, how VLA models produce robot actions ↗, how AI systems process video over time ↗, and how to evaluate an agent in its full workflow ↗. For a broader map, visit the AI field guide ↗.