Physical AI.
When models meet matter.

Physical AI is a system in a closed loop.

Here, “physical AI” means an AI-enabled system that senses a physical environment, selects or predicts an action, and observes what happens after that action. The phrase is used for a range of approaches; it does not specify one model architecture or promise that a system can operate without people.

A robot is more than its policy or language model. The working system can include cameras and force sensors, software that estimates the current state, a model or policy that chooses a response, a controller that turns the response into movement, motors that carry it out, and safeguards that bound or stop the motion.

A robot can follow a fixed sequence without using AI. An AI model can also be one component inside a larger robot controller. The important question is how the whole system senses, acts, checks, and handles a result it did not expect.

A VLA model connects an instruction to a robot action.

Vision-language-action (VLA) models combine visual observations and language instructions with action outputs. Google DeepMind’s RT-2 study co-fine-tuned vision-language models on web data and robot trajectories, representing robot actions as tokens the model could produce. Its experiments showed that this approach could transfer some learned concepts to robot tasks that were not directly represented in its robot demonstrations.

That result demonstrates a research direction, not a general-purpose robot ready for every room. The surrounding controller still needs to check the action’s timing, position, force, and operating limits. A model that proposes “pick up the cup” does not by itself establish where the cup is, whether the gripper can reach it, or whether the action completed.

VLA models are one approach within physical AI. A robot may instead use a task-specific policy, a conventional planner, a learned perception component, or a combination. For the action representations, training, and evaluation in more depth, read the dedicated guide to VLA models ↗. For the related idea of predicting how an environment may change before acting, read the guide to AI world models ↗.

Shared experience helps, but hardware still matters.

Collecting robot experience takes physical trials: a team has to operate a machine and record what it senses and does. The BAKU researchers describe this as a central cost in robot learning, which is one reason teams study how to reuse demonstrations and train across tasks.

Robot-learning data is tied to physical setups: the camera angle, gripper, control frequency, workspace, and way a demonstration was collected. Open X-Embodiment assembled 60 robot datasets from 34 research labs. Its collection covers more than one million real-robot trajectories across 22 embodiments, and the researchers report positive transfer between some of the platforms they tested.

This is evidence that a diverse training set can help a policy carry skills across some robot types. It does not show that a single policy will work on any machine without adaptation. A new robot can move differently, sense differently, or encounter objects and layouts missing from its training data.

Every claim about transfer should name the robot, task, environment, and test conditions. “Works across embodiments” is a summary of specific experiments, not a substitute for checking the hardware a team intends to use.

A convincing simulation is a place to test—not the final proof.

Simulation helps teams explore many scenes and actions without physically running each one. The BEHAVIOR-1K project built a benchmark around 1,000 everyday activities in richly specified simulated environments. Its authors found that the long-horizon tasks and complex manipulation in the benchmark remained challenging for then-current robot-learning approaches, and included an initial study of transferring a solution from a simulated apartment to a physical mobile manipulator.

The transfer step matters because small differences can change the outcome: a sensor reading shifts, an object slips, a surface adds friction, or a movement takes longer than expected. A policy can succeed in a simulator and still need tuning, safeguards, or a different workflow on the real robot.

Use simulation to widen coverage and debug safely. Use controlled tests on the target hardware to evaluate whether the system performs the intended task under real operating conditions.

Measure recovery and safe stopping, as well as success.

A useful evaluation gives the robot a known starting state, a precise goal, and a way to verify the final physical result. Test ordinary cases alongside shifted object positions, occlusion, missing items, interrupted actions, and situations where the safest outcome is to stop.

1. Task completion. Did the object, machine, or workspace reach the required state?
2. Robustness. What changes when objects, camera views, lighting, or starting positions vary?
3. Recovery. Does the system detect a failed grasp or unexpected movement, then recover or return control?
4. Operational cost. Record time, retries, operator interventions, and any setup-specific tuning.
5. Safety boundary. Check that speed, force, workspace, and stop behavior are enforced by the control system.

A 2026 SO-101 benchmark preprint evaluates several policies across four real-world manipulation tasks and analyzes failures as well as task scores. Its reported results vary by task, with execution instability and recovery behavior among the issues examined. That is a useful model for evaluation: look inside the failures, and keep conclusions tied to the tested setup.

For software agents, evaluation follows actions through tools and application state. For physical systems, it must also verify movement and its consequences. Continue with the guide to evaluating AI agents ↗ and how an agent chooses its next action ↗.

Connect perception, prediction, and action.

For the visual component, read how computer vision turns images into signals ↗. For the prediction loop, explore how world models estimate possible futures ↗. Then use the AI field guide ↗ to follow more connections across modern AI systems.