Embodied AI and world models

Current physical-AI work combines learned perception, latent dynamics, language-conditioned planning, simulation, and fast control. Google DeepMind describes a layered Gemini Robotics 2 system; MIT reports future-state-conditioned planning in VLASH; NVIDIA discusses world action models; Hugging Face maps the expanding simulation substrate. These are not one architecture or one independent benchmark. Together they motivate a systems hypothesis: long-horizon physical competence needs models that predict how action changes state, plus control loops that remain stable when prediction is wrong.

The unresolved bottlenecks are old and concrete: partial observability, contact dynamics, latency, uncertainty, causal prediction under novel actions, sim-to-real transfer, safe exploration, and independent physical validation.

A state-space model separates hidden state x_t, control u_t, and observation z_t:

x_{t+1} = f(x_t,u_t) + w_t
z_t = h(x_t) + v_t.

For linear Gaussian dynamics, the Kalman prediction is

x̂^-_t = A x̂_{t-1} + B u_{t-1} and P^-_t = A P_{t-1} Aᵀ + Q.

The observation update uses gain K_t = P^-_t Hᵀ(H P^-_t Hᵀ + R)^{-1} and x̂_t = x̂^-_t + K_t(z_t - Hx̂^-_t).

Worked scalar example: predicted position is 10, predicted variance 4, sensor variance 1, and observation 12. Gain is K=4/(4+1)=0.8; updated estimate is 10 + .8(2)=11.6; updated variance is (1-.8)4=.8. The estimate moves toward the more precise sensor while retaining uncertainty.

A learned world model replaces known f with f_θ. Model-predictive control evaluates candidate action sequences in the model, executes the first action, observes the world, and replans. Closed-loop replanning limits—but does not remove—model error.

1. 1960 — Kalman filtering. Rudolf Kalman made recursive state estimation practical for noisy dynamic systems. ASME (opens in a new tab).
2. 1966 — Shakey. SRI's robot integrated perception, a world model, planning, and action in a simplified environment. SRI history (opens in a new tab).
3. 1982 — Marr's levels of vision. David Marr distinguished computational goal, representation/algorithm, and physical implementation. MIT Press (opens in a new tab).
4. 1986 — Subsumption. Rodney Brooks built behavior layers for real-time robots, challenging centralized representations. IEEE (opens in a new tab).
5. 1990 — Symbol grounding. Stevan Harnad asked how internal symbols acquire meaning rather than merely relate to other symbols. Physica D (opens in a new tab)90087-6).
6. 1995 — Internal motor models. Wolpert, Ghahramani, and Jordan analyzed internal models for sensorimotor integration. Science (opens in a new tab).
7. 1999 — Predictive coding. Rao and Ballard modeled cortical feedback as predictions and feedforward signals as residual errors. Nature Neuroscience (opens in a new tab).
8. 2005 — Stanley. Stanford's autonomous vehicle won the DARPA Grand Challenge with a layered probabilistic robotics system. Journal of Field Robotics (opens in a new tab).
9. 2006 — SLAM tutorials. Durrant-Whyte and Bailey synthesized simultaneous localization and mapping as joint state estimation. IEEE (opens in a new tab).
10. 2007 — Constructing imagined experiences. Demis Hassabis, Dharshan Kumaran, Seralynne Vann, and Eleanor Maguire found that patients with hippocampal amnesia struggled to imagine coherent novel experiences. This is human neuroscience evidence and a conceptual precursor, not an AI world-model experiment. Primary source (opens in a new tab).
11. 2010 — Autonomous helicopter control (cross-link). See the complete autonomous helicopter aerobatics milestone in Machine-learning foundations.

1. 2017: Neuroscience-inspired AI. Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick mapped neuroscience findings to testable AI research directions. Label it as a synthesis and research agenda. Primary source (opens in a new tab).
2. 2018 | World Models. David Ha and Jürgen Schmidhuber trained a visual encoder, recurrent dynamics model, and compact controller, then optimized behavior inside imagined rollouts. The work joined predictive state modeling and model-based control, but its evidence came from simple game environments and exposed the risk that a controller exploits model errors. Primary paper (opens in a new tab).
3. 2019 | PlaNet. Danijar Hafner and collaborators learned latent dynamics from pixels and planned online in the learned state space. The system improved data efficiency on continuous-control benchmarks, while short horizons, simulated environments, and model bias constrained the conclusion. ICML (opens in a new tab).
4. 2020 | MuZero. Julian Schrittwieser and collaborators learned only the reward, policy, and value features needed for tree search. This relaxed the need to reconstruct observations, but the learned latent state was task-specific, expensive to train, and not a general interpretable world model. Nature (opens in a new tab).
5. 2022: MineDojo. Linxi “Jim” Fan and collaborators assembled a large Minecraft environment, knowledge base, task set, and learning resources. Simulator breadth does not prove physical or social-world transfer. NeurIPS (opens in a new tab).
6. 2023: VIMA. Yunfan Jiang, Agrim Gupta, Zichen Zhang, Guanzhi Wang, Li Fei-Fei, Yuke Zhu, Linxi “Jim” Fan, and collaborators studied robot manipulation conditioned by multimodal prompts. Performance remained bounded by the benchmark, embodiment, and simulation assumptions. Primary source (opens in a new tab).
7. 2023 | Voyager (cross-link). See the complete Voyager milestone in Agent planning and cognitive architectures.
8. 2023 | RT-2. Anthony Brohan and collaborators co-fine-tuned vision-language models on robot trajectories and web-scale image-language data, representing actions as tokens. The experiments showed transfer to novel tabletop instructions, but remained bounded to particular robot embodiments, data, latency, and safety conditions. Primary paper (opens in a new tab).
9. 2025 | DreamerV3. Danijar Hafner and collaborators reported one configuration across more than 150 tasks and learning diamonds in Minecraft from pixels and actions. The result advanced generality within reinforcement-learning benchmarks, but much evidence remained simulation-heavy and real-world safety, sensing, and sample costs were unresolved. Nature (opens in a new tab).

Objective: test when a learned world model improves control and when compounding error breaks it.

Environment: CartPole or another standard low-dimensional simulator with fixed version and seeds.

Procedure: collect random-policy transitions (s,a,s'); train a small network to predict Δs; implement random-shooting MPC that samples short action sequences, rolls them through the model, and chooses the sequence with highest predicted return. Compare with a reactive policy and with MPC using the simulator's true dynamics if available. Evaluate horizons 1, 5, 10, and 20; then alter mass or friction at test time.

Metrics: episode return, failure rate, one-step prediction error, multi-step rollout error, compute latency, and performance under changed dynamics.

Expected result: learned MPC should improve over random control at short horizons. Long rollout error and dynamics shift should degrade planning; closed-loop replanning should help.

Success criterion: significant return gain at one prespecified horizon, with an honest error curve showing where the model ceases to be useful.

Falsification: no improvement over a matched reactive baseline, or the gain disappears when planning compute is counted.

Safety note: use simulation. Do not deploy an unverified learned controller on physical hardware.

Claim: layered predictive/planning/control systems outperform a monolithic reactive policy on long-horizon tasks with disturbance.

• Baseline: reactive policy matched for training data.
• Intervention: learned dynamics plus replanning and a low-level controller.
• Primary metric: task completion under prespecified disturbances.
• Secondary metrics: latency, energy proxy, interventions, and unsafe-state visits.
• Uncertainty: at least 100 paired seeds across nominal and shifted dynamics.
• Stopping rule: freeze models and disturbance distribution before final evaluation.
• Disconfirming result: no robust completion gain or layered latency creates more failures.

Google DeepMind (opens in a new tab), NVIDIA Research (opens in a new tab), MIT CSAIL (opens in a new tab), Berkeley AI Research (opens in a new tab), Physical Intelligence (opens in a new tab), World Labs (opens in a new tab), AMI Labs (opens in a new tab), Chelsea Finn (opens in a new tab), Sergey Levine (opens in a new tab), Pieter Abbeel (opens in a new tab), and Fei-Fei Li (opens in a new tab). Current reports to monitor include Gemini Robotics 2 (opens in a new tab), VLASH (opens in a new tab), and simulation for physical AI (opens in a new tab).

• Medium confidence, 12–36 months: physical-AI reports will separate perception, dynamics prediction, planning, and control latency while also publishing integrated end-to-end results. Disconfirming signal: monolithic policies dominate across transfer, safety, and latency.
• Medium confidence, 18–48 months: simulation benchmarks will include more calibrated measures of contact and rare-event fidelity, not only visual realism. Disconfirming signal: real-robot performance shows weak dependence on those measures.

---