Robot World Models

Robots, like humans, benefit from imagining the outcome of an action before they take it. In unfamiliar and unstructured environments, this ability depends on world models, which are internal simulators that allow a robot to anticipate how the world will respond to its actions. Existing physics engines and video prediction systems rely on extensive manual design, and struggle to generate reliable, action-consistent predictions outside narrowly defined scenarios. Moreover, many current approaches frame world modeling as passive forecasting, predicting future observations without capturing how a robot’s actions causally shape its environment. This motivates the development of controllable world models that accept a robot’s native actions and support learning through imagination for reinforcement learning, planning, and control. Recent research on action-conditioned world models aim to jointly model future evolution and the causal effects of actions, enabling candidate policies to be rolled out and refined within the model itself. This allows for grounding high-level objectives into executable sensorimotor rollouts while retaining physical fidelity.

This workshop brings together researchers at the intersection of perception, control, and learning to advance world models that generalize across tasks, environments, and embodiments while remaining grounded in real robot dynamics. By integrating ideas from computer vision, multimodal representation learning, and dynamics modeling, including semantic abstractions and hierarchical structures across temporal scales. By providing a focused forum to compare emerging approaches and evaluation practices across manipulation, navigation, and long-horizon decision-making, the workshop aims to clarify open challenges and shape a shared research agenda for deployment of world-model-based learning in real-world robotics.