Skip to content
Agientry

World Models

WaterfallPagination
WORLD MODELS · TIMELINE

World Models: 36 Years from Internal Models to WAMs

1990–2026 · Five lineages · 33 milestones

Early explorationLatent-space eraVideo generation · Physical AI · VLAWAM unification1990200020102018202020222023202420252026Model-based RL · Latent DynamicsVideo-generation World ModelsJEPA Predictive EmbeddingsVLA · World Action ModelsEmbodied / Physical World ModelsWorld Models1990 · SchmidhuberPILCO2011 · Deisenroth & RasmussenWorld Models2018 · Ha & SchmidhuberPlaNet2018 · DeepMindMuZero2019 · DeepMindDreamer2019 · Hafner et al.DreamerV22020 · Hafner et al.TD-MPC2022 · Hansen et al.DreamerV32023 · Hafner et al.I-JEPA2023 · Meta AIGAIA-12023 · WayveUniSim2023 · GoogleSora2024 · OpenAIGenie2024 · DeepMindV-JEPA2024 · Meta AI1X World Model2024 · 1X TechnologiesGenie 22024 · DeepMindCosmos2025 · NVIDIAV-JEPA 22025 · Meta AIGenie 32025 · DeepMindRT-22023 · Google DeepMindOpenVLA2024 · Stanford et al.π02024 · Physical IntelligenceGR00T N12025 · NVIDIAHelix2025 · Figure AIWAMs Survey2026 · Fudan University · Shanghai Innovation Institute · NUSACT2023 · StanfordRDT / H-RDT2024 · Tsinghuaπ0.52025 · Physical IntelligenceOpenVLA-OFT2025 · Stanford · BerkeleyGR00T N1.72026 · NVIDIAXiaomi-Robotics-02026 · XiaomiGigaWorld-Policy-02026 · GigaWorld Team
The x-axis is non-linear (1990–2017 compressed). Hover a node for details.← swipe to explore the full timeline →
WAM PHYLOGENY · 2023–2026

Branching Evolution: From VLA to World Action Models

61 site-collected papers · four lineages · taxonomy per arXiv:2605.12090

2023202420252026timeVLA · p(a|o,l)Cascaded WAMJoint WAMWorld models · p(o′|o,a)RT-2OctoHelixGR00T N1Pointmap-VLACosFly-VLAAC-VLAJoyNexusPOT-VLAFM-VLAMoE-VLAN₀-VTLAτFutureRTCRL²-VLAπR²Ego2RobotAtlasVLAG0.5HAFWAMs SurveyHyWorldVLADeVALeapBot-WARobust-WAMAeroActWorldDiTN₀-TWAMX-NavDPω-0SimWAMDynaWMSiamJEPAOrbis 2When to PlanKoopman DreamerPhysCoReWorld TranslationMusic-JEPAViTacWorldRobot-Factored WMGS-AgentFeelWorldPhyLatentPhyAITinyVLAMolmoActSpatial ForcingX-VLASmolVLAInternVLA-A1ABot-M0LDA-1BDM0Xiaomi-Robotics-0GigaWorld-Policy-0Fast-WAMHy-Embodied-0.5-VLAEventVLAInternVLA-A1.5Xiaomi-Robotics-1
Vertical axis is paper date (non-linear; the dense 2026 band is expanded). WAM unifies VLA's p(a|o,l) and world models' p(o′|o,a) into the joint distribution p(o′,a|o,l), splitting into cascaded (imagine, then act) and joint (one model generates both) families; the bottom band is the RoboDojo arena snapshot (Aug 2026, 34 models). Hover for details; click a node to open the paper.
PAPERJul 16, 2026

AeroAct: Action-Centered World-Action Models for Language-Conditioned Quadrotor Flight

Language-conditioned quadrotor flight requires a policy to ground semantic goals, anticipate the visual consequences of ego-motion, and output control references that remain smooth and dynamically executable under rapidly changing first-person views. Existing aerial vision-language navigation and vision-language-action methods commonly use discrete actions, high-level waypoints, or instantaneous velocity commands, which provide limited supervision about how flight actions change future observations. We present AeroAct, an action-centered world-action model (WAM) for quadrotor navigation. To the best of our knowledge, AeroAct is the first WAM instantiated and demonstrated for real-world aerial flight. The model adapts a pretrained video diffusion Transformer to predict local trajectory-action chunks from egocentric visual history, proprioception, and language. Future first-person frames are used during training as dense consequence supervision, while deployment directly decodes actions without generating future video. To obtain aligned visual, state, language, and dynamically feasible action data, we build a DiffAero-based pipeline with complementary Isaac Lab and 3D Gaussian splatting renderers. We further introduce a low-cost handheld collection device that couples camera observations with motion estimates to recreate flight-like egocentric trajectories, and a self-guidance procedure that improves temporal consistency across overlapping trajectory chunks. Closed-loop simulation and real-world experiments show that temporal visual context improves target tracking and object-search performance, and that WAM-based policies can be executed on a physical quadrotor.

Xinhong Zhang, Qiyuan Zhu, Yubo Huang

UAVQuadrotor
Astribot Lumo-2 — 20+ household tasks, one foundation model, real robot
VIDEOJul 16, 2026

Astribot Lumo-2 — 20+ household tasks, one foundation model, real robot

Astribot's Lumo-2 spans 20+ household tasks on a real robot with one foundation model — dexterous manipulation, physical understanding, temporal reasoning, human-robot collaboration, long-horizon execution.

ACT-2 Preview — first robotics model unifying generalization with reliability
VIDEOJul 16, 2026

ACT-2 Preview — first robotics model unifying generalization with reliability

Tony Zhao's ACT-2 Preview: a single fine-tuning example teaches a new generalizable behavior — zero-shot, real unseen homes, 99% success rate.

T800 robot fights in China — robots being blind is the big issue
VIDEOJul 16, 2026

T800 robot fights in China — robots being blind is the big issue

CIX team watched T800 robot fights in China — the biggest issue is robots being blind. They share ideas to fix it.

PAPERJul 13, 2026

See like a Robot: Robot-Centric Pointmaps for Vision-Language-Action Models

This paper addresses the frame mismatch in VLA models between camera-frame observation and robot-frame action by introducing robot-centric pointmaps—images whose pixels store 3D coordinates in the robot frame. Pointmaps provide robot-frame 3D geometry while preserving image structure, enabling cross-viewpoint generalization across diverse camera setups.

Byungkun Lee, Dongyoon Hwang, Dongjin Kim

VLAPointmap
PAPERJul 11, 2026

ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memory

Recent VLM and VLA systems have improved robotic perception and action prediction, yet long-horizon embodied agents still require a general runtime layer for reasoning, memory, tool use, verification, and cross-embodiment execution. We present ABot-AgentOS, a general robotic Agent Operating System that sits above low-level controllers and provides a deliberative agent layer for scene-conditioned planning, context-isolated skill execution, multi-stage verification, multi-modal memory, and edge-cloud collaboration. To evaluate such systems, we introduce EmbodiedWorldBench, an executable benchmark with 16 indoor, outdoor, and hybrid scenes, four difficulty levels, and over 200 tasks involving navigation, object search, NPC dialogue, dynamic events, and trace-grounded scoring. ABot-AgentOS further introduces Universal Multi-modal Graph Memory, a persistent source-grounded substrate that converts dialogue, visual observations, spatial context, temporal relations, and task traces into typed nodes and edges. A failure-driven self-evolution loop converts diagnosed memory failures into gated runtime evo-assets that are promoted only to later evaluation splits, preventing current-split ground-truth leakage while enabling continual improvement. On an initial EmbodiedWorldBench subset, ABot-AgentOS improves over a single-controller baseline in both task success and goal completion. Across memory benchmarks, ABot-AgentOS Static achieves 87.5 on LoCoMo, 59.9 on OpenEQA EM-EQA, 88.6 on Mem-Gallery, and 76.5 Acc@All on NExT-QA; self-evolution further improves LoCoMo to 88.7, OpenEQA to 60.4, and Mem-Gallery to 89.0. These results suggest that a general Agent OS layer can improve long-horizon embodied execution while providing persistent, auditable memory for continual interaction.

Jiayi Tian, Shiao Liu, Yuting Xu

AgentOSEmbodiedWorldBench
PAPERJul 9, 2026

Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents

Language-conditioned manipulation requires both precise contact-rich control and robust reasoning over language, scenes, and long horizons. End-to-end Vision-Language-Action (VLA) models provide strong local visuomotor skills, but they are trained on in-distribution task trajectories and often fail under deployment perturbations such as semantic retargeting, goal re-binding, spatial-layout shifts, and unstable local contacts. LLM coding agents provide complementary semantic and compositional reasoning, but purely analytic primitives struggle with irregular grasping, constrained placement, and articulated-object interaction. We present Harness VLA, a memory-augmented agentic framework that exposes a frozen VLA as a retryable contact-rich primitive and composes it with a small fixed library of analytic primitives for grounding, staging, transport, navigation, and release. Rather than expanding the skill library, the harness learns the operating range of these fixed primitives from task-specific execution traces, global success rules, and failure models. By lifting semantic re-grounding, non-contact execution, and VLA re-staging to the planner while reserving the frozen VLA for local contact-rich phases, Harness VLA extends pretrained VLAs beyond their original trajectory distribution without finetuning. Across perturbed tabletop, household kitchen, and clean-to-randomized bimanual manipulation, Harness VLA improves over the strongest relevant baselines by 38.6 and 27.1 percentage points on LIBERO-Pro and RoboCasa365, respectively, and reaches 58.4% on RoboTwin C2R. Code is available at https://github.com/RLinf/RPent.

Yixian Zhang, Huanming Zhang, Feng Gao

VLARobot Manipulation
PAPERJul 8, 2026

EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data

Egocentric human data offers scalable supervision for robot manipulation. However, behavior cloning entangles transferable content like objects, scenes, and task semantics, with non-transferable factors like human morphology, head motion, and behavioral style. We study whether World Action Models (WAMs) provide a better training signal by requiring policies to predict not only actions, but also how the scene evolves. The central question is what world representation best enables human-to-robot transfer. We hypothesize that an effective world target should abstract appearance, capture agent-invariant physical effects, and separate camera motion from environment change. We introduce EgoWAM, a controlled human-robot co-training framework that fixes the policy backbone, action head, and data mixture while varying only the world prediction target, comparing Pixel, DINO, and 3D motion flow. Across three real-world bimanual tasks, WAM co-training scales more effectively with in-the-wild egocentric human data than behavior cloning. Pixel-based prediction transfers weakly, while DINO and 3D flow yield substantial gains: DINO improves out-of-distribution object and scene generalization by up to 4x, and 3D flow improves in-domain performance by 20-30%. More details: https://gatech-rl2.github.io/egowam.github.io

Baoyu Li, Xinchen Yin, Mengying Lin

World ModelsRobot Manipulation
LingBot-World 2.0 Opens Infinite Worlds
VIDEOJul 8, 2026

LingBot-World 2.0 Opens Infinite Worlds

Robbyant open-sources LingBot-World 2.0 for hour-long interactive 720p worlds.

PAPERJul 2, 2026

Actuator Reality Shaping for Zero-Shot Sim-to-Real Robot Learning

Sim-to-real transfer in robot learning is often limited by discrepancies between the ideal actuator dynamics assumed during policy training and the nonlinear, hardware-dependent behavior of physical motors. While conventional approaches attempt to bridge this gap by increasing simulator fidelity through system identification, domain randomization, or learned actuator models, we introduce an alternative paradigm: actuator reality shaping. Instead of modifying the simulator to match the real world, our method shapes the closed-loop behavior of physical actuators to match the idealized second-order reference dynamics used in simulation. By equipping each joint with a two-degree-of-freedom feedforward–feedback controller, we decouple reference-response shaping from robust stabilization, thereby providing a standardized actuator interface for reinforcement learning policies. As a result, policies trained only with the prescribed reference model can be deployed zero-shot on real hardware without task-level fine-tuning or learned actuator models. We validate the approach on a single-joint high-gear-ratio servo under external loads and a 7-DOF robotic arm reaching task, where actuator reality shaping substantially reduces sim-to-real tracking error and improves zero-shot task performance compared with standard servo-control and representative real-to-sim-to-real baselines. We further demonstrate zero-shot transfer on a wheeled-legged robot driving over a slope and a humanoid robot walking, suggesting that actuator reality shaping can serve as a reusable interface for robot learning across diverse hardware platforms. Project page: https://syamamori.github.io/ActuatorRealityShaping.github.io/

Satoshi Yamamori, Koji Ishihara, Kenjiro Minamikawa

Sim-to-RealZero-Shot
PAPERJul 2, 2026

VLA-Corrector: Lightweight Detect-and-Correct Inference for Adaptive Action Horizon

Vision-Language-Action (VLA) foundation models have recently achieved strong progress in embodied intelligence. To reduce policy-call frequency while preserving temporal coherence, most generative policies adopt an action chunk mechanism, executing multiple future actions in an open-loop manner under a fixed action horizon. However, this "predict-then-blindly-execute" paradigm sacrifices closed-loop reactivity: in contact-rich physical interactions, even small local perturbations can rapidly amplify within the open-loop blind spot, leading to compounding errors and ultimately task failure. To address this limitation, we propose VLA-Corrector, a lightweight corrective inference framework for action-chunked VLA policies. Without modifying the backbone policy weights, VLA-Corrector introduces a lightweight Latent-space Vision Monitor (LVM) that continuously compares predicted and actual visual feature evolution, enabling online detection of visual dynamics deviations. Once persistent deviation is detected, the system triggers a truncation event, discards the remaining stale actions, and invokes corrective replanning via Online Gradient Guidance (OGG). The detect-and-correct mechanism of VLA-Corrector naturally induces an event-triggered adaptive action horizon: it preserves long-horizon execution when the current chunk remains reliable, and invokes short-horizon corrective replanning when execution begins to drift. In doing so, VLA-Corrector mitigates the trade-off imposed by static horizons between execution robustness and policy-call frequency. It can be integrated into different VLA models without further retraining the VLA backbone, interrupting compounding errors while preserving much of the efficiency benefit of action chunking and substantially improving robustness in long-horizon, contact-rich robotic manipulation tasks.

Yi Pan, Miao Pan, Qi Lu

VLARobotic Manipulation
NVIDIA SimFoundry turns real scenes into simulation worlds
VIDEOJul 2, 2026

NVIDIA SimFoundry turns real scenes into simulation worlds

NVIDIA GEAR SimFoundry builds simulation-ready worlds from a single image or video for real2sim2real robot training.

As an Amazon Associate, we earn from qualifying purchases.