Skip to content
Agientry

World Models

WaterfallPagination
WORLD MODELS · TIMELINE

World Models: 36 Years from Internal Models to WAMs

1990–2026 · Five lineages · 33 milestones

Early explorationLatent-space eraVideo generation · Physical AI · VLAWAM unification1990200020102018202020222023202420252026Model-based RL · Latent DynamicsVideo-generation World ModelsJEPA Predictive EmbeddingsVLA · World Action ModelsEmbodied / Physical World ModelsWorld Models1990 · SchmidhuberPILCO2011 · Deisenroth & RasmussenWorld Models2018 · Ha & SchmidhuberPlaNet2018 · DeepMindMuZero2019 · DeepMindDreamer2019 · Hafner et al.DreamerV22020 · Hafner et al.TD-MPC2022 · Hansen et al.DreamerV32023 · Hafner et al.I-JEPA2023 · Meta AIGAIA-12023 · WayveUniSim2023 · GoogleSora2024 · OpenAIGenie2024 · DeepMindV-JEPA2024 · Meta AI1X World Model2024 · 1X TechnologiesGenie 22024 · DeepMindCosmos2025 · NVIDIAV-JEPA 22025 · Meta AIGenie 32025 · DeepMindRT-22023 · Google DeepMindOpenVLA2024 · Stanford et al.π02024 · Physical IntelligenceGR00T N12025 · NVIDIAHelix2025 · Figure AIWAMs Survey2026 · Fudan University · Shanghai Innovation Institute · NUSACT2023 · StanfordRDT / H-RDT2024 · Tsinghuaπ0.52025 · Physical IntelligenceOpenVLA-OFT2025 · Stanford · BerkeleyGR00T N1.72026 · NVIDIAXiaomi-Robotics-02026 · XiaomiGigaWorld-Policy-02026 · GigaWorld Team
The x-axis is non-linear (1990–2017 compressed). Hover a node for details.← swipe to explore the full timeline →
WAM PHYLOGENY · 2023–2026

Branching Evolution: From VLA to World Action Models

61 site-collected papers · four lineages · taxonomy per arXiv:2605.12090

2023202420252026timeVLA · p(a|o,l)Cascaded WAMJoint WAMWorld models · p(o′|o,a)RT-2OctoHelixGR00T N1Pointmap-VLACosFly-VLAAC-VLAJoyNexusPOT-VLAFM-VLAMoE-VLAN₀-VTLAτFutureRTCRL²-VLAπR²Ego2RobotAtlasVLAG0.5HAFWAMs SurveyHyWorldVLADeVALeapBot-WARobust-WAMAeroActWorldDiTN₀-TWAMX-NavDPω-0SimWAMDynaWMSiamJEPAOrbis 2When to PlanKoopman DreamerPhysCoReWorld TranslationMusic-JEPAViTacWorldRobot-Factored WMGS-AgentFeelWorldPhyLatentPhyAITinyVLAMolmoActSpatial ForcingX-VLASmolVLAInternVLA-A1ABot-M0LDA-1BDM0Xiaomi-Robotics-0GigaWorld-Policy-0Fast-WAMHy-Embodied-0.5-VLAEventVLAInternVLA-A1.5Xiaomi-Robotics-1
Vertical axis is paper date (non-linear; the dense 2026 band is expanded). WAM unifies VLA's p(a|o,l) and world models' p(o′|o,a) into the joint distribution p(o′,a|o,l), splitting into cascaded (imagine, then act) and joint (one model generates both) families; the bottom band is the RoboDojo arena snapshot (Aug 2026, 34 models). Hover for details; click a node to open the paper.
PAPERJul 28, 2026

πR²: Reactive Real-time Flow Policies

Generalist manipulation policies increasingly take the form of action-chunking flow policies built on large pretrained backbones. Such chunks run open-loop, so the policy cannot react to sensory input arriving mid-execution, sacrificing reactivity. Replanning more often would restore it, but the perception-to-action pipeline (a large backbone plus multiple denoising steps) is too slow: this latency forbids frequent replanning and leaves committed actions stale, making such policies ill-suited for dynamic, closed-loop control. We present πR², which makes these policies reactive and real-time while retaining large backbones, expressive multi-modal policies, and multi-action prediction. Built on the per-position noise schedule of diffusion forcing, πR² contributes two ideas. First, it splits conditioning into a fast channel (proprioception, fresh every tick) and an asynchronously updated slow channel (vision-language features), so the policy reacts to proprioception within a chunk while tolerating stale vision. Second, a latency-adaptive flow schedule treats in-flight actions as inpainting conditioning and emits actions in one denoising step per call, letting one trained model adapt to varying hardware latency. Requiring minimal modification to existing architectures, πR² can be finetuned from a pretrained policy: applied to GR00T-N1.7 on a real xArm6+XHand platform, it replans closed-loop roughly 4× faster than the base policy (~$25$Hz on an A5000 GPU), acting on a fresh observation every $40$ms. Across simulation and real-world manipulation tasks, πR² improves the success rate by up to 23% in simulation and 30% in the real world over the strongest baseline. Project page: https://pi-r2-flow.github.io/

Sungjae Park, Shubham Tulsiani

VLAreactive
PAPERJul 28, 2026

HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone

Learning deployable manipulation policies is bottlenecked by the scarcity of data that is both high-fidelity and scalable. Real-robot teleoperation is accurate but costly to scale; robot-free UMI capture scales readily, and current practice uses the resulting data mainly for pre-training, adding a small real-robot "anchor" at post-training. We ask whether raising the fidelity of robot-free UMI data, rather than shrinking the real-robot fraction, can remove that anchor. We present HiFi-UMI, a portable UMI data-production system co-designed for trajectory accuracy, inter-gripper relative pose, synchronization, and field of view: head-mounted offline stereo-inertial SLAM, native rather than reconstructed relative pose, a shared microsecond GPIO trigger, and two wide-angle cameras per hand covering ~200 degrees. It reaches 3 mm workspace-local end-effector accuracy without external tracking infrastructure. Using this corpus, we demonstrate zero-robot post-training: a policy post-trained solely on HiFi-UMI demonstrations deploys directly on a real robot and matches in-domain teleoperation across three backbones spanning the vision-language-action and world-action-model families, with success-rate differences of -2.5, +3.1, and -0.6 percentage points on StarVLA-QwenPI, OpenPI-pi_0.5, and LingBot-VA; the strongest policy reaches 85% on a precision insertion task, even though the teleoperation baseline is collected in the evaluation scene and no HiFi-UMI trajectory is. Pre-training on 4,000 hours from the same corpus lowers action error on ten unseen tasks by 41% and, on StarVLA-QwenPI, raises real-robot success by a further 18.1 percentage points. We open-source HiFi-UMI-2K, 2,000 hours of microsecond-synchronized, ultra-wide-FoV demonstrations, each automatically reconstructed and validated through simulation replay, as a large-scale, high-fidelity resource for the robot-learning community.

Simple AI, :, Yuteng Wei

UMIVLA
DexRobot open-sources SO101 Pick Cube SFT checkpoint and DM0.5 LoRA SFT workflow
VIDEOJul 28, 2026

DexRobot open-sources SO101 Pick Cube SFT checkpoint and DM0.5 LoRA SFT workflow

DexRobot open-sources the SO101 Pick Cube SFT checkpoint and full LoRA SFT workflow for DM0.5; after 50 episodes of orange-cube training, the robot picks the cube and places it on the plate.

CoorDex: Humanoids Walk and Manipulate Simultaneously
VIDEOJul 28, 2026

CoorDex: Humanoids Walk and Manipulate Simultaneously

CoorDex enables humanoids to walk and manipulate simultaneously by coordinating body and dexhand motion priors in latent space; AnyBody achieves coordinated whole-body control from sparse keypoints.

Being-H0.8: Latent Tactile World Action Model
VIDEOJul 28, 2026

Being-H0.8: Latent Tactile World Action Model

Brings touch into large-scale embodied foundation model pretraining, unifying vision, touch, action, and future world dynamics in a shared latent space

PAPERJul 27, 2026

The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation

Multi-turn long-horizon planning is critical for foundation model agents, yet how to fundamentally improve it remains unclear. Existing models are trained on uncontrollable and opaque Internet data, making it difficult to identify how planning ability is acquired, shaped, and integrated. To address this challenge, we introduce a unified and controlled multi-turn environment that enables precise control. It allows systematically study long-horizon planning across three stages. (1) Planning ability acquisition during pre-training. We study data format, distribution, and quality. Explicit world model construction through CoT state transition modeling yields stronger long-horizon generalization. Atomic skills alone are insufficient for compositional generalization, whereas a litte long-horizon data works. Moreover, suboptimal trajectories severely impair performance because errors amplify over long horizons. (2) Planning ability shaping via GRPO and OPD post-training. Through mutual information, we distinguish general planning patterns from task-specific planning knowledge. For planning patterns, we identify three application regions of post-training: unnecessary, effective, and unsupported. OPD has a broader effective region than GRPO under low-quality and long-horizon settings, as it provides more consistent update directions. For planning knowledge, distilling unseen procedures from a teacher with different knowledge may impair student's prior world modeling without fully establishing new knowledge. (3) Planning ability integration through MOPD post-training. We show that multi-teacher on-policy distillation (MOPD) integrates capabilities by converging to shared planning-pattern across environments. Compatible patterns enable cross-environment generalization, partially shared patterns support continual learning, while completely conflicting patterns cause severe interference.

Tianyi Men, Zhuoran Jin, Kang Liu

World Models
PAPERJul 27, 2026

τ: Learning Touch-Augmented Vision-Language-Action Models from Future Visual Supervision

Learning the informative tactile representation while effectively adapting it to pretrained Vision-Language-Action (VLA) models remains challenging at both the data and modeling levels. At the data level, limited task-specific demonstrations constrain representation quality, whereas large-scale pretraining incurs substantial costs. At the modeling level, existing methods either focus on instantaneous contact states or model temporal interaction dynamics using 6D wrench sequences, leaving high-dimensional tactile signals underexplored. To address these challenges, we present τ, a touch-augmented VLA framework that learns an action-conditioned spatiotemporal tactile representation from future visual supervision inspired by the Joint-Embedding Predictive Architecture (JEPA), and fuses it with vision-language features for action generation under limited data. This supervision operates in latent space and is used only during training, adding no deployment overhead. We also introduce TacAura, a dataset of synchronized vision, proprioception, and vision-based tactile signals across four representative contact-rich manipulation tasks. Experiments show that τ outperforms existing models and generalizes to unseen objects and scenes, delivering improved manipulation performance and robustness

Ning Cheng, Jinan Xu, Wanlin Li

VLAVision-language-action (VLA)
PAPERJul 27, 2026

FeelWorld: Visuo-Tactile World Model for Hierarchical Contact Prediction and Planning

Humans plan physical interactions by imagining the possible outcomes of candidate actions. However, existing visual world models primarily capture appearance dynamics while overlooking the tactile states that govern contact-rich interactions, potentially producing imagined futures that appear visually plausible but violate physical dynamics. We introduce FeelWorld, a hierarchical visuo-tactile world model that jointly predicts future visual latents and three tactile states. FeelWorld organizes these states hierarchically as contact state, a 3D tactile latent that encodes force-related information, and slip state. These states are jointly predicted by a shared latent dynamics model with explicit supervision. To prevent irrelevant tactile signals during free-space motion from degrading visual prediction, we introduce a contact-gated asymmetric attention mechanism that maintains a visual-only prediction pathway before contact and enables joint visuo-tactile dynamics prediction during contact. The model is further trained with autoregressive rollouts and context noise injection to improve robustness to compounding errors. The predicted contact and slip states also support contact-aware CEM planning. Experiments on chip grasping, fruit grasping, and USB insertion show that FeelWorld reduces 10-step LPIPS from 0.084 to 0.058 and maintains an LPIPS that is 61% lower than that of the visual baseline after an 80-step autoregressive rollout. FeelWorld also achieves an average zero-shot planning success rate of 81.7%, providing an effective approach for incorporating tactile sensing into world models.

Wenxuan Ma, Chaofan Zhang, Chao Xue

World Models
PAPERJul 27, 2026

DeVA: Decoupled Video-Action Model with physical guidance for robot policy learning

Generalizable robot manipulation requires policies that can anticipate how visual scenes evolve while executing language instructions. While recent Vision-Language-Action models benefit from large-scale pretraining, their predominantly static pretraining objectives provide limited supervision for physical dynamics and temporal causality, leaving control-relevant knowledge to be learned from downstream robot demonstrations. Video generative models offer a promising foundation by encoding rich spatiotemporal priors through future predictions. However, existing Video-Action Models either couple video and action prediction in a shared backbone, making policy adaptation harder to optimize, or under-utilize video information when guiding the action branch. In this work, we introduce DeVA, a Decoupled Video-Action model with specialized video and action experts, multi-level feature transfer, and physically salient guidance. DeVA transfers representations from multiple video layers to the action expert, enabling rich information exchange while making policy learning more tractable. It further supervises intermediate video features and the action stream with physically salient guidance (affordance/depth). Experiments on both simulation benchmarks and real-world deployment demonstrate strong performance with limited data, faster convergence than a unified architecture, and clear performance gains from physical guidance.

Mengqi Zhang, Sahil Khose, Simar Kareer

VLAVision-language-action (VLA)
PAPERJul 27, 2026

FutureRTC: Real-Time Robot Execution with Anticipatory-Conditioned Action Chunking

Real-time deployment of Vision-Language-Action (VLA) policies necessitates asynchronous execution, wherein subsequent action chunks are computed concurrently with the execution of the current chunk, leading to prediction-execution misalignment and manifesting as inter-chunk discontinuities. Existing methods either superficially smooth chunk boundaries, require costly policy optimization, or exclusively forward-predict proprioceptive states yet neglect critical visual observations. In this paper, we propose FutureRTC, a plug-and-play adaptation framework that predicts execution-time observations and states for asynchronous VLA control without modifying the underlying policy. Specifically, FutureRTC features a state correction module to compensate for the discrepancy between rolled-forward and actual execution-time proprioceptive states and an observation prediction module that forecasts execution-time visual representations by leveraging robot motion as an explicit physical prior through motion-aware feature transport and reconstruction. Furthermore, we introduce a policy consistency loss to align the action chunks generated from predicted contexts with those produced under the expected execution-time inputs of the VLA policy. Extensive experiments across simulated and real-world environments demonstrate that FutureRTC achieves superior robustness to inference delays, resulting in smoother trajectories, faster execution, and consistently higher task success rates.

Hai Jiang, Yixian Zou, Binbin Liang

VLAVision-language-action (VLA)
PAPERJul 27, 2026

WorldDiT: A Unified Diffusion Architecture for World and Action Modeling

Many recent robot policies pursue stronger control by using large pretrained vision-language models (VLMs) as the action backbone. We introduce WorldDiT, a unified diffusion transformer architecture that couples action generation with visual world modeling and achieves strong performance without a large pretrained VLM action backbone. During training, a single diffusion transformer generates continuous action chunks and predicts normalized RGB patch targets from future camera frames. Across four LIBERO simulation suites, WorldDiT lies on the reported Pareto frontier for total model parameters and mean success among methods reporting all four suites. These results provide a strong sub-billion-parameter baseline for future scaling studies.

Sen Wang, R. Gnana Praveen, Bidhan Roy

World Models
RoFacto: Action-Conditioned Diffusion World Model
VIDEOJul 27, 2026

RoFacto: Action-Conditioned Diffusion World Model

RoFacto is an action-conditioned diffusion world model that sees actions beyond mere numbers, robots act through bodies not vectors

As an Amazon Associate, we earn from qualifying purchases.