#Robot Manipulation (103)

ASPIRE is the continual-learning system NVIDIA open-sourced with the University of Michigan, UIUC, Berkeley and CMU (paper arXiv:2607.00272, Apache-2.0). No gradient descent: a coding agent repeatedly writes, runs and repairs robot control programs (code-as-policy) inside a closed-loop execution engine, promotes only verified fixes into a reusable skill library, and uses evolutionary search to escape single-trajectory patching. Driven by Claude Opus 5.5 it averages 99% success across all 80 LIBERO-Pro tasks and 100% on representative Robosuite and BEHAVIOR-1K tasks; perturbed manipulation improves by up to 77%, and unseen long-horizon tasks reach 31% zero-shot against a 4% baseline. On hardware, skills discovered in simulation transfer across embodiments to a real dual-arm station, and online learning even teaches the robot to solve a Rubik's Cube.

Code-as-PolicySkill Library

Python884784

0:28

Liber-0: Zero-Shot Robot Manipulation

@CyberRobooo · 17 views · 2026-11-04

World-action modelsZero-Shot Manipulation

Democratizing Robot-Use Agents: A Unified Harness with Open Models

OpenRUA's five-layer harness lets Qwen reach 31.39 without depth on RoboDojo and 53.50 zero-shot on physical tasks, while exposing grounding, geometry, and hardware-support limits.

BLOG

Robot-Use AgentsAgent Harness

0:09

PhysiClaw Gives AI Agents a Physical iPhone Loop

@stevencheng · 21 views · 2026-11-03

Physical AIRobotic Manipulation

T2-z: Test-Time Optimization of Environment Latents for Robotic World Models

Predicting how the world responds to an action requires more than what can be seen: robots must account for environmental factors that are neither directly observable nor explicitly specified, from physical properties such as friction and mass to human-defined environmental rules. We introduce T2-Z for test-time optimization of environment latents, which augments a robotic world model with a learnable environment latent to condition world predictions. This compact latent captures underlying environmental variation without supervision of what environmental factors exist or what values they take. During training, T2-Z alternates between two stages: a world-model optimization stage that fixes the latents and learns general action-conditioned dynamics across environments; and an environment-latent optimization stage that freezes the world model and optimizes the conditioning latents, allowing latent structure to emerge purely through world prediction. At test time, the world model remains frozen and a newly initialized environment latent is optimized from one or more test-time interactions, adapting the model to previously unseen environmental configurations. Across 7 simulated settings and 4 real-world robotic tasks, T2-Z reduces object prediction error metric from 15.20 to 7.00 and raises action success from 33.73% to 72.96%. Its emergent latents capture structured representations of individual and compounded factors across diverse visual scenes. Ablations further show that T2-Z can efficiently infer environment latents from as little as a single test-time interaction trajectory.

Chenyuan Zhou, Yining Hong, Huang Huang · 61 views · Oct 15, 2026

Robotic World ModelsTest-Time Optimization

Oct 15, 2026

StereoPolicy: Improving Robotic Manipulation Policies via Stereo Perception

StereoPolicy learns task-conditioned binocular representations directly from synchronized stereo pairs instead of reconstructing explicit depth or point clouds. A Stereo Transformer alternates within-image and cross-image attention over shared 2D features, and the fused representation integrates with diffusion policies, pi0.5, and GR00T-N1.5. Experiments across simulation and real tabletop and bimanual mobile manipulation show consistent gains over RGB, RGB-D, point-cloud, and multi-view baselines, together with practical guidance on stereo baseline and working distance.

Evans Han, Yunfan Jiang, Yingke Wang · 57 views · May 11, 2026

Stereo visionRobot Manipulation

May 11, 2026

Fast-VLA: You Only Think Once for Robot Manipulation

Vision-language-action (VLA) policies commonly place a multi-billion-parameter model inside the feedback loop, repeatedly recomputing task context whenever they predict a new action chunk. Yet task understanding changes far more slowly than the robot state. We introduce Fast-VLA, which invokes a frozen 4B VLM once per episode to contextualize the instruction and initial views as fixed, multi-layer key-value task memory. During execution, a lightweight online actor fuses learned time- and arm-indexed queries with current visual and proprioceptive feedback to form action queries. These queries retrieve task context from the fixed memory and predict action chunks without re-running the foundation model. Fast-VLA achieves 92.46% and 92.40% success on RoboTwin 2.0 Clean and Randomized, and 95.95% and 79.11% on LIBERO and LIBERO-Plus evaluation. Against three continually conditioned baselines on the same hardware, it reduces RoboTwin amortized policy inference time by 10.9-17.1x and requires 1.22 GiB of post-prefill online resident memory. Task-memory swaps further show that the actor relies on episode-matched task memory. These results show that task context can be established once while closed-loop control remains responsive throughout execution. Project page: https://garfield-kh.github.io/FastVLA

Kehong Gong, Guanli Hou, Xiaoxu Zheng · 92 views · Oct 15, 2026

Vision-language-action (VLA)Robot Manipulation

Oct 15, 2026

0:12

OM-1 Learns Stair Grasping from Human Demonstrations

@robotsdigest · 54 views · 2026-11-02

HumanoidRobot Manipulation

From RT-1 to MolmoAct 2: A Brief History of Vision-Language-Action Models

A complete history of VLAs from SayCan and RT-1 through RT-2, RT-X, Octo, OpenVLA, pi0, SmolVLA and MolmoAct 2, including the author's action-discretization experiment.

BLOG

Vision-language-action (VLA)Robot Learning

0:58

I3L Turns Human Takeovers Into Better Robot Policies

@robotsdigest · 81 views · 2026-10-31

Imitation LearningInteractive Learning

FORCA: A Force-Aware Coding Agent for Contact-Rich Robot Manipulation

Coding agents can translate language instructions into robot programs. For contact-rich manipulation, a key challenge is to generate and refine control logic that responds to physical interaction. We present FORCA, a coding agent that takes visual observations and wrist force/torque (F/T) measurements as input to write Python programs for contact probing and force-responsive motion. Between executions, the agent inspects force records alongside visual outcomes to diagnose failures and revise both motion and force-feedback control logic. FORCA requires no demonstrations, task-specific training, or simulation. Across five real-robot tasks, we demonstrate its effectiveness in three forms of contact-rich manipulation: improving manipulation accuracy, adapting to changes in object or surface properties, and regulating contact under surface variation and object motion. Compared with a vision-only variant of the same agent, FORCA improves grasp accuracy and task success and reduces contact loss in wiping and peeling.

Xiao Tan, Yixuan Wang, Zhexi Wang · 403 views · Oct 22, 2026

Contact-Rich ManipulationCoding Agent

Oct 22, 2026

ASPIRE: Agentic /Skills Discovery for Robotics

ASPIRE diagnoses robot-program failures from multimodal traces, repairs executable policies through evolutionary search, and compounds verified fixes into a reusable skill library that transfers across tasks, embodiments, and real robots.

Runyu Lu, Yubo Wu, Ethan Kou · 357 views · Jun 30, 2026

ASPIREAgentic Robotics

Jun 30, 2026

FAR: Failure-Aware Retry for Test-Time Recovery and Continual Policy Improvement

Robot policies trained on offline demonstrations fail inevitably at deployment, and naive retries repeat the same mistake because failure states are out of distribution for the pretrained policy. FAR (Failure-Aware Retry) keeps the scene after a failure, returns the arm to its start pose, and retries: an IQL-style conservative critic attributes the failure to the action chunks whose value drop falls in the lowest percentile, candidate chunks sampled from the current policy are distance-filtered and critic-ranked into top-K positives, and a diffusion preference-optimization loss updates the policy for only 10 gradient steps (1.5-3 s on a single A5000) while temporally smoothed Gaussian perturbations add local exploration during retries. Successful recovery trajectories then feed an advantage-weighted denoising objective for continual policy improvement. Across nine ManiSkill, RoboSuite and RoboMimic tasks FAR beats standard diffusion policy by 17.6 points on average, and on three xArm real-world tasks by 11.7 points (80.0% vs 68.3%), while cutting the environment resets that online learning needs.

Haoran Hao, Shahram Najam Syed, Jeffrey Ichnowski · 184 views · Jul 1, 2026

Failure RecoveryTest-Time Adaptation

Jul 1, 2026

PACE: Temporal-Role Tactile Conditioning for Chunk-Based Insertion Policies

Contact-rich insertion requires a robot to combine visual alignment with tactile adjustment. Existing visuotactile chunk-based policies often collapse tactile feedback into one generation-time pathway, so persistent contact state, abrupt contact events, and the future contact consequences of an action are modeled implicitly in a single representation. PACE assigns one tactile stream three temporal roles. An exponential-smoothed contact state modulates action-chunk generation through layer-wise feature conditioning. A short-window tactile difference forms an event cue and drives a gated residual correction over only the first steps of the chunk. A training-only head predicts future tactile increments from decoder states as an auxiliary representation-learning objective. This preserves chunk-level action continuity while giving slowly varying contact phases and fast collision or jamming events separate pathways. Across five real insertion tasks, PACE reaches 90.7 percent average success, more than 44 percentage points above vision-only and single-path tactile-fusion baselines. It also averages 88.7 percent on six ManipulationNet geometry and clearance settings, including 0.1 mm clearance. Ablations show complementary contributions from Phase, Event, and Pred., while cross-signal and cross-architecture tests extend the principle to force/torque, joint current, and diffusion-policy backbones.

Zhiyuan Chen, Shixiong Xu, Shuai Chen · 242 views · Oct 12, 2026

Tactile SensingRobot Manipulation

Oct 12, 2026

LeWAM: End-to-End World and Action Modeling with JEPAs

Joint-embedding predictive architectures (JEPAs) learn compact latent world models without pixel reconstruction. Existing JEPA pipelines typically train visual representations and action-conditioned dynamics before adding a control policy or solver. We introduce LeWAM, an end-to-end JEPA for control with: (1) a joint training objective over next latent state prediction and action flow matching, and (2) an isotropic Gaussian constraint on the latent state, enabling a single model to support direct policy execution and efficient planning by evaluating sampled actions with its learned latent dynamics. Compared with prior JEPAs in visual planning, LeWAM improves success rate from 28.6% to 89.7% on contact-rich tasks and from 10.9% to 46.2% on long-horizon tasks, and plans 32.3 times faster. On behavior cloning tasks in simulation and physical robots, LeWAM is also competitive with established methods. Interpretability analyses show that LeWAM learns control-relevant representations and action distributions effectively guiding test-time planning and control.

Minghao Fu, Tavis Siebert, Eryk Halicki · 172 views · Oct 8, 2026

JEPAWorld Models

Oct 8, 2026

Agentic World Action Models

Recent world action models provide visual foresight and future prediction for robot manipulation, but lack explicit mechanisms for compositional skill reuse. Agentic robotics combines code-based planning and reasoning with reusable skills, but often lacks predictive grounding in physical dynamics. We introduce AW-0, an agentic world action model framework that integrates code-based planning with joint video and action prediction. We jointly model code, future video, and robot actions as three coupled generative streams within a unified mixture-of-transformers architecture. Within this architecture a code expert generates parameterized skill routines and control logic while video and action experts generate future visual states and low-level motor commands. Code serves two complementary roles: it structures multi-step prediction and control within the world model and represents skills as reusable programs for subsequent agentic execution. We further introduce an automated curation pipeline that labels robot and human video trajectories with aligned programs and consolidates recurring routines into reusable skill libraries. Evaluations on RoboTwin 2.0 and physical bimanual manipulation tasks show improved task performance over VLA and WAM baselines, alongside gains in language following, semantic reasoning and robustness, demonstrating that agentic code enables connection between planning, prediction and execution within WAMs.

Liu, Zhuoyang, Niu, Dantong, Zhang, Botong · 242 views · Oct 21, 2026

World-action modelsVision-language-action (VLA)

Oct 21, 2026

AW-0: Planning With Code Inside a Joint World Action Model

AW-0 jointly denoises a generated skill program, future video and action chunk. Code guides control without being executed by default, while a gated hybrid mode can run verified programs.

BLOG

World-action modelsEmbodied AI

AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics

Agentic robot systems pair a language-model planner with a robot harness, but recent gains have largely come from stronger closed-source planners, better harnesses, and external memory rather than from improving the planner itself. AgenticRoboRL investigates whether coding-agent post-training can improve robot planners across both VLA skill invocation and control-code generation. Its recipe combines supervised fine-tuning on successful teacher trajectories with multi-turn reinforcement learning using task success as the terminal reward. A recording proxy preserves each sampled response and its actual context, while harness-specific adapters associate model calls with task outcomes and exclude infrastructure-invalid episodes. Only the planner is trained; perception and execution components remain fixed. With a 9B planner, success rises from 18.1% to 71.0% on LIBERO-PRO and from 19.2% to 72.2% on LIBERO-Plus. With the CaP-X control-code harness, the same post-training pipeline raises success from 2.7% to 28.5%.

Xinxin Liu, Bingxin Xu, Wenrui Bao · 173 views · Oct 20, 2026

Agentic RoboticsRobot Manipulation

Oct 20, 2026

0:45

Phone video to robot digital twin in under an hour

@realfxw · 131 views · 2026-10-24

RoboSimStudioDigital twins

2:52

OneWM: World and Action Models That Strengthen Each Other

@Selen7005717917 · 146 views · 2026-10-24

World ModelsRobot Manipulation

Can World and Action Modeling Benefit Each Other under Unified Causal Training?

World and action modeling are closely related, yet their reciprocal benefits under unified causal training remain unclear. We introduce OneWM, a simple framework for unified causal training that treats actions as both conditions for world prediction and targets for policy learning within a shared video-action architecture. Using this framework, we investigate whether learning forward dynamics conditioned on specified actions improves action learning, and whether learning to generate actions in turn improves action-conditioned world modeling. We use LIBERO, LIBERO-Plus, and RoboCasa365 as embodied testbeds to evaluate both policy learning and world modeling, complemented by real-world policy experiments. Learning the visual consequences of specified actions improves policy performance and generalization across changing scenes. In turn, action learning improves multi-view world-prediction fidelity under the same architecture. Together, these results answer both questions positively, supporting reciprocal benefits between world modeling and action generation under unified causal training.

Yue Su, Bokai Lin, Xinkai Wang · 198 views · Oct 15, 2026

World ModelsRobot Manipulation

Oct 15, 2026

Make π0.5 Learn to Call Astra for Reliable Robot Manipulation

A frozen pi0.5 policy uses a lightweight CALL head to predict when external reasoning is needed. Astra takes over only for selected recovery events, raising LIBERO success from 95.5% to 97.5% while keeping outside calls sparse.

Jianli Sun, Chengxiang Li, Yonglin Tian · 213 views · Oct 9, 2026

Vision-language-action (VLA)Robot Manipulation

Oct 9, 2026

T-WLA: Bridging Slow Vision and Fast Touch

T-WLA shares a history-aware visual memory between a slow vision-language-action policy and a fast tactile residual editor. The visual branch produces an action chunk and memory at 39.7 Hz; the lightweight tactile editor reuses them to correct only unexecuted actions at 119.1 Hz. It improves tactile-conditioned hand-motion prediction and contact-rich manipulation, reaching 50.0% average success on four real-robot tasks with 30 demonstrations each.

Kaichen Zhou, Yuzhen Chen, Hannah Yao · 268 views · Oct 20, 2026

Tactile SensingVisuotactile Manipulation

Oct 20, 2026

3D Point World Models: Point Completion Enables More Accurate Dynamics Learning

Learning predictive models of the world enables robotic control through planning and can allow robots to improvise on new tasks. Video-based dynamics models lack explicit 3D structure and accumulate geometric inconsistencies, while existing partial-point-cloud world models remain sensitive to occlusion and rollout drift. 3DPWM first completes segmented object geometry from a single calibrated RGB-D view and robot joint angles, then learns action-conditioned per-point dynamics over the completed scene. Experiments across seven datasets and two embodiments show accurate 100-300 step rollouts, stronger reward correlation, improved open-loop and closed-loop planning, adaptation to an unseen long-horizon task, and sim-to-real transfer.

Skand Peri, Hung Nguyen, Chanho Kim · 190 views · Jun 30, 2026

3D World ModelPoint Cloud Completion

Jun 30, 2026

As an Amazon Associate, we earn from qualifying purchases.