#Imitation Learning (53)

Waterfall Pagination

2:23

Pixel: A Low-Cost Full-Stack Humanoid for Mobile Manipulation

@rsalakhu · 23 views · 2026-11-03

HumanoidMobile Manipulation

0:18

FLoRe Learns Force-Reactive Humanoid Loco-Manipulation

@SihangLi659492 · 22 views · 2026-11-03

HumanoidForce control

StereoPolicy: Improving Robotic Manipulation Policies via Stereo Perception

StereoPolicy learns task-conditioned binocular representations directly from synchronized stereo pairs instead of reconstructing explicit depth or point clouds. A Stereo Transformer alternates within-image and cross-image attention over shared 2D features, and the fused representation integrates with diffusion policies, pi0.5, and GR00T-N1.5. Experiments across simulation and real tabletop and bimanual mobile manipulation show consistent gains over RGB, RGB-D, point-cloud, and multi-view baselines, together with practical guidance on stereo baseline and working distance.

Evans Han, Yunfan Jiang, Yingke Wang · 62 views · May 11, 2026

Stereo visionRobot Manipulation

May 11, 2026

From RT-1 to MolmoAct 2: A Brief History of Vision-Language-Action Models

A complete history of VLAs from SayCan and RT-1 through RT-2, RT-X, Octo, OpenVLA, pi0, SmolVLA and MolmoAct 2, including the author's action-discretization experiment.

BLOG

Vision-language-action (VLA)Robot Learning

METATRON: A Promptable Whole-Body Execution Layer for Humanoids

Humanoid controllers often require task-specific training despite sharing the need for balance and whole-body coordination. We present Metatron, a promptable whole-body execution layer combining a generative base policy with a programmable skill layer. The base policy receives partial goals, collectively called a prompt, that can combine timed body-part targets, velocity commands, and style cues. It produces diverse whole-body motions consistent with these goals. Training combines motion capture, synthetic recovery trajectories, and retargeted scene-interaction motions. After training, the policy is frozen. Stateful programs called skills express task intent as prompts at each control step, drawing rough geometric sketches that the policy turns into coordinated movement. Skills are ordinary code, refined by coding agents that analyze motion data and fit compact rules without task-specific policy training. On the physical Unitree G1, one frozen checkpoint supports walking, recovery, teleoperated object pickup, and sitting on furniture. Comparative evaluations show higher reaching success and push robustness than the tested reference-tracking pipelines. The same recipe produces climbing and acrobatics on a simulated character. Metatron enables coding agents to turn learned motor capabilities into new skills through code, without further policy training.

Chen Tessler, Yifeng Jiang, Alec Otto · 316 views · Oct 27, 2026

HumanoidWhole-Body Control

Oct 27, 2026

0:58

I3L Turns Human Takeovers Into Better Robot Policies

@robotsdigest · 85 views · 2026-10-31

Imitation LearningInteractive Learning

EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data

EgoScale is a human-to-dexterous-manipulation transfer framework. A VLA policy is pretrained on 20,854 hours of action-labeled egocentric human video, revealing a log-linear relationship between data scale and validation loss. Lightweight aligned human-robot mid-training then grounds the learned motor prior in robot control, improving average success by 54% over a no-pretraining baseline on a 22-DoF dexterous hand and transferring to lower-DoF hands.

Ruijie Zheng, Dantong Niu, Yuqi Xie · 148 views · Feb 18, 2026

Dexterous ManipulationVLA

Feb 18, 2026

4:02

FACT: Learning Dexterous Manipulation from Video and Force

@RavenHuang4 · 114 views · 2026-10-28

FACTDexterous Manipulation

PACE: Temporal-Role Tactile Conditioning for Chunk-Based Insertion Policies

Contact-rich insertion requires a robot to combine visual alignment with tactile adjustment. Existing visuotactile chunk-based policies often collapse tactile feedback into one generation-time pathway, so persistent contact state, abrupt contact events, and the future contact consequences of an action are modeled implicitly in a single representation. PACE assigns one tactile stream three temporal roles. An exponential-smoothed contact state modulates action-chunk generation through layer-wise feature conditioning. A short-window tactile difference forms an event cue and drives a gated residual correction over only the first steps of the chunk. A training-only head predicts future tactile increments from decoder states as an auxiliary representation-learning objective. This preserves chunk-level action continuity while giving slowly varying contact phases and fast collision or jamming events separate pathways. Across five real insertion tasks, PACE reaches 90.7 percent average success, more than 44 percentage points above vision-only and single-path tactile-fusion baselines. It also averages 88.7 percent on six ManipulationNet geometry and clearance settings, including 0.1 mm clearance. Ablations show complementary contributions from Phase, Event, and Pred., while cross-signal and cross-architecture tests extend the principle to force/torque, joint current, and diffusion-policy backbones.

Zhiyuan Chen, Shixiong Xu, Shuai Chen · 243 views · Oct 12, 2026

Tactile SensingRobot Manipulation

Oct 12, 2026

From Video to Vision: Learning Generalizable Humanoid Carrying from Human Motion

Most video-driven humanoid imitation policies still need an upper-level module to generate reference trajectories at deployment, and the resulting controllers often depend on global position in a motion-capture studio. MOVER replaces that dependency with a three-stage teacher-assistant-student pipeline. It first recovers human motion from 100 videos with GENMO, retargets it to a 29-DoF Tiangong 3.0 humanoid through GMR, and augments object geometry, scale, and pose to obtain 4,000 reference trajectories. A teacher policy learns strict motion and object tracking; an assistant relaxes global body-position tracking to improve placement generalization; and a DAgger-PPO student removes proprioceptive position and reference-joint-position inputs. At deployment, a head-mounted RGB-D camera and an object tag provide relative object pose, while QuatNet predicts the reference anchor orientation from object pose, joint state, and target location. The paper evaluates four object placements and two shape-size configurations in simulation, then demonstrates carrying on the physical robot without motion capture. Remaining limitations include limited demonstration diversity, imperfect target localization without global proprioception, and a small real-world trial count.

Ziluo Ding · 252 views · Oct 10, 2026

HumanoidObject Carrying

Oct 10, 2026

DONIC: Generalist Dexterous Bimanual Tracking Control from Human Demonstrations

Despite recent progress in dexterous reference tracking, existing models focus primarily on single-hand manipulation and a limited range of interactions, leaving the bimanual dexterous behaviors common in real life unaddressed. The scarcity of contact-rich bimanual manipulation data and the increased complexity of coordinating contacts make developing a generalist bimanual tracker particularly challenging. Therefore, we present 1) DONIC, the first generalist bimanual tracker jointly trained across operation categories and human-motion sources, together with 2) BiHOI, a multi-view dataset of 1,575 sequences spanning 229 objects and emphasizing sustained bimanual interaction, where reconstructed interactions are retargeted into robot references. DONIC combines proximal policy optimization with behavioral cloning in a feedback loop that recycles verified successful executions into training demonstrations. An object-centered per-hand reward maintains object-motion feedback while supervising each hand according to its reference contact role. Clip-Centered Advantages remove reference-dependent advantage offsets while preserving within-trajectory differences. Together, these designs enable one single policy to learn coordinated manipulation across heterogeneous references. Without finetuning, a single checkpoint achieves 86.79% success on unseen trajectories of training objects and 57.32% on held-out objects, surpassing the state-of-the-art bimanual tracking baselines by 77.36 and 45.73 percentage points.

Mingze Sun, Ziang Cao, Yuan Fei · 218 views · Oct 21, 2026

Dexterous ManipulationBimanual Manipulation

Oct 21, 2026

1:24

DONIC: One Policy, Two Dexterous Hands for Bimanual Tracking

@HuggingPapers · 151 views · 2026-10-22

Dexterous ManipulationBimanual Control

SimpleVLN: Towards Simple and Frontier Vision-Language Navigation

Vision-and-Language Navigation (VLN) requires embodied agents to ground language in visual observations and make long-horizon decisions. Existing methods often address these demands through task-specific architectures. As multimodal large language models (MLLMs) become increasingly capable, an important question remains: Have we truly unlocked the potential of native MLLMs for VLN before introducing task-specific architecture? We present SimpleVLN, an architecture-preserving framework developed through studies of navigation formulation, data construction, and optimization. At the formulation level, SimpleVLN combines single-turn decision inputs with fixed-budget global-local memory to efficiently use navigation history without unbounded context growth, while action chunks enable efficient output of multiple primitive actions per model invocation. For data construction, we combine expert demonstrations with policy-dependent DAgger experience and develop a scalable synthesis pipeline that generates large-scale tasks for reinforcement learning. At the optimization level, a progressive IL-DAgger-RL pipeline advances from action imitation to task-level outcome optimization. Comprehensive experiments validate the effectiveness of these three aspects. We instantiate SimpleVLN across multiple MLLM backbones and model sizes, demonstrating its generality. SimpleVLN achieves 76.7% and 80.1% success rates on the R2R-CE and RxR-CE splits, respectively, substantially outperforming existing methods. These results show that the navigation capabilities of native MLLMs can be effectively unlocked without task-specific architectural designs.

Lingyi Hong, Weiren Cai, Jiayu Chen · 264 views · Oct 16, 2026

Vision-language navigationEmbodied AI

Oct 16, 2026

Perceptive Humanoid Parkour: Chaining Dynamic Human Skills via Motion Matching

While recent advances in humanoid locomotion have achieved stable walking on varied terrains, capturing the agility and adaptivity of highly dynamic human motions remains an open challenge. In particular, agile parkour in complex environments demands not only low-level robustness, but also human-like motion expressiveness, long-horizon skill composition, and perception-driven decision-making. In this paper, we present Perceptive Humanoid Parkour (PHP), a modular framework that enables humanoid robots to autonomously perform long-horizon, vision-based parkour across challenging obstacle courses. Our approach first leverages motion matching, formulated as nearest-neighbor search in a feature space, to compose retargeted atomic human skills into long-horizon kinematic trajectories. This framework enables the flexible composition and smooth transition of complex skill chains while preserving the elegance and fluidity of dynamic human motions. Next, we train motion-tracking reinforcement learning (RL) expert policies for these composed motions, and distill them into a single depth-based, multi-skill student policy, using a combination of DAgger and RL. Crucially, the combination of perception and skill composition enables autonomous, context-aware decision-making: using only onboard depth sensing and a discrete 2D velocity command, the robot selects and executes whether to step over, climb onto, vault or roll off obstacles of varying geometries and heights. We validate our framework with extensive real-world experiments on a Unitree G1 humanoid robot, demonstrating highly dynamic parkour skills such as climbing tall obstacles up to 1.25m (96% robot height), as well as long-horizon multi-obstacle traversal with closed-loop adaptation to real-time obstacle perturbations.

Zhen Wu, Xiaoyu Huang, Lujie Yang · 227 views · Feb 17, 2026

HumanoidParkour

Feb 17, 2026

PriMoRetarget: Humanoid Motion Transfer Through Prioritized Motion Retargeting

Motion retargeting is a critical preprocessing step for transferring human motion data to humanoid robots. While existing methods are effective for single-character motions, interactions involving multiple characters and objects introduce additional contacts and spatial relationships that must be preserved. Retargeting is commonly formulated as a weighted multi-objective optimization problem, where conflicting objectives can require careful tuning of relative weights and can still compromise important motion characteristics. We introduce PriMoRetarget, a prioritized motion-retargeting framework that replaces relative objective weighting with an explicit priority hierarchy. Lower-priority updates are constrained to preserve higher-priority objectives, allowing critical characteristics such as foot placement and interaction contacts to take precedence over lower-priority pose objectives. The method is evaluated on four datasets spanning single-character motion, character-object manipulation, and character-character interactions. The resulting motions better preserve task-relevant interactions and provide reference trajectories that can be tracked more reliably than those produced by widely used retargeting methods.

Nicholas Ioannidis, Dun Yang, Max Smith · 269 views · Oct 8, 2026

HumanoidMotion retargeting

Oct 8, 2026

Juno: Taming Predictive Latents for Vision-Language-Action Models

Joint-embedding predictive architectures (JEPAs) predict masked or future observations in representation space, offering a natural source of predictive latents for vision-language-action (VLA) models. Yet making these latents useful across pretraining, policy learning, and deployment requires addressing three failures: mismatch with embodiment-specific control, interference with action learning, and teacher miscalibration under distribution shifts. We introduce Juno, a unified framework built around one action-conditioned JEPA that serves as a control-aligned representation backbone, a predictive teacher, and an adaptable dynamics model. During pretraining, we train it on embodiment-matched trajectories and use a dynamic CLS loss to transfer motion-weighted patch dynamics to a compact global state. During policy learning, we fuse current-frame JEPA patches into VLA perception and use a decoupled reasoning branch with separate transformation parameters to distill future latent states for action generation. During deployment, we adapt the world model on all observed transitions, including failed rollouts, freeze the adapted teacher, and re-align the policy on verified executions using LoRA adapters and a trainable action head, without expert corrections or task rewards. On SimplerEnv, Juno raises average success from 60.9% to 68.5% over Qwen3GR00T, the strongest baseline, and test-time adaptation further reaches 72.7%; on a real robot, it retains 70%–75% success under background, height, and object shifts where the base policy collapses to 0%.

Yuchen Zhu, Chenyi Xu, Yulin Zhang · 219 views · Oct 7, 2026

Vision-language-action (VLA)World Models

Oct 7, 2026

LoopWAM: Closed-Loop Generalist Policy Improvement from Deployment Rollouts

Scaling expert demonstrations alone does not enable generalist robot policies to improve from deployment. Real-world rollouts contain successes, suboptimal behavior, and failures: imitating every recorded action can reinforce poor behavior, while filtering out failures discards evidence of physical interactions and their consequences. We introduce LoopWAM, a closed-loop world-action model that learns directly from this full quality spectrum, using rollouts from both itself and other deployed policies. Source-dependent noise sampling unifies behavior learning and action-conditioned world modeling, while quality steering and quality-weighted supervision retain full video supervision across qualities and regulate action imitation. Video and action tokens interact throughout parallel denoising, allowing learned consequences to inform action generation directly. On RoboChallenge Table30-V2, three deployment-and-training rounds raise success from 23.67% to 43.33%, a 19.7-percentage-point gain, with a final score of 56.80. LoopWAM also achieves 94.5% average success on RoboTwin 2.0 and 49.5% overall success on RoboCasa365 without additional embodied pretraining, obtaining the best overall results among the compared methods across all three benchmarks. On challenging shirt folding task, two rounds of deployment-driven improvement after single-task post-training raise success from 13.20% to 55.56%. These results demonstrate closed-loop improvement from mixed-quality experience under both multi-task learning and single-task post-training. Project page: https://loop-wam.github.io/

Wenxuan Guo, Yingping Liang, Zhaoguo Wang · 363 views · Oct 13, 2026

World-action modelsSelf-Improvement

Oct 13, 2026

PanTact: Teaching Whole-Body Dexterity with a Wearable Contact Interface

PanTact treats contact, not joint trajectories, as the transfer currency from people to humanoids. Wearable patches are assembled magnetically from 44x44 mm FSR modules; a custom 50x50 mm PCB streams 16 channels at 1 kHz over a CRC-checked RS-485 daisy chain, one Vive tracker per patch maps sensor centers into the world frame, and contact states are binarized against a per-channel adaptive baseline with threshold max(0.35 beta, r_min). The policy observes body-masked 3D egocentric point clouds, both wrist images and proprioception, and at 10 Hz predicts an action chunk of H=16 steps containing a look-at point, EEF poses with gripper widths, and per-sensor contact goals (binary state plus 3D position). A soft-minimum proximity d(p,q) with T = 0.02 m hands each active contact to whichever robot surface is currently nearest, and active goals are offset by delta along the pressing direction to form virtual goals that generate pressing force, so the G1 holds objects without any tactile sensor on its body. Three control formulations trade off differently: training-free upper-body IK, WBIK whose dense reference is executed by SONIC, and a sparse contact tracker trained with PPO over 4096 simulation environments for 30k iterations (50 Hz, 29 joints, S=8 upper-body / S=16 whole-body contact slots). On a 29-DoF Unitree G1: Grasp-and-Embrace reaches 85% with upper-body IK and 80% with the sparse tracker against 0% for Humanoid UMI; Sweep-and-Stow reaches 90.63% end to end while Humanoid UMI++ scores 18.75% on each stage and 0% end to end; Carry-and-Push is 90% versus 40%. In simulation replay, adding contact goals lowers contact distance for all three controllers at almost no cost in EEF accuracy.

Xu, Xiaomeng, Zhang, Han, Ruan, Zhian · 563 views · Oct 8, 2026

Whole-Body ManipulationTactile Sensing

Oct 8, 2026

MobileVISTA: Generative Data Augmentation for Pose Generalization in Mobile Manipulation

Mobile manipulators such as humanoid robots are increasingly deployed in dynamic, unstructured environments to perform dexterous manipulation tasks. However, end-to-end manipulation policies trained to imitate demonstration data collected from a single robot pose are brittle: even centimeter-scale deviations in robot pose at deployment can drive ego-centric observations and end-effector trajectories out of the training distribution, leading to sharp drops in performance. We introduce MobileVISTA, a data generation framework that transforms demonstrations captured at canonical poses into diverse, pose-perturbed training data by jointly (1) augmenting egocentric visual observations and (2) retargeting actions to compensate for base pose changes. Unlike prior methods, which assume a camera rigidly mounted off the actuated chain or non-trivial articulated robot geometry largely out of frame, MobileVISTA targets compatibility with egocentric platforms (e.g., humanoids) where the camera is both influenced by and must observe the robot's kinematic chain as it moves. We study MobileVISTA in simulated tasks spanning humanoid and bimanual embodiments, and on a real Galaxea R1 Pro. We find policies trained on MobileVISTA-augmented data demonstrate improved robustness to previously out-of-distribution poses encountered at test time, without additional demonstration collection or a trained generative model. Additionally, we find MobileVISTA's benefit is largest on tested humanoids, where the camera rides the actuated chain and the robot fills much of the frame. Additional videos and appendix can be found on our website: https://mobilevista.github.io

Wistreich, Suzannah, Tian, Stephen, Huang, Isabella · 319 views · Oct 5, 2026

Mobile ManipulationData Augmentation

Oct 5, 2026

RoboPrompt: Intuitive Robot Policy Steering with Sparse Human Input

End-to-end robot policies trained through imitation learning remain constrained by limited data diversity, making reliable zero-shot deployment in real-world settings challenging. Shared-autonomy methods enable human correction through teleoperation, but specialized hardware and operator training hinder deployment at scale. Other approaches incorporate human guidance as additional policy inputs, often requiring architectural changes and dedicated training for steerability, which limits their applicability across policies. We present RoboPrompt, a general-purpose, lightweight robot policy steering system that enables users to guide policy behavior through intuitive, sparse inputs, including drawn traces, target points, and coarse directional instructions. RoboPrompt decouples human-intention translation from the underlying policy: a reusable module converts human guidance into action drafts, which are refined through the diffusion or flow-matching dynamics of the base policy. By controlling action generation in noise space, RoboPrompt balances human intent with the policy prior without modifying the base policy architecture or fine-tuning it for steerability. Experiments demonstrate effective steering across Diffusion Policy, π_0.5, and FastWAM. We further use steered rollouts for online policy improvement through DAgger. After 2-3 rounds of iteration, average success rates increase by 15.5% for π_0.5 across three tasks and by 21.3% across three policies(Diffusion Policy, π_0.5, FastWAM) on the Insert Bread task, while average human intervention counts decrease by 44.0% (2.86 to 1.60) and 81.9% (2.60 to 0.47), respectively.

Yanwen Zou, Chenyang Shi, Guoxuan Xu · 510 views · Oct 7, 2026

ManipulationImitation Learning

Oct 7, 2026

2:57

Workhorse: Humanoid Robots Learn Whole-Body Locomanipulation from Human Data

@robotsdigest · 588 views · 2026-10-07

HumanoidLocoManipulation

0:25

DynaRetarget: Learning Humanoid Robot Motions from Human Demonstrations

@spaceandtech_ · 662 views · 2026-10-02

HumanoidMotion retargeting

1:07

Uni-VLaT: adding whole-body touch to pretrained VLAs lifts humanoid loco-manipulation from 32% to 75%

@stepjamUK · 628 views · 2026-09-29

Uni-VLaTTactile Perception

2:17

Self-Adaptive VLA: learning from its own failed rollouts to fix itself

@icefox1104 · 849 views · 2026-09-28

VLASelf-Adaptive VLA

As an Amazon Associate, we earn from qualifying purchases.