Skip to content
Tags

#Sim-to-Real (66)

CAP: Continuously Adaptive Perception-Blind Humanoid Locomotion via Learned Denoising

CAP: Continuously Adaptive Perception-Blind Humanoid Locomotion via Learned Denoising

Humanoid locomotion across complex terrain demands forward-looking exteroception to anticipate obstacles, yet this signal is unreliable in real-world deployment, failing partially and intermittently. Existing perceptive policies often assume that depth observations remain clean and in-distribution, while recent attempts to unify perceptive and blind control typically route or switch between separate sub-policies, leaving recoverable information in partially corrupted depth unexploited. We instead propose CAP, a single-stage humanoid locomotion policy that recovers this signal with a perceptive world-model encoder trained as a learned denoiser to reconstruct clean depth from a corrupted input, together with a co-active proprioceptive variational encoder that supplies depth-free body-state information. A coupled training recipe pairs a depth-noise curriculum on the world-model input with world-model feature dropout on the policy-facing latent, exposing the policy to failures across the entire perception-quality spectrum. In simulation, CAP matches or improves upon perceptive baselines when depth remains informative, and degrades more smoothly than a binary-switching baseline as perception worsens. On the Unitree G1, controlled trials and indoor-outdoor deployments demonstrate perception-robust locomotion under intermittent occlusion, real-sensor corruption, and outdoor depth artifacts.

Chen, Hongjin, Xu, Zijun, Ma, ShihaoSep 10, 2026
Humanoid LocomotionWorld ModelsDepth DenoisingSep 10, 2026
Show-Harness: Just a VLM Agent Can Play Robots

Show-Harness: Just a VLM Agent Can Play Robots

Foundation vision-language models (VLMs) exhibit broad intelligence about the world, yet translating this intelligence into robot control remains challenging. We present Show-Harness, an Embodied Harness that enables VLMs to "play" robots through a compact semantic interface linking intent to action. Show-Harness exposes discrete semantic action units that VLMs can naturally reason over, while embodiment-specific interpreters deterministically ground them into local robot actions, keeping the VLM directly responsible for fine-grained physical decisions. Through the same interface, Show-Harness demonstrates the feasibility of (1) directly unlocking closed-source frontier VLMs for zero-shot robot control, and (2) adapting small-scale open-source VLMs for low-cost deployment with just a few GPU-hours of fine-tuning. We further develop GUMI (GUI Manipulation Interface), which extends the same semantic action space to GUI-based demonstration collection, allowing humans and agents to "play" robots across embodiments without specialized teleoperation hardware. Extensive experiments show that Show-Harness-equipped VLM agents generalize robustly across tasks, embodiments, and environments, outperforming representative agentic and VLA paradigms. These results suggest that the right interface can unlock substantial embodied capability from foundation VLMs, without requiring additional model capacity or costly embodiment-specific pretraining.

Yanzhe Chen, Zechen Bai, Zhijun CaoSep 9, 2026
Vision-language modelsAgentic HarnessSemantic Action InterfaceSep 9, 2026
LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World

LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World

Training vision-language-action (VLA) policies for humanoid loco-manipulation is constrained by the high cost and complexity of collecting human teleoperation demonstrations. VLA policies fine-tuned in simulators have, until now, failed to transfer effectively in humanoid loco-manipulation tasks. We present LEGS (Loco-manipulation via Embodied Gaussian Splatting), a hybrid simulator that composites a mesh foreground (robot, objects, props) over a photorealistic 3D Gaussian Splatting (3DGS) background reconstructed from a handheld scene capture. LEGS uses a procedural motion-primitive generator to synthesize labeled demonstrations at scale without human teleoperation, and a deterministic two-stage color calibration to align the rendered 3DGS image to the robot's deployment camera. On a Unitree G1 humanoid robot, across three pick-and-place tasks of increasing whole-body difficulty and three VLA backbones (psi_0, pi_0.5, GR00T N1.6), a policy trained purely on LEGS data matches or exceeds one trained on human teleoperation demos on every experiment. It also outperforms a mesh-only simulation baseline that ablates the effect of the 3DGS background, showing that photorealistic rendering is a key enabler for synthetic data transfer. Humanoid motion is recorded independently of scene appearance in LEGS, allowing the same auto-generated demonstrations to be re-rendered under new backgrounds and object meshes–covering a new scene at more than 15x lower cost than teleoperation–to augment training data for robustness to scene variations. Under combined object-and-scene appearance shift, the policy trained on re-rendered LEGS-AUG data maintains task success while the baseline trained on teleoperation data fails entirely. Our project page is located at https://legsvla.github.io/.

Kim, Hojune, Chen, Timothy, Sun, JiankaiMay 31, 2026
HumanoidVLA3D Gaussian SplattingMay 31, 2026
FADA: Few-Shot Domain Adaptation via Dynamics Alignment for Humanoid Control

FADA: Few-Shot Domain Adaptation via Dynamics Alignment for Humanoid Control

High-precision humanoid control is limited by target-domain dynamics mismatch, where the same control objective can induce different realized motions under changes in terrain, payload, or actuator response. Existing methods either pursue zero-shot transfer through domain randomization or in-context adaptation without target-domain specialization, or require heavy adaptation pipelines that leverage target-domain data, such as model calibration, residual learning, or policy retraining. In this paper, we present FADA (Few-Shot Domain Adaptation via Dynamics Alignment), a three-stage Planner-Inverse Dynamics Model (Planner-IDM) framework for few-shot adaptation in humanoid control. FADA first trains an oracle policy with privileged information and then distills the oracle behavior into a deployable Planner-IDM student through DAgger. At deployment, FADA freezes the planner and finetunes only the IDM using approximately 2 minutes of target-domain rollouts with standard supervised learning. Rather than requiring optimal demonstrations or rewards, FADA uses the paired actions and observations that are observed during these rollouts as supervision, aligning the IDM's action generation with target-domain dynamics. Experiments show that FADA outperforms both in-context and end-to-end adaptation baselines, improving task performance under dynamics shifts and enabling real humanoid robots to execute diverse high-precision whole-body tasks. Implementation details and qualitative hardware rollout videos are available at https://lecar-lab.github.io/FADA-humanoid/.

Angchen Xie, Nikhil Sobanbabu, Ishayu ShikhareJun 26, 2026
HumanoidFew-Shot LearningDomain AdaptationJun 26, 2026
TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model

TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model

We study the problem of navigating cluttered indoor environments with a humanoid robot. Unlike conventional methods that model navigation as a 2D path planning problem, humanoid traversal in cluttered environments requires continuous geometry-aware whole-body adaptation, including coordinated arm placement, torso adjustment, and gait modulation for collision-free movement through complex 3D spaces. We introduce TANGO, the first whole-body vision-language navigation framework for language-conditioned humanoid traversal in cluttered environments. Given a natural-language instruction and egocentric RGB observations, TANGO directly predicts 29-DoF joint-space actions for downstream whole-body control. We train TANGO entirely in simulation by synthesizing diverse collision-free traversal behaviors via global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and RL-based tracking. This pipeline provides dynamically feasible action supervision for learning language-conditioned whole-body policies. In extensive simulation experiments, TANGO demonstrates state-of-the-art performance in vision-language navigation, while outperforming strong modular baselines in navigating challenging scenes requiring obstacle negotiation. Lastly, we deploy TANGO zero-shot on a Unitree G1 humanoid robot, and observe robust language-guided traversal in cluttered real-world scenes without training on any real-world navigation data.

Anqi Li, Yuxin Chen, Zhaobo LiSep 8, 2026
HumanoidVLANavigationSep 8, 2026
How to Build a Humanoid Robot: A Seven-Layer Constraint Chain from Joint Modules to Sim-to-Real

How to Build a Humanoid Robot: A Seven-Layer Constraint Chain from Joint Modules to Sim-to-Real

A humanoid robot is not an assembly of seven modules but a stack of physics equations that set each other's boundary conditions. This article computes the whole-machine stack layer by layer: joint motor modules (declared torque versus real quasi-static CoP demand — knee margins across three vendors converge to 2.23-2.42x while BHL's knee has only 1.51x) -> IMU (lever-arm pseudo-acceleration is 21,752x the sensor noise floor, so mounting position matters four orders of magnitude more than the datasheet) -> materials and structure (three BOM revisions of AgiBot's X1 as a load-path history: every part entering the closed-chain drivetrain upgraded to 7075-T6 / TC4 / 17-4PH) -> sensors (fix the observation space before the shopping list) -> battery and BMS ('all joints at peak simultaneously' is physically impossible: G1's 46,062 W against a 421 Wh pack is 109 C) -> software control and the CAN-level low-side boards (22 nodes at 500 Hz on one bus is 130.9% load, so it must be split into four) -> simulation training and sim-to-real: domain randomization, sim2sim, zero calibration (ATOM01's 2.093 rad waist-yaw assembly offset, the |q| < 1e-2 rad acceptance gate, and write_motor_flash() being a no-op in three of the four motor drivers), plus 10 of 13 real failure modes being hardware calibration rather than simulation fidelity. Every figure comes from programmatic parsing of the five machines' public model files, deployment and calibration source, plus official vendor specifications, and is recomputable.

BLOG

HumanoidJoint modulesIMUBMSSim-to-Real