Skip to content
Tags

#Reinforcement Learning (114)

CAP: Continuously Adaptive Perception-Blind Humanoid Locomotion via Learned Denoising

CAP: Continuously Adaptive Perception-Blind Humanoid Locomotion via Learned Denoising

Humanoid locomotion across complex terrain demands forward-looking exteroception to anticipate obstacles, yet this signal is unreliable in real-world deployment, failing partially and intermittently. Existing perceptive policies often assume that depth observations remain clean and in-distribution, while recent attempts to unify perceptive and blind control typically route or switch between separate sub-policies, leaving recoverable information in partially corrupted depth unexploited. We instead propose CAP, a single-stage humanoid locomotion policy that recovers this signal with a perceptive world-model encoder trained as a learned denoiser to reconstruct clean depth from a corrupted input, together with a co-active proprioceptive variational encoder that supplies depth-free body-state information. A coupled training recipe pairs a depth-noise curriculum on the world-model input with world-model feature dropout on the policy-facing latent, exposing the policy to failures across the entire perception-quality spectrum. In simulation, CAP matches or improves upon perceptive baselines when depth remains informative, and degrades more smoothly than a binary-switching baseline as perception worsens. On the Unitree G1, controlled trials and indoor-outdoor deployments demonstrate perception-robust locomotion under intermittent occlusion, real-sensor corruption, and outdoor depth artifacts.

Chen, Hongjin, Xu, Zijun, Ma, ShihaoSep 10, 2026
Humanoid LocomotionWorld ModelsDepth DenoisingSep 10, 2026
ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search

ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search

GRPO is the representative RL method for improving LLM reasoning, yet it only ever sees one sparse reward at the end of a whole trajectory: when the rewards inside a sampled group land close together, the group-relative advantage collapses into noise and the policy learns almost nothing. ReST-RL reconnects policy optimization and value-guided search into a single self-training pipeline. Stage one, ReST-GRPO, first filters out low-information prompts by reward standard deviation, then draws prefixes from each prompt's highest-reward trajectory under a discrete exponential distribution and uses them as fresh online-GRPO starting contexts. A selected prefix is context only; its suffix is re-sampled and optimized rather than imitated. Stage two, VM-MCTS, runs MCTS under the now-static policy to self-collect value targets and trains a value model that predicts expected terminal reward. At inference the same model both allocates the tree search through UCT and ranks completed candidates in a Best-of-N fashion, so search and verification share one state-value scale. On coding benchmarks including APPS, BigCodeBench and HumanEval, Qwen3-8B moves from 0.503 to 0.689 average. In matched policy-value controls, ReST-GRPO + VM-MCTS reaches 0.642 on APPS-500 while GRPO + VM-MCTS reaches only 0.538, so the stage-one distributional shift survives value learning. End-to-end accounting puts ReST-GRPO at 1,752 GPU-hours against 2,080 for GRPO, hitting a 9% gain in 71 hours instead of 207. A value model trained only on code trajectories also transfers to MATH, Omni-MATH and GPQA-Diamond without target-domain tuning.

Sining Zhoubian, Dan Zhang, Jie TangAug 27, 2025
Reinforcement LearningGRPOMonte Carlo Tree SearchAug 27, 2025
TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model

TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model

We study the problem of navigating cluttered indoor environments with a humanoid robot. Unlike conventional methods that model navigation as a 2D path planning problem, humanoid traversal in cluttered environments requires continuous geometry-aware whole-body adaptation, including coordinated arm placement, torso adjustment, and gait modulation for collision-free movement through complex 3D spaces. We introduce TANGO, the first whole-body vision-language navigation framework for language-conditioned humanoid traversal in cluttered environments. Given a natural-language instruction and egocentric RGB observations, TANGO directly predicts 29-DoF joint-space actions for downstream whole-body control. We train TANGO entirely in simulation by synthesizing diverse collision-free traversal behaviors via global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and RL-based tracking. This pipeline provides dynamically feasible action supervision for learning language-conditioned whole-body policies. In extensive simulation experiments, TANGO demonstrates state-of-the-art performance in vision-language navigation, while outperforming strong modular baselines in navigating challenging scenes requiring obstacle negotiation. Lastly, we deploy TANGO zero-shot on a Unitree G1 humanoid robot, and observe robust language-guided traversal in cluttered real-world scenes without training on any real-world navigation data.

Anqi Li, Yuxin Chen, Zhaobo LiSep 8, 2026
HumanoidVLANavigationSep 8, 2026
Towards Adaptable Humanoid Control via Adaptive Motion Tracking

Towards Adaptable Humanoid Control via Adaptive Motion Tracking

Humanoid robots are envisioned to adapt demonstrated motions to diverse real-world conditions while accurately preserving motion patterns. Existing motion prior approaches enable well adaptability with a few motions but often sacrifice imitation accuracy, whereas motion-tracking methods achieve accurate imitation yet require many training motions and a test-time target motion to adapt. To combine their strengths, we introduce AdaMimic, a novel motion tracking algorithm that enables adaptable humanoid control from a single reference motion. To reduce data dependence while ensuring adaptability, our method first creates an augmented dataset by sparsifying the single reference motion into keyframes and applying light editing with minimal physical assumptions. A policy is then initialized by tracking these sparse keyframes to generate dense intermediate motions, and adapters are subsequently trained to adjust tracking speed and refine low-level actions based on the adjustment, enabling flexible time warping that further improves imitation accuracy and adaptability. We validate these significant improvements in our approach in both simulation and the real-world Unitree G1 humanoid robot in multiple tasks across a wide range of adaptation conditions. Videos and code are available at https://taohuang13.github.io/adamimic.github.io/.

Huang, Tao, Wang, Huayi, Ren, JunliOct 16, 2025
HumanoidMotion TrackingReinforcement LearningOct 16, 2025