Papers

ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation

ScaleNav studies language-conditioned local navigation, where a robot must reach a text-specified target that is visible in its initial observation and stop near the described location. The method starts from action-free Internet egocentric videos, learns how observations evolve as a camera approaches an instructed target, aligns the resulting visual priors with executable action chunks using action-labeled simulation, and then jointly fine-tunes video and action prediction. The authors report 86.5% success on a single-view Short-Horizon OVON adaptation, 94.9% on HabitatGS-LN, and 83.3% overall success across 60 real-robot trials on a Unitree Go2 in three indoor environments.

Local NavigationVision-language-action (VLA)

Yihui Yang, Yuanhong Yu, Jiawei Wang · 131 views · Oct 19, 2026

Learning Multi-Phase Robot Loco-Manipulation via Phase-Conditioned Policy Optimization

Complex whole-body loco-manipulation requires executing diverse maneuvers, such as crouching, grasping, and locomotion, within a single long-horizon task. While performing simple pick-and-place behaviors is well-established, scaling to complex, long-horizon tasks remains difficult. Currently, this challenge is addressed by relying on human motion data and correspondingly humanoid platforms, which limits platform independence and introduces substantial data processing overhead. Instead, we extend proximal policy optimization (PPO) to train a single policy for multi-phase tasks across diverse robotic hardware, leveraging prior task structure to shape behavior. Our method adapts PPO using a phase-conditioned critic, phase-gated reward functions, and a multi-phase reset curriculum to guide exploration. To support varied hardware, we employ a decoupled hierarchical control architecture that separates lower-level balancing dynamics from high-level phase execution. We demonstrate multi-stage loco-manipulation skills on hardware on both the Unitree G1 and a Boston Dynamics Spot equipped with a manipulator arm, and validate our design choices through ablations. Our approach enables the acquisition of complex loco-manipulation behaviors purely through environment interaction while directly incorporating high-level task structure into policy learning.

Whole-Body Loco-ManipulationPhase-Conditioned PPO

Simone Manni, Annika Wollschläger, Dario Bellicoso · 291 views · Oct 26, 2026

SuperNav: An Agentic Navigation System for Any Task in Any Scene

General-purpose service robots need navigation systems that can handle diverse human requests in unfamiliar environments, combining task generality with scene generality. Some existing methods fine-tune multimodal large language models (MLLMs) to predict navigation actions, making their behavior dependent on the coverage of navigation training data and potentially limiting generalization to new requests and environments. Our key insight is to let the MLLM focus on interpreting requests, understanding scenes, and making decisions while preserving its general-purpose capabilities and delegating motion execution to navigation tools. To realize this idea, we introduce SuperNav, which equips a pretrained MLLM with a specialized agent harness without navigation-specific fine-tuning of the MLLM. Our harness supports these decisions with Navigation Skills, agent-oriented Tools for physical interaction, and task-progress and context management. A unified visual-point interface connects decision-making to motion by allowing the model to specify destinations directly in images and revise its decisions from execution feedback. Together, these components support sustained navigation across different task requirements and environments. SuperNav outperforms four evaluated baselines on instance-level, multi-object, and demand-driven tasks. Category-level evaluation on HM3D and deployment on a real quadruped robot further demonstrate its applicability across environments. Project Page: https://zju3dv.github.io/SuperNav/

Object NavigationEmbodied Agent

Jinkai Zhang, Jingyi Xu, Yuanhong Yu · 463 views · Oct 8, 2026

Agile and Generalized Legged Locomotion via Attention-Based Neural Map Encoding

Achieving agile and generalized legged locomotion across terrains requires tight integration of perception and control, especially under occlusions and sparse footholds. Existing methods have demonstrated agility on parkour courses but often rely on end-to-end sensorimotor models with limited generalization and interpretability. By contrast, methods targeting generalized locomotion typically exhibit limited agility and struggle with visual occlusions. We introduce a unified reinforcement learning (RL) framework for agile and generalized locomotion that incorporates a novel attention-based map encoder in the control policy. This encoder extracts local and global mapping features and uses attention mechanisms to focus on salient regions, producing an interpretable and generalized embedding for RL-based control. We further propose a learning-based mapping pipeline that provides fast, uncertainty-aware terrain representations robust to noise and occlusions, serving as policy inputs. It uses neural networks to convert depth observations into local elevations with uncertainties, and fuses them with odometry. The pipeline also integrates with parallel simulation so that we can train controllers with online mapping, aiding sim-to-real transfer. We validate our framework with the proposed mapping pipeline on a quadruped and a biped robot, and the resulting controllers demonstrate strong agility and generalization to unseen terrains in simulation and in real-world experiments.

Legged LocomotionReinforcement Learning

Chong Zhang, Victor Klemm, Fan Yang · 517 views · Jan 13, 2026

UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation

Mobile manipulation requires a robot to navigate to a target object or receptacle and then perform intended manipulation. However, reaching the vicinity of the target does not guarantee a manipulation-ready base pose, a problem known as last-mile navigation. Prior methods for last-mile navigation either rely on manual pose annotation or task-specific training, limiting their scalability to open-vocabulary settings with fine-grained spatial constraints. We propose UniLM-Nav, a unified framework for zero-shot open-vocabulary last-mile navigation. UniLM-Nav decomposes last-mile navigation into view selection, task-conditioned affordance grounding, and geometry-aware base-pose reasoning, all resolved with a shared multimodal large language model (MLLM) backend. Specifically, UniLM-Nav first selects a reference view that best captures the target object or receptacle from recently collected observations. It then grounds task-relevant affordance point in the selected view and lifts the result into the robot-centric coordinate frame. Finally, conditioned on the grounded affordance, task context, and robot geometry, it infers a manipulation-ready base pose for the robot. We evaluate UniLM-Nav on the OVMM benchmark, where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points. Analyses show that the components of our method are crucial to final performance, and that the choice of MLLM also has a substantial effect. We further deploy UniLM-Nav on a Unitree B2 quadruped robot with a 6-DoF Unitree Z1 manipulator, validating its applicability to real-world mobile manipulation tasks.

Mobile ManipulationLast-Mile Navigation

Zhuofan Zhang, Tianxu Wang, Guoxi Zhang · 404 views · Jul 7, 2026

Wheels + Legs + Hierarchical RL: Behind ETH Zurich's 8.3 km Autonomous Delivery

A complete wheeled-legged system from ETH Zurich's Robotic Systems Lab (Science Robotics): the low-level locomotion policy, trained by privileged learning (PPO teacher, GRU student imitation) with CPG gait primitives and the state estimator removed, switches autonomously between driving, trotting, and knee-crawling, peaking at 5.0 m/s on flat ground; the high-level navigation policy outputs bounded Beta-distributed velocity commands at 10 Hz, consuming the LLC hidden state and a visited-position buffer for exploration. Trained in WFC procedurally generated navigation worlds with dynamic obstacles, it completed an 8.3 km delivery across Zurich's Glattpark with minimal intervention; in point-goal comparison it was the only method with zero collisions, planned in 0.34 ms (~3 orders of magnitude faster), and achieved a mechanical COT of 0.16 (53% lower, 3x faster than ANYmal).

Wheeled-Leggedhierarchical RL

Lee, Joonho, Bjelonic, Marko, Reske, Alexander · 618 views · May 3, 2024

GeoFlow-SLAM: A Robust Tightly-Coupled RGBD-Inertial and Legged Odometry Fusion SLAM for Dynamic Legged Robotics

GeoFlow-SLAM (Horizon Robotics, IROS 2025) targets three stacked failure modes of quadruped localization: motion blur and IMU distortion from fast locomotion, and feature scarcity in texture-less scenes. Three interlocking contributions: dual-stream optical-flow tracking aided by prior map points and poses (a 3D-to-2D reprojection stream plus a 2D-to-2D fallback stream); a pose-initialization priority chain degrading from leg odometry to IMU, PnP, GICP, and finally a constant-velocity model; and a tightly-coupled optimization framework placing depth-to-map point-to-plane and GICP relative-pose residuals alongside visual reprojection and IMU pre-integration in one factor graph. On four real Go2 sequences (office, texture-less corridor, underground parking, open plaza) it is the only method that succeeds on Seq2-4, with 88% improvement over VINS-RGBD on Seq1. ScanNet mean ATE 0.0779 m (~70% better than ORB-SLAM3), TUM 0.0129 m; global plane-feature maps are still built in the structure-less texture-less TUM extremes. Code and data are open source (HorizonRobotics/GeoFlowSlam).

SLAMLocalization

Xiao, Tingyang, Zhou, Xiaolin, Liu, Liu · 700 views · Mar 18, 2025

Contact-Guided Exploration for Non-Prehensile Locomanipulation with Multi-Critic RL

The paper turns 25 grasp-algorithm candidates into a retractable exploration objective, separates task, exploration, and regularization signals with three value heads, and schedules their advantages during training. The full method reaches 94.1% simulation success with 4.4% tipover and transports unseen IKEA objects in 40/58 zero-shot hardware trials.

Multi-Critic RLIsaac Lab

Simone Tolomei, Mayank Mittal, Franco Angelini · 917 views · Aug 28, 2026

ABot-N1: Toward a General Visual Language Navigation Foundation Model

ABot-N1 is a general visual-language navigation foundation model built on a slow-fast dual-system architecture: a 4B slow VLM reasoner emits explicit chain-of-thought plus dual pixel goals (Target Pixel + Affordance Pixel), while a 2B fast action expert decodes continuous SE(2) waypoints via QFormer action queries. The unified pixel-goal interface covers five tasks — point-goal, object-goal, POI-goal, instruction-following and person-following — in a single 30M-sample multi-task checkpoint, further aligned by GRPO post-training with format/target/safety rewards. Two closed-loop benchmarks are released (ABotN-PointBench and ABotN-POIBench). ABot-N1 sets new SOTA on all five benchmarks, boosting POI entrance arrival to 77.3% (+35.0 pp) and reaching 92.9%/95.4% outdoor/indoor point-goal SR, with full deployment on the TuTu quadruped running on a Jetson AGX Orin.

VLNNavigation Foundation Model

Ruiyan Gong, Yingnan Guo, Junjun Hu · 868 views · Jul 11, 2026

DECOWAM: Decoupled Whole-Body World-Action Model for Legged Mobile Manipulation

DECOWAM adapts a frozen FastWAM video-action backbone to legged mobile manipulation via decoupled interfaces — an action-equivalent future bottleneck, adversarial base/arm factorization, and ego-motion-aware video conditioning — cutting Stage-2 trainable parameters 232x while leading real-robot deployment at 58.2% success.

VLAWorld Models

Siyuan Ma, Boshi Zhang, Yutian Zhang · 850 views · Aug 20, 2026

Motion Priors Reimagined: Adapting Flat-Terrain Skills for Complex Quadruped Mobility

Reinforcement learning (RL)-based motion imitation methods trained on demonstration data can effectively learn natural and expressive motions with minimal reward engineering but often struggle to generalize to novel environments. We address this by proposing a hierarchical RL framework in which a low-level policy is first pre-trained to imitate animal motions on flat ground, thereby establishing motion priors. A subsequent high-level, goal-conditioned policy then builds on these priors, learning residual corrections that enable perceptive locomotion, local obstacle avoidance, and goal-directed navigation across diverse and rugged terrains. Simulation experiments illustrate the effectiveness of learned residuals in adapting to progressively challenging uneven terrains while still preserving the locomotion characteristics provided by the motion priors. Furthermore, our results demonstrate improvements in motion regularization over baseline models trained without motion priors under similar reward setups. Real-world experiments with an ANYmal-D quadruped robot confirm our policy's capability to generalize animal-like locomotion skills to complex terrains, demonstrating smooth and efficient locomotion and local navigation performance amidst challenging terrains with obstacles.

ANYmalQuadruped robots

Zewei Zhang, Chenhao Li, Takahiro Miki · 637 views · May 21, 2025

DELTA: Deformable Elevation-Based Local Terrain Attention Encoder for Sparse-Terrain Quadrupedal Locomotion

Stable quadrupedal locomotion on sparse terrain requires selecting state-relevant terrain evidence for precise foot placement. Model-based foothold planners provide precise foothold selection but rely heavily on explicit model assumptions. Recent attention-based map encoding (AME) studies show that end-to-end reinforcement learning (RL) can learn implicit foothold guidance. However, the computational cost of dense AME encoding grows with map resolution, limiting its scalability to fine-grained sparse terrain. We propose DELTA, a Deformable Elevation-Based Local Terrain Attention encoder. DELTA predicts state-conditioned sampling locations, forms terrain evidence tokens from adaptive local elevation patches, and attends only to a fixed-size token set. With fixed sampling and patch settings, DELTA's encoder cost is independent of map resolution. Experiments show that DELTA achieves final traversal performance comparable to AME at the standard resolution while improving learning efficiency. This fixed encoder cost enables the use of higher-resolution terrain maps, improving traversal on fine-grained sparse terrain. DELTA also demonstrates strong generalization to unseen mixed evaluation courses composed of continuous and discrete terrain elements. Beyond simulation, DELTA demonstrates successful sim-to-real transfer on RAIBO2. Analysis of the learned sampling offsets and attention weights shows that DELTA samples steppable regions and attends to terrain evidence relevant to future touchdowns without foothold labels or attention supervision.

Sim-to-RealQuadruped robots

Sanghyun Park, Moonkyu Jung, Jemin Hwangbo · 612 views · Aug 22, 2026

As an Amazon Associate, we earn from qualifying purchases.