RPL: Learning Robust Humanoid Perceptive Locomotion on Challenging Terrains

Three hard things at once: multi-direction, hard terrain, unknown payload

Perceptive humanoid locomotion has matured quickly on single-direction tasks: policies that only walk forward now handle slopes, staircases and stepping stones with real reliability. Change the question to "the same terrains in both directions, while carrying something of unknown weight" and most of that reliability evaporates. The failure is not in the controller, it is in the geometry of perception. A front camera pointed at the path behind you is useless when walking backward; footholds drift to the edge of the field of view when strafing; and a payload changes the whole-body dynamics without ever appearing in the observation vector. RPL is Amazon FAR (Frontier AI & Robotics)'s open-source answer to exactly this combination, with Carnegie Mellon, Stanford and UC Berkeley, published at CoRL 2026 (arXiv:2602.03002, Apache-2.0) on the 29-DoF Unitree G1.

The method is a two-stage framework. Stage one trains one expert policy per terrain type with privileged height-map observations, pushing locomotion and upper-body skills to their limits on the terrain each is best suited for. Stage two distills those experts into a single transformer policy that sees only depth cameras, with multiple cameras covering every direction the robot might walk. Two distillation-time techniques make multi-directional walking robust: depth feature scaling gated by the velocity command, and random side masking, which respectively handle asymmetric multi-view depth and terrain widths never seen in training. Underneath sits a multi-depth rendering system that ray-casts against both the robot's own moving meshes and the static terrain, running 5x faster than the quickest depth pipeline in existing simulators. On the real robot the result is bidirectional walking with a 2 kg payload over 20 degree slopes, staircases with 22, 25 and 30 cm treads, and 25x25 cm stepping stones separated by 60 cm gaps.

The two-stage pipeline: terrain experts, then multi-depth distillation

The repository layout mirrors the pipeline. rpl_training/ holds everything outside the Holosoma core: the loco-manipulation environment, manager terms, presets and launchers. rpl_training/data/models/ ships four teacher checkpoints (slopes, stairs_up, stairs_down, stepping_stones_columns), which distillation loads by default; you can point it at your own directory with the same per-terrain layout, or at one W&B checkpoint per terrain. rpl_sim2sim/ contains the MuJoCo course on the sim side and the depth-student wrapper for holosoma_inference on the policy side, plus exported ONNX students.

Two statements in the README frame how to read the code. This is a research release only: no long-term support is promised and RPL features will not flow back into Holosoma mainline. And training uses IsaacGym rather than IsaacSim because the IsaacSim API does not implement force perturbation yet, which is precisely the mechanism that produces payload adaptivity. Those two sentences tell you which parts of the repo are reusable engineering and which are one-off experiment configuration.

Stage one: multi-actor PPO with terrain-routed height-map teachers

The expert stage follows FALCON's multi-actor PPO. Lower and upper body are separate actors with their own policy and value networks sharing one environment step. Both are four-layer MLPs of 512/256/256/128 units with ELU activations; the lower-body actor and critic additionally consume a height-map perception observation while the upper-body actor sees proprioception only. The teachers switch to mixture-of-experts structures: a CNN height-map encoder with per-terrain expert heads for the lower body, and a terrain-routed MLP for the upper body, one expert per terrain, selected with EXPERT=stones|stairs|slopes to reproduce a given FALCON expert's final training stage. Parallelism follows FALCON's recipe: 4096 environments across 4 GPUs.

The terrain is a 10x20 tile heightfield at 0.05 m horizontal scale, mixed 0.2 slopes, 0.25 stairs up, 0.25 stairs down and 0.3 stepping-stone columns, with max slope 0.7, step widths of 0.30-0.35 m, stones of 0.25-0.40 m with column gaps of 0.05-0.7 m and pit depths of 0.4-2.5 m. The command space covers linear velocity, heading, waist yaw and pitch, base height (a -0.3 to 0 m offset on the 0.75 m default absolute height) and an upper-body reference pose drawn from the bundled ACCAD motion library, with FALCON-parity switches such as freezing the upper-body reference for a resample interval and resetting joints directly onto the motion. Yaw and heading commands are masked off on stepping stones so the policy is not asked to turn on sparse footholds.

Payload adaptivity is already present at this stage. The environment applies a random vertical force at the end effectors: -100 to 5 N in the distillation preset (-60 to 5 N for the experts), with the application point sampled along the hand-to-extension direction at a 0-1 ratio and resampled together with the commands, and the z component zeroed with 25% probability. The applied force also enters the policy as an observation scaled by 0.1, and a force-scale curriculum with 0.05 steps ramps the pressure up gradually. A 2 kg object is about 19.6 N, comfortably inside that distribution: the policy meets payloads on hardware having already trained against them.

The student: two depth cameras into a vision transformer

Distillation uses the multi-actor variant of DAgger to compress the four terrain teachers into one student. The lower-body student is a VisionTransformerEncoder: a CNN tokeniser turns the two cameras' depth images (2 channels, 27x48, kernel 4, stride 2, hidden channels 16/32/32) into tokens, which feed a transformer with embedding dimension 64, 4 heads, 4 blocks, feed-forward width 256 and positional encoding, followed by a 512/256/256/128 MLP head. The upper-body student is a plain MLP. The teacher sees the noise-free height map; the student sees noisy, delayed, occluded depth, and closing that gap is the entire point of the stage. The loss is MSE with Adam at learning rate 1e-3 for 20000 iterations over 8192 environments (1024 per GPU on 8 GPUs).

Depth generation follows FALCON exactly: ray-cast at 240x135 with a 101.41 degree horizontal field of view and clipping to 0.1-2.0 m, plus noise at 0.1x standard deviation and 5% dropout with value -2.5, camera pose randomisation of plus or minus 2.5 cm and 2.5-3 degrees, field-of-view randomisation of plus or minus 5 degrees, then bicubic resizing with antialiasing down to 48x27. The cameras run at 10 Hz into a 4-frame ring buffer read at delay index 2, so sensor latency is modelled explicitly rather than ignored. Both cameras mount on torso_link: the front at offset [0.135, 0.06, 0.02] with 75 degrees pitch, the back at [-0.125, -0.06, 0.02] with 75 degrees pitch plus 180 degrees yaw, which is the physical source of the bidirectional capability. Ray casting also intersects 22 robot body STL meshes, so hands and legs occlude the terrain correctly in the robot's own cameras.

The two robustification techniques: velocity gating and random side masking

Multi-directional walking makes depth observations asymmetric. Walking backward up a staircase, the front camera looks at ground already covered while the rear camera sees the steps. A policy that trusts both depth streams unconditionally gets misled by whichever camera cannot see the terrain in the current walking direction. The first technique, depth feature scaling by velocity command, multiplies the depth features by a gate computed from the linear velocity command, with k=4.0, threshold 0.2 and a hard cutoff below 0.1: standing still or creeping suppresses the depth features, walking briskly toward what a camera sees releases them. Implementation-wise it is just a standalone command observation group fed into the vision transformer's gating branch, so deployment costs nothing extra.

The second technique, random side masking, erases a 90x50 pixel box on one side of the depth image during training (the two sides differ by 30 pixels; 10% no mask, 45% small mask, 45% large mask), sampled once per step and shared across cameras. It applies to slopes, stairs up and stairs down but never to stepping stones. The motivation is blunt: real terrain widths are always a little narrower or wider than the training course, and a depth image with one side erased forces the policy to place feet using only the half it can see. The paper's ablation makes the division of labour explicit. Training losses are nearly identical across variants, but dropping random side masking collapses on unseen narrow terrain widths, dropping velocity gating collapses on asymmetric multi-view inputs (front camera on stairs, rear on stepping stones), and only the full combination walks stairs bidirectionally under unseen geometry and asymmetric perception.

Multi-depth rendering: ray-casting the robot against itself

The compute bottleneck of depth distillation is rendering, not the policy: every environment, every camera, every step needs a depth image, and the depth cameras in mainstream simulators either cannot intersect dynamic meshes (IsaacSim's Warp RayCasterCamera) or are too expensive to scale across cameras (IsaacGym PhysX depth, IsaacSim RTX TiledCamera). RPL's multi-depth pipeline ray-casts with Warp against both the static terrain mesh and 22 robot meshes that update with the pose, which is what lets camera count scale horizontally at 1024 parallel environments.

Pipeline	Dynamic meshes	1 camera (VRAM GB / s per iter)	2 cameras	4 cameras
IsaacGym PhysX	yes	16.8 / 35.6	22.5 / 70.1	33.9 / 146.5
IsaacSim RTX	yes	17.5 / 5.3	23.2 / 7.6	34.4 / 12.6
IsaacSim Warp	no	12.8 / 3.5	15.1 / 5.9	20.7 / 9.1
RPL	yes	13.3 / 1.3	14.6 / 1.5	17.3 / 1.9

These are the paper's numbers on a single L40S at 1024 environments and 240x135 depth resolution. Going from one to four cameras, RPL's iteration time rises from 1.3 s to 1.9 s while the fastest baseline, IsaacSim Warp without dynamic-mesh support, rises from 3.5 s to 9.1 s: a 5x gap, and RPL additionally models robot self-occlusion. VRAM stays on the lowest curve throughout (13.3-17.3 GB). The pipeline lives inside the Holosoma submodule and is RPL's main infrastructure-level contribution.

How many cameras does multi-directional walking need

The relationship between camera count and walking direction is measured systematically. Under bidirectional locomotion (forward and backward only) and omnidirectional locomotion (adding lateral strafing), the paper compares 1 downward camera, 2 front-and-back cameras and 4 cameras covering all planar directions, reporting achieved terrain difficulty level out of 6:

Task	Camera config	Slopes	Stairs up	Stairs down	Stepping stones
Bidirectional (expert ceiling)	privileged height map	6.0	6.0	6.0	6.0
Bidirectional	1 (down)	6.0	6.0	5.9	5.1
Bidirectional	2 (front+back)	6.0	6.0	6.0	6.0
Omnidirectional	1 (down)	6.0	6.0	5.9	3.0
Omnidirectional	2 (front+back)	6.0	6.0	5.9	4.5
Omnidirectional	4 (all directions)	6.0	6.0	6.0	4.6

The reading is instructive. Stepping stones are the only terrain that drops off, because sparse footholds with gaps up to 70 cm are strongly coupled to walking direction and a downward camera cannot see the line the next foot must land on; two front-and-back cameras restore bidirectional performance to the expert ceiling. Even four cameras reach only 4.6 under omnidirectional control, leaving strafing across stones as an open problem. RPL's hardware and distillation defaults use the two front-and-back cameras, the best value cell in this table.

Hardware evidence: 2 kg payloads, 20 degree slopes, 60 cm stone gaps

Hardware experiments run on an indoor course at Amazon FAR: a wooden track chaining slopes, staircases and stepping-stone platforms. The policy walks it forward and backward while carrying a 2 kg payload, covering 20 degree slopes, staircases with 22, 25 and 30 cm treads, and 25x25 cm stepping stones separated by 60 cm gaps. The same policy demonstrates whole-body teleoperation, tracking commanded base velocity and end-effector pose with base height and torso orientation adjustable during stance. Every hardware clip on the project site comes from this one policy (videos are labelled as sped up 2x); there is no per-terrain model swapping.

The payload experiment design deserves its own mention. In simulation the per-hand weight is swept from 0 kg to 2.5 kg while comparing the full method against variants without random side masking and without velocity gating. Training losses are nearly identical across the three; all separation appears in out-of-distribution settings. Training curves alone cannot certify robustness here, only out-of-distribution terrain and asymmetric perception can, which is the most honest methodological point in the project.

Sim2sim: a MuJoCo course and an ONNX student

Most of the behaviour is reproducible without hardware. The repository ships a MuJoCo sim2sim course where the sim and the policy run as two processes communicating over Unitree sdk2py (DDS), with depth images passed through shared memory. That structure is identical to the real deployment, so the robot's software stack is exercised in simulation first. The policy side loads an exported ONNX student (iteration 19999 exports ship in the repo, alongside a reference original model) with three observation groups: proprioceptive history of 5 frames updated every step, depth history of 2 frames sampled every 5 steps, and a single-frame linear velocity command, which is the input to the velocity gate. An Xbox controller is strongly recommended for driving it (keyboard works): press 9 in the MuJoCo window to release the elastic band holding the robot, start on the pad to launch the policy, R2 to toggle stand and walk.

Repository boundaries: research release, Holosoma submodule, FALCON lineage

Three boundaries matter when reading this repository. First, the core algorithms (Warp multi-depth cameras, multi-actor PPO and DAgger, the vision transformer and CNN modules, terrain generation) live in the thirdparty/holosoma submodule; cloning without --recurse-submodules leaves every preset in rpl_training/ unresolvable. Second, many configs carry "FALCON parity" comments marking them as resolved values from FAR's internal FALCON-Terrains experiments: reward term masks, stance resampling probabilities, force budgets, even details like evaluating resets with the previous episode length. Six parity test files in the repo are the regression guards for those resolutions. Third, this is a research release: issues or direct email are the support channel, and merging RPL features into Holosoma mainline is explicitly a separate effort. For anyone reproducing it, the path is clear: train experts on 4 GPUs or use the four shipped teachers, distill for 20000 iterations on 8 GPUs, export ONNX, run sim2sim, and only then decide whether hardware is next.