Humanoid Mobility via Attention-based Contacts

Preliminaries: Contact Plans and Attention over Terrain

The planner models nine contact-active bodies: the two feet, two knees, two hands, two elbows, and pelvis. For every body part it predicts a target position and a binary occupancy value. The position is expressed in a gravity-aligned frame centered at the robot base and rotated by the base yaw, making the representation invariant to the robot's absolute world position and heading. A body with no expected contact in the planning horizon is assigned a zero target.

Terrain is represented as a 17 by 11 grid of three-dimensional points around the robot. Instead of compressing the map into a single vector before reasoning about contact, the planner keeps the points as attention keys and values. Proprioception and the goal form a query that selects relevant terrain regions. This is a useful inductive bias for locomotion: the relation between a body part and a candidate support point remains explicit, and the predicted location can be constrained to an observed surface.

The tracker uses an asymmetric actor-critic design. The actor receives only observations available on the real robot: proprioceptive history, a local height scan, contact targets for nine bodies, and a goal command. Training critics can access privileged simulation state. Stages 1 and 2 use a reference-following critic. Stage 3 adds a separate goal-reaching critic, and advantages for the two regimes are normalized independently so that their different reward scales do not interfere.

Method

Contact-Aware Retargeting and the Whole-Body Tracker

Human motions from different sources are first converted into a common floating-base and Cartesian landmark representation. The system then performs constrained nonlinear inverse kinematics for the Unitree G1. Position objectives track hips, knees, ankles, toes, shoulders, elbows, hands, chest, and head, while orientation objectives align limb segments with the human skeleton. Joint limits, foot-ground non-penetration, minimum-distance constraints against boxes, stairs, and ladder rungs, and temporal smoothness make the retargeted motion physically plausible.

In parallel, the retargeting pipeline detects contacts involving the pelvis, knees, feet, elbows, and hands. It records the active body part, contact onset, and world-frame location. The resulting dataset therefore contains both robot-compatible kinematics and explicit body-environment interaction labels.

The Stage-1 tracker is a three-layer multilayer perceptron with hidden widths 512, 256, and 128 and ELU activations. Its 468 input features consist of a five-step proprioceptive history, a 9 by 7 height scan, nine contact coordinates, and a three-dimensional goal command. Invalid contact targets are encoded as zero vectors. The network predicts 23 joint-position offsets at 50 Hz. These offsets are scaled by 0.25 and added to the default pose to produce targets for joint-level PD control at 200 Hz.

The tracker is trained on simulations whose terrain is spatially aligned with each reference clip. Curriculum state initialization starts near the end of a clip and gradually expands the initial phase backward as tracking improves. The robot first learns to complete the final part of a demonstrated motion, then learns to execute the whole sequence from earlier states. This produces a controller that has internalized whole-body coordination before a contact planner is introduced.

An Attention Planner That Selects Contacts on the Map

At each control step, the planner predicts the next contact for each of the nine bodies within a 1.5 second horizon, or 75 control steps at 50 Hz. It receives a five-step proprioceptive history, a bounded horizontal goal direction with relative heading, and 187 terrain points. The map spans a 17 by 11 local grid, and the goal changes both where the robot should go and which interaction strategy is appropriate.

A two-layer convolutional encoder with 5 by 5 kernels and zero padding processes the height map through 1, 16, and 48 channels. Per-point linear features and encoded local terrain features form the attention keys and values. Proprioception, the goal, and a 64-dimensional pooled terrain feature are fused into a 96-dimensional context. A learned embedding for each body passes this context through feature-wise linear modulation to create nine body-specific queries.

The attention module uses 96 dimensions, 32 heads, and one layer. It produces an affordance distribution over the 187 map points for each body. The decoder uses temperature-scaled normalized weights and retains the top k points. Quantitative experiments use k equal to one, so every commanded contact coincides with an observed terrain point. The complete attention planner has 786,993 parameters, compared with 833,322 for an MLP baseline that shares the interface, supervision, and terrain-point decoder.

Hindsight Contact Labels

Planner supervision comes from what the controller can actually execute. After an episode, the system scans the recorded trajectory backward to locate the next contact onset for each body. Vertical force above 1 N marks a contact candidate. Segments shorter than ten control steps, or 0.2 seconds, are discarded as transient contacts. The next valid contact within the 1.5 second horizon is transformed into the body-yaw frame of each source timestep and retained if it lies inside the local scan footprint.

For reference-following episodes, success requires starting within the first 10 percent of the clip and progressing to at least 90 percent of its duration. For goal-reaching episodes, success requires at some point bringing the robot within 0.5 m of the goal horizontally and 0.1 m vertically. Only successful trajectories enter the planner buffer. Goal-reaching samples are organized by terrain tile, with quality-based eviction and balanced sampling across terrain types and tiles.

Thin ladder rungs expose a discretization problem: the nearest horizontal grid cell may miss the support. The authors therefore convert each true contact into a soft three-dimensional Gaussian over map points. For body b and map point i, the target is

$$p^{*}_{b,i}=\frac{\exp\left(-\|\mathbf h_i-\mathbf p_b^*\|_2^2/(2\sigma_a^2)\right)}{\sum_{j=1}^{LW}\exp\left(-\|\mathbf h_j-\mathbf p_b^*\|_2^2/(2\sigma_a^2)\right)}$$

Here, $\mathbf h_i$ is a terrain point, $\mathbf p_b^*$ is the hindsight contact target, and $\sigma_a$ is 0.1 m. The equation assigns mass to nearby cells instead of placing all supervision on one cell. The planner outputs point weights $\alpha_{b,i}$ and an occupancy probability $\hat c_b$. Its objective combines masked cross-entropy for location and binary cross-entropy for occurrence:

$$\mathcal L_{\mathrm{planner}}=-\frac{\sum_{n,b}c_{n,b}^*\sum_i p_{n,b,i}^*\log\alpha_{n,b,i}}{\max(1,\sum_{n,b}c_{n,b}^*)}-\frac{1}{9N}\sum_{n,b}\left[c_{n,b}^*\log\hat c_{n,b}+(1-c_{n,b}^*)\log(1-\hat c_{n,b})\right]$$

The location term is averaged only over valid future contacts, while the occurrence term is averaged over every one of the nine bodies in every sample. This avoids diluting the spatial objective with bodies that are not expected to touch the environment during the horizon, while still forcing the planner to distinguish contact from non-contact.

Gradual Transfer from Reference Contacts to Predicted Contacts

Stage 2 introduces the planner on the same reference-anchored terrain and trains it jointly with the tracker. The tracker cannot switch abruptly from oracle contacts to imperfect planner outputs. Each environment chooses either the complete reference contact command or the complete predicted command at every control step, with the probability of prediction controlled by an annealing schedule. Contact coordinates are not interpolated between the two sources.

The input and reward schedules start at different times. A transfer function has the form

$$\eta_\xi(u)=\begin{cases}0,&u\le u_\xi\\\frac{u-u_\xi}{N_{\mathrm{ann}}},&u_\xi<u<u_\xi+N_{\mathrm{ann}}\\1,&u\ge u_\xi+N_{\mathrm{ann}}\end{cases}$$

where u is training progress, $u_\xi$ is the start of that schedule, and $N_{\mathrm{ann}}$ is its duration. The input schedule changes which contacts the actor sees. The later reward schedule reduces the three reference-contact terms and increases the planned-contact term. The actor therefore first encounters predicted targets while retaining reference guidance, then receives stronger incentives to realize those targets accurately.

The planned-contact reward uses a Gaussian kernel over occupancy-gated body errors:

$$r_{\mathrm{planned}}=\exp\left(-\frac{\sum_b m_b\|\mathbf p_b^{\mathrm{exec},B}-\hat{\mathbf p}_b\|^2}{\sigma_c^2\max(1,\sum_b m_b)}\right)$$

In this expression, $\mathbf p_b^{\mathrm{exec},B}$ is the executed contact in body-yaw coordinates, $\hat{\mathbf p}_b$ is the planner target, and $m_b$ marks a valid target. The denominator normalizes over active contacts and has a lower bound of one. Reward kernels use a scale of $\sigma_c=1$ in the corresponding SI units.

Stage 3: A Bootstrapping Loop on Procedural Terrain

The third stage introduces terrains that have no reference motion: platforms with randomized orientation, stepping stones, gaps, pallets of varying width, balance beams, climbable obstacles, and ladders with different rung spacing. The planner predicts contacts from the local map, proprioception, and goal. The tracker executes them. The controller is optimized with both goal rewards and contact-tracking rewards, and successful rollouts are returned to the planner buffer as new hindsight labels.

The goal reward combines task progress before reaching the target with a separate terminal or stand-at-goal condition. The progress form is

$$r_{\mathrm{move2goal}}=(1-g_{xy})g_v+g_{xy}g_z$$

where $g_{xy}$ indicates that the horizontal goal distance is below 0.5 m, $g_v$ checks forward speed against a terrain-dependent cap, and $g_z$ handles vertical progress for get-up terrain. A separate stand-at-goal term requires the horizontal, vertical, and heading errors to be small and penalizes missing foot contact, torso tilt, and joint deviation. The speed cap is 0.8 m/s on configured stair terrain and 2.0 m/s elsewhere.

The goal-reaching critic receives the goal command, horizontal distance, normalized episode time, vertical offset, and predicted contacts. The reference-tracking critic receives privileged reference kinematics, reference contact positions, and reference contact flags. A dispatcher routes each simulated environment to the appropriate critic. Separate value networks allow reference imitation and goal-reaching exploration to coexist without sharing an advantage scale.

The result is a learning loop rather than a one-way training pipeline. The planner proposes contacts, the tracker attempts them, task success filters trajectories, hindsight labels convert successful interactions into supervision, and the improved planner changes the targets seen in subsequent controller training:

flowchart LR
  A["Human motion corpus"] --> B["Contact-aware retargeting
kinematics plus contact labels"]
  B --> C["Stage 1
contact-conditioned tracker"]
  C --> D["Stage 2
attention contact planner"]
  D --> E["Annealed reference-to-plan transfer"]
  E --> F["Stage 3
goal-reaching RL fine-tuning"]
  F --> G{"Successful rollout"}
  G -- "Hindsight labels" --> D
  F --> H["Deployment policy"]
  I["LiDAR neural map"] --> D
  J["Proprioception and goal"] --> D
  H --> K["Feet knees hands elbows pelvis"]
  K --> H
      

LiDAR Neural Mapping for Thin Supports

The planner needs a local observation that preserves ladder rungs and other thin structures. The paper adapts depth-based neural mapping to a head-mounted Livox MID-360 LiDAR. Each scan is projected into local grid cells, with the highest point retained. A convolutional network then selects one of three modes per cell: keep the measured height, use a neighborhood average, or discard the value and leave the cell empty. This mode classification is designed to tolerate arbitrary scan sparsity and point noise.

The disagreement among the three mode logits supplies an uncertainty score. Local predictions are fused into a global map using odometry. The global update includes a prior for overhanging structures. If a new prediction shares a grid cell with a confident existing height and lies more than 0.15 m below it, the update is rejected unless its elevation angle is above negative 40 degrees. This prevents a later scan of the floor beneath a ladder rung from erasing the earlier observation of the rung.

The local grid covers 6 m by 3 m in the gravity-aligned LiDAR frame at 3 cm resolution. It runs on the CPU of a Jetson Orin NX and updates within 30 ms for each 10 Hz scan containing 20,000 points. Because the LiDAR is head mounted, occlusion is less severe than on platforms with body-mounted depth sensors. High-uncertainty regions are inpainted with the neighborhood minimum and passed through a median filter before being used by the policy.

Experimental Evidence

The simulation study separates three terrain groups. Reference-anchored terrain is paired with training motions and tests retention. Goal-reaching terrain is procedurally generated for Stage 3 and provides a zero-shot test for Stage-2 systems. Unseen terrain introduces new obstacle compositions, task-irrelevant distractors, and surface roughness. The first two groups each contain four categories defined by support density and the body parts needed for support: dense foot-only, sparse foot-only, dense whole-body, and sparse whole-body.

Category-level results aggregate 3,000 episodes per method per category over three evaluation seeds. Terrain-specific comparisons use 1,000 episodes per method and axis, with the ladder axis pooling four variants. Success usually means reaching within 0.5 m of the active goal horizontally; sitting, standing, and get-up transitions instead require vertical error below 0.1 m. Curated composite courses also require the target height, passage through the prescribed route, avoidance of the pit, and no departure from the traversal lane.

The study reports Wilson 95 percent confidence intervals for success and episode-cluster-robust intervals for contact error. Sensor noise, physics randomization, and action delay are disabled in these evaluations so that differences are associated with terrain-dependent behavior rather than random deployment conditions.

The first ablation asks whether contact commands alone are enough for whole-body tracking. On reference-anchored terrain, the contact-only tracker achieves at least 98.5 percent success in every category. Adding reference root and joint kinematics does not produce a consistent improvement. This directly supports the representation argument: in the collected motion distribution, the controller does not need a full kinematic input to execute the skill once contacts are specified.

The second ablation compares the attention planner with a flattened-map MLP. The architectures have similar scale, 786,993 versus 833,322 parameters, and share the contact interface and loss. After Stage-3 fine-tuning, attention is better across goal-reaching categories and shows clear gains on unseen composition and distractors. The weakest baseline conditions are also the most semantically demanding: sparse whole-body supports require exact placement, while distractors require rejecting visually salient but irrelevant geometry.

The third ablation examines whether Stage-3 reinforcement learning actually expands the skill distribution. For the attention system, Stage 3 retains high success on reference-anchored terrain while improving success and contact accuracy on every goal-reaching and unseen category. The largest gain is on sparse whole-body terrain, where success rises by 54.8 percentage points. This is the strongest evidence that a planner trained only on demonstrated contacts cannot simply transfer unchanged to all constrained multi-limb supports.

The final quantitative comparison uses MGMT, which couples diffusion-based whole-body motion generation with a tracker conditioned on full reference kinematics. The proposed system has higher success in all eleven terrain categories. The qualitative comparison is especially informative: from the same robot state, the contact planner selects points on stepping stones and ladder rungs, whereas the generated MGMT trajectory can pass through unsupported regions. A full-body trajectory is not equivalent to an explicit promise that each critical limb will meet a valid support.

Eight curated composite courses combine sparse footholds and whole-body obstacles. Each method and course is evaluated for 1,000 episodes. The proposed system averages 92.5 percent success and MGMT averages 58.8 percent. The proposed system also wins every one of the eight courses. These courses test more than individual terrain recognition because the robot must connect walking, climbing, crawling, and rising behaviors in a single traversal.

The real-robot platform is a Unitree G1 with a Livox MID-360 and a Jetson Orin NX. DLIO estimates the root state and generates a deskewed point cloud; neural mapping creates the height observation; the policy runs onboard. The demonstrations include stairs, ladders, boxes, sitting, standing, getting up, sparse terrain, and composite routes. A single learned policy changes behavior from the same environment when the goal position or heading changes, which argues against a simple terrain-class to fixed-skill lookup.

The recovery examples show that the controller is not limited to replaying successful reference clips. It retries a failed stand-up, recovers after an unexpected fall, continues after a perception-induced tip-over, and repeatedly attempts to escape a stuck ladder configuration. The authors note that these corrective trajectories are not represented in the original motion dataset.

Training Cost and Implementation Details

All policies are trained in Isaac Lab. Stage 1 takes approximately one day on three RTX 4090 GPUs. Stage 2 takes approximately one day and 21 hours on two GPUs. Stage 3 takes approximately two days and six hours on four GPUs. The tracker uses a customized proximal policy optimization algorithm, while the planner is supervised on rollout data. During Stage 3, planner optimization is decoupled from the on-policy loop and performs ten replay-buffer updates per PPO iteration.

Domain randomization samples friction, restitution, torso mass, center-of-mass offset, joint stiffness, damping, and per-joint default offsets. Actor observations receive noise, and the height scan also receives Gaussian noise plus an elevation offset for each episode. The goal-reaching terrain curriculum spans ten difficulty levels controlled by step height, gap width, or stone spacing. An environment advances after reaching the target and completing the episode without early termination; it is demoted when the closest qualifying goal distance is greater than 1.0 m.

These details clarify how many interfaces must remain consistent. Contact labels originate in retargeting, planner predictions depend on a particular map representation, the reward measures executed contacts in body-yaw coordinates, and hindsight labels are generated only from trajectories that pass the task-specific success test. A mismatch in any one of these conventions can make a seemingly small planning error appear as a control failure.

Discussion

The contact interface reduces the dimensionality of planning in a way that preserves the hardest part of the task. A full-body reference specifies root motion, all joint trajectories, and their timing. Matching that reference closely does not guarantee that a hand or foot meets the intended support. A contact target specifies the interaction that must remain true and leaves the intervening motion unconstrained. This is a more natural interface for a controller that must adapt to geometry.

Attention also creates a direct relationship between perception and action. The planner's normalized weights are both a terrain affordance map and a decoding mechanism. Because top-k contacts are selected from observed terrain points, the method avoids independent free-space regression followed by a post-hoc collision or support check. The same architecture can choose a staircase edge, a narrow ladder rung, a box face, or a knee support according to the body query and current state.

Goal conditioning makes the contact abstraction more flexible than a set of learned terrain-specific primitives. The same box can induce sitting, stepping, hand-assisted climbing, or sideways climbing depending on the target location and orientation. The planner reasons jointly about geometry and intent. This result is important because realistic environments contain ambiguous affordances whose correct use depends on what the operator asks the robot to achieve.

The hindsight loop is equally important methodologically. Reinforcement learning does not merely tune an existing imitation policy. Successful exploration produces new demonstrations in contact space. Those demonstrations can cover terrain that was absent from the original corpus, and the planner can improve without storing or tracking a full generated trajectory.

Limitations

The authors explicitly limit the representation to terrain-centric 2.5-dimensional interactions. The map is effective for floors, steps, boxes, and ladder rungs, but overhead contact, hanging, arbitrary side structures, and manipulation would require a richer three-dimensional representation. A local height map implicitly privileges surfaces suitable for support from above.

The second stated limitation is scope: the system performs local goal-conditioned locomotion, not global navigation. On unstructured real-world routes, the demonstrations sometimes use manually specified intermediate waypoints. The policy handles local contact and motion between those goals, but it does not decide the full sequence of goals across a large environment.

The third stated limitation is expressiveness. A backflip and a vertical jump can have essentially the same contact sequence despite requiring very different aerial motion. Contact determines when and where support occurs but does not encode all dynamic intent that is invisible in that sequence. Semantic task information or additional motion constraints would be needed for such cases.

An independent limitation is the scale of the real-world evidence. The simulation compares methods over thousands of episodes per category and reports confidence intervals, while the real-robot section emphasizes capability breadth across many environments and recovery cases. It does not provide an equally large, randomized, per-category real-world benchmark for every baseline. This does not invalidate the demonstrations, but it limits claims about real-world statistical advantage.

Reproducibility is also constrained by the absence of public training code, checkpoints, and data-processing tools. The paper gives network widths, parameter counts, contact thresholds, stage budgets, and many reward terms, but a complete recreation also depends on retargeting constraints, replay-buffer details, terrain generators, loss schedules, and deployment synchronization. Those details matter for a contact interface that must remain geometrically consistent from map to planner to controller.

Conclusion

The main contribution is not an isolated attention layer. It is a different division of labor between motion data, perception, planning, and reinforcement learning. Demonstrations provide contact priors during training. An attention planner predicts explicit terrain contacts for nine body parts. A contact-conditioned tracker turns those plans into coordinated motion. Goal-reaching reinforcement learning then explores beyond the data and returns successful contact trajectories as new supervision.

The reported evidence supports this progression. Contact-only tracking reaches at least 98.5 percent success on every reference-anchored category. Attention planning outperforms a similar-size MLP in generalization-sensitive conditions. Stage-3 adaptation adds 54.8 percentage points on sparse whole-body terrain. The complete system reaches 92.5 percent average success on eight composite courses versus 58.8 percent for MGMT. Real-world experiments demonstrate onboard perception and contact-rich behavior without reference input at deployment.

The next step is to extend the same interface beyond support surfaces. If contacts can be predicted on arbitrary 3D structures and combined with semantic or global goal planning, walking, climbing, recovery, and manipulation may be expressed as different sequences of body-environment interactions rather than separate skill libraries.

Key Takeaway

When a reference motion is reduced to contact, the robot is no longer asked to copy a trajectory; it is asked to negotiate a set of physical constraints with the world.