RoboDuet: Learning a Cooperative Policy for Whole-body Legged Loco-Manipulation

Method: A Duet Rather Than One Large Controller

Stage one trains the leg policy while the arm is fixed at $[0,0.8,0.8,0,0,0]$. Keeping the arm present during training exposes the legs to its mass and inertia from the beginning. The authors intentionally avoid training an unloaded quadruped first and attaching the arm later, because that would create a large dynamics shift after deployment.

The gait is represented by a behavior vector:

$$ \mathbf{b}_t=[\theta_1^{cmd},\theta_2^{cmd},\theta_3^{cmd},f^{cmd},h_z^{cmd},\phi^{cmd},s^{cmd},h_z^{f,cmd}] $$

Here the three angle terms set phase offsets, $f^{cmd}$ is stepping frequency, $h_z^{cmd}$ is body height, $\phi^{cmd}$ is base orientation, $s^{cmd}$ is foot clearance, and $h_z^{f,cmd}$ is swing height. To focus on pose tracking, the paper fixes phase offsets to $[0.5,0,0]$, stepping frequency to 3 Hz, and swing height to 0.06 m.

The leg network receives a clock vector so that it can infer the gait phase of all four feet:

$$ \mathbf{t}_t=[\sin(2\pi t^{FR}),\sin(2\pi t^{FL}),\sin(2\pi t^{RR}),\sin(2\pi t^{RL})] $$

The per-foot phases are obtained from the command offsets:

$$ [t^{FR},t^{FL},t^{RR},t^{RL}] = [t+\theta_2^{cmd}+\theta_3^{cmd}, t+\theta_1^{cmd}+\theta_3^{cmd}, t+\theta_1^{cmd}, t+\theta_2^{cmd}] $$

When the commanded base velocity is zero, the clock is set to $[1,1,1,1]$. This suppresses marching in place. The detail matters because residual stepping creates base vibration that is transmitted through the arm and makes precise manipulation harder.

Stage one retains most rewards from the blind locomotion design but removes body-height tracking, allowing the robot to choose a height that suits the task. The authors also add an energy penalty because the arm increases the load on the leg motors and can cause overheating:

$$ r_{energy}^{loco}=-0.00004\sum_i|\tau_i\dot q_i|^2 $$

In this expression, $\tau_i$ and $\dot q_i$ are the torque and velocity of leg joint $i$. Squaring the product penalizes both high torque and fast motion, which is more relevant to thermal safety than a simple velocity cost.

Stage two activates both networks while preserving their input and output dimensions. The arm policy observes its own joint state, the target pose, base orientation, and its previous action. Its first six outputs drive the arm position targets. Its last two outputs pass through $\tanh$, are scaled by 0.4 in the simulator, and then replace the locomotion pitch and roll commands.

This low-bandwidth channel corresponds to the physical role of the body. Raising the shoulder may require a pitch adjustment; moving the arm laterally may require a roll adjustment; reaching past the front legs may require new foot placement. The leg policy remains responsible for balance, but it can now cooperate with the arm rather than treating the arm as a fixed payload.

Target orientation is sampled in Z-Y-X order and converted into axis-angle coordinates:

$$ R=R_{yaw}R_{pitch}R_{roll} = \begin{bmatrix} r_{11}&r_{12}&r_{13}\\ r_{21}&r_{22}&r_{23}\\ r_{31}&r_{32}&r_{33} \end{bmatrix} $$

$$ [\alpha,\beta,\gamma] = [\tan^{-1}(r_{21}/r_{11}), \tan^{-1}(r_{32}/r_{22}), \tan^{-1}(r_{13}/r_{33})] $$

Position and orientation errors are combined through two exponential terms:

$$ r_{\chi_t}=e^{-w\Delta(l,p,y)}e^{-\Delta(\alpha,\beta,\gamma)} $$

$$ \Delta(l,p,y)=\sum_{u\in\{l,p,y\}}k_u|u_t-u^{cmd}| $$

$$ \Delta(\alpha,\beta,\gamma)=\sum_{u\in\{\alpha,\beta,\gamma\}}k_u|u_t-u^{cmd}| $$

The coefficients $k_u$ normalize each variable by its sampling range, and $w$ balances Cartesian position against orientation. In stage two, the original foot-placement reward is replaced by a hip-joint constraint, and velocity rewards are halved to allow the feet to move for whole-body posture control.

flowchart LR
  A[Target end-effector pose] --> B[Arm policy]
  S1[Arm joint state] --> B
  S2[Base orientation] --> B
  B --> C[Six arm offsets]
  B --> D[Pitch and roll guidance]
  D --> E[Replace body commands]
  E --> F[Stage-one leg policy]
  F --> G[Twelve leg actions]
  C --> H[Concatenate into 18 actions]
  G --> H
  H --> I[Leg torques and arm targets]
  I --> S1
  I --> S2
      

Figure 1: The implemented RoboDuet loop. The arm policy's final two outputs become body-posture guidance, not motor commands.

Figure 2: The paper's two-stage architecture. Stage one learns locomotion with a fixed arm; stage two trains both policies with adjusted rewards.

Training and Evaluation Setup

Training uses IsaacGym with 4,096 parallel agents and PPO for 50,000 iterations, repeated over three random seeds. The two-stage variants allocate 10,000 iterations to stage one and 40,000 to stage two. Each cooperative policy uses 512, 256, and 128 hidden units with ELU activations. The unified baseline doubles the hidden width after merging the feature extractors.

The evaluation target space is slightly larger than the training distribution. Radius is trained from 0.30 to 0.70 m and evaluated from 0.20 to 0.80 m. Angular ranges are also widened, including orientation coordinates up to half of $\pi$. The study samples 200,000 target commands and allows four seconds to settle, followed by two seconds of averaged error.

A target is successful when position error is at most 0.03 m and angular error is at most $\pi/36$. Survival is tested by applying 10 to 20 N to the base for two seconds and requiring the body height to stay above 0.26 m. Solvability counts targets that can be reached without self-collision, while workspace measures the convex hull volume of successful targets.

What the Ablation Shows

The ablation separates the cooperative architecture from the two-stage training recipe. Unified trains one network directly in stage two. Two-Stage uses one network but first learns locomotion. Cooperated uses two policies but starts directly in stage two. RoboDuet combines cooperative policies with the locomotion prior.

Two-stage training has its clearest effect on robustness. In the still condition, survival rises from 87.35% for Unified to 95.66% for Two-Stage, an 8.31-point increase. In the moving condition it rises from 93.03% to 99.09%. The first stage therefore does more than speed convergence: it establishes a gait that remains stable after manipulation rewards are introduced.

The cooperative architecture expands workspace. Cooperated achieves slightly better tracking than RoboDuet in several positional metrics, but it is more aggressive because it lacks the gait prior. RoboDuet gives up a small amount of single-metric accuracy to reach 98.20% still survival, 99.96% moving survival, and the largest workspace in both conditions. That trade-off is important for a physical robot, where a few millimeters of tracking error are less useful if the base falls.

Against floating-base inverse kinematics under the same error threshold, RoboDuet improves solvability by 14.83%. The arm hardware is unchanged. The gain comes from allowing the body and feet to move so that endpoints outside the static arm workspace become reachable.

Figure 3: RoboDuet expands the reachable set relative to floating-base inverse kinematics under a matched tracking threshold.

Zero-shot transfer is tested on Go2 and A1, both carrying the same arm. Still workspace remains near 0.84 to 0.86 and still survival remains above 96%. Moving workspace falls to roughly 0.81 to 0.82 on the new bases, while perturbed survival remains 93.21% on Go2 and 94.34% on A1. Transfer is useful, but it is not lossless.

Real-Robot Evidence

The extreme-pose test contains five targets. Two are close to the ground and intersect the leg space. Two are much higher than the training distribution. The fifth sits near the arm's maximum reach and points toward the robot's head. Each target is repeated eight times.

Floating-base inverse kinematics cannot solve three of the five targets and has a 60% IK failure rate. Its self-collision rate is 50%, because the base does not move to avoid the legs or nearby structure. RoboDuet has no IK-failure category and reduces self-collision to 5%. For physically unreachable targets, it settles on a stable approximate pose instead of losing balance.

Figure 4: Five extreme target poses. RoboDuet maintains stable approximations when a target cannot be reached exactly.

The real-world transfer task moves a doll among the floor, a 20 cm chair, a 60 cm cabinet, and a 100 cm standing desk. Three operators perform five trials per segment, giving fifteen trials per interval. A success requires completing the transfer without dropping the object. The operators use VR handles, so the task tests teleoperated whole-body control rather than autonomous perception and planning.

Across all segments, RoboDuet succeeds 39 of 60 times, or 65.0%, versus 32 of 60, or 53.3%, for AFB+ID. The difference is about 11.7 percentage points. The paper's headline of at least 23% is a relative improvement on the harder transfer tasks, not a universal 23-point gain. This distinction matters when interpreting the abstract.

The 100 cm to floor segment exposes the benefit of whole-body actuation. The arm must operate near full extension while maintaining a usable orientation. RoboDuet lowers or inclines the base to reduce the vertical distance to the desk and succeeds 9/15 times. AFB+ID succeeds 4/15 times. Ground-level grasping tests the complementary ability to avoid collisions near the legs.

Figure 5: The transfer task requires coordinated changes in base position, base orientation, and foot support.

Figure 6: RoboDuet spans cross-platform deployment, real-world loco-manipulation, and whole-body posture adjustment.

The zero-shot real deployment moves the learned controller from Go1 plus ARX5 to Go2 plus ARX5. Total system mass rises from 18.35 kg to 21.05 kg, a 14.7% increase, without retraining stage two. The paper shows the two systems following the same trajectory at four corresponding moments.

How the Paper Maps to the Code

The training loop makes the claimed cooperation concrete. The arm policy runs first, its last two outputs are passed to the environment planner, the leg policy runs second, and only the first six arm outputs are included in the motor-action vector.

actions_arm = self.alg_arm.act(...)
self.env.plan(actions_arm[..., -self.env.num_plan_actions:])
actions_dog = self.alg_dog.act(...)
ret = self.env.step(actions_dog,
                    actions_arm[..., :-self.env.num_plan_actions])

The same split appears in scripts/play_by_key.py. The deployment path sends the last two values to env.plan and passes the remaining six values to the arm controller.

actions_arm = arm_policy(obs)
env.plan(actions_arm[..., -2:])
actions_dog = dog_policy(dog_obs)
env.step(actions_dog, actions_arm[..., :-2])

The configuration files define six executable arm actions, eight arm-policy outputs, and eighteen whole-system actions. The hybrid controller returns torques for the legs and position targets for the arm, which explains why the concatenated vector can drive the simulated and physical systems.

num_actions_arm = 6
num_actions_arm_cd = 8
num_actions = 18

The environment scales the two guidance values by 0.4 and writes them into the locomotion pitch and roll commands. The training script switches on stage two at iteration 10,000. These details distinguish an implemented cooperative policy from a diagram that merely says two networks exchange features.

Limitations

The authors explicitly scope the work to low-level control. Fully autonomous task execution still requires a high-level planner and more advanced collision detection. The real experiments use VR handles for velocity and end-effector commands, so the system demonstrates teleoperated loco-manipulation, not autonomous object discovery and grasping.

Terrain coverage is limited. The method can handle slopes and gravel while maintaining a fixed target pose, but it has not demonstrated loco-manipulation on stairs. Stairs introduce rapid changes in contact, body height, and support, making the problem substantially different from walking on a mostly flat support surface.

Zero-shot transfer is restricted to robots with similar morphology and physical dimensions. The experiments cover Go1, Go2, and A1 with the same arm. Only Go1 uses a stage-two policy tailored to that platform. Moving workspace and survival both drop after transfer, so the result should not be generalized to arbitrary robot-arm pairs.

The real trials are also small. The extreme-pose study uses five targets with eight repetitions each, and the height-transfer study uses fifteen trials per segment. These numbers establish feasibility, but they do not characterize robustness across lighting, friction, object geometry, and operator behavior. The paper's main contribution is the control architecture and its measured evidence, not a claim of general autonomous manipulation.

Conclusion

RoboDuet's important idea is structural rather than merely architectural scale. The leg policy keeps authority over locomotion and balance. The arm policy keeps authority over 6D tracking. The two are connected by a small posture-guidance channel, so the arm can ask the body to move without flooding the legs with task-specific features.

The evidence supports both parts of that design. Two-stage training improves perturbation survival, cooperative policies expand workspace, and the combined system reduces real-world self-collision from 50% to 5%. It transfers a Go1-trained controller to a Go2 system that is 14.7% heavier. The result is a practical low-level controller for a legged manipulator, not a finished autonomous robot.

The next step is to connect this controller to visual target estimation, planning, and autonomous collision reasoning. A second direction is extending the cooperative interface beyond two posture values so the legs can jointly reason about footholds and arm reach on stairs or irregular terrain. Those extensions are necessary before the duet becomes a fully autonomous mobile manipulator.

RoboDuet does not attach an arm to a quadruped and call it whole-body control. It gives the arm a way to request a different body, while leaving the legs in charge of carrying it safely.