Skip to content
←Back to Papers

PAPER DEEP DIVE

Hybrid ExpertMulti-Task WalkingLegged robots

MoE-Loco: Mixture of Experts for Multitask Locomotion

We present MoE-Loco, a Mixture of Experts (MoE) framework for multitask locomotion for legged robots. Our method enables a single policy to handle diverse terrains, including bars, pits, stairs, slopes, and baffles, while supporting quadrupedal and bipedal gaits. Using MoE, we mitigate the gradient conflicts that typically arise in multitask reinforcement learning, improving both training efficiency and performance. Our experiments demonstrate that different experts naturally specialize in distinct locomotion behaviors, which can be leveraged for task migration and skill composition. We further validate our approach in both simulation and real-world deployment, showcasing its robustness and adaptability.

MoE-Loco: Mixture of Experts for Multitask Locomotion

Paper: MoE-Loco: Mixture of Experts for Multitask Locomotion
Authors: Runhan Huang, Shaoting Zhu, Yilun Du, Hang Zhao (Tsinghua IIIS / Shanghai Qi Zhi Institute / MIT)
Link: arXiv:2503.08564 | Project: moe-loco.github.io | Platform: Unitree Go2

One-line summary: Replacing a single MLP policy with a Mixture of Experts architecture lets one policy handle bars, pits, stairs, slopes, and baffles across quadrupedal and bipedal gaits, boosting mixed-task success from 57.1% to 87.9%, with naturally specialized experts recomposable into new skills.

Background and Motivation

Reinforcement learning has become the dominant paradigm for training robot locomotion policies, but single-task RL success has not naturally extended to multitask settings. Learning a unified policy across multiple terrains and gait modes remains challenging. The core problem is gradient conflict in multitask reinforcement learning (MTRL): when gradient directions from different tasks diverge, simple neural network architectures lead to poor training efficiency and even model divergence. This is especially severe when spanning quadrupedal and bipedal gaits—fundamentally different kinematics, dynamics, and reward structures.

Existing work attempts diverse terrains in parallel simulation training, but gradient conflicts in MTRL remain unresolved. ManyQuadrupeds focuses on unified policies across quadruped categories. MELA uses pretrained expert models but only addresses basic skills with heavy reward engineering. MTAC attempts hierarchical RL across terrains but handles only one gait with three simple terrains, without real-world deployment.

MoE-Loco's core idea: introduce the Mixture of Experts architecture into locomotion policy training, letting gradients naturally flow to specialized expert modules, structurally mitigating gradient conflicts. Different experts naturally specialize in distinct locomotion behaviors, which can be leveraged for task migration and skill composition—manually adjusting expert weight ratios to synthesize new skills.

Method

1. Task Definition

This work focuses on 9 challenging locomotion tasks spanning quadrupedal gaits (bar crossing, pit crossing, baffle crawling, stair climbing, slope walking) and bipedal gaits (standing, plane walking, slope walking, stair descending). The robot is controlled via joystick velocity commands, with a one-hot vector $g$ indicating quadrupedal ($g=0$) or bipedal ($g=1$) gait. Multitask locomotion is defined as an MDP $\langle S_{\tau},A_{\tau},T_{\tau},R_{\tau},\gamma_{\tau}\rangle$, where different terrains correspond to different state subspaces $S_{\tau}\subseteq S$, reward $R$ varies by gait, and termination conditions depend on gait type. The goal is a single universal policy maximizing cumulative reward across tasks:

$$J(\pi)=\mathbb{E}\left[\sum_{\tau}\sum_{t=0}^{\infty}\gamma^{t}R_{\tau}(s_{t},a_{t},s_{t+1})\right]$$

2. MoE-Based Multitask Locomotion Learning

Uses a two-stage teacher-student framework with PPO. State space includes four observation types: proprioception $\boldsymbol{p}_t$ (projected gravity, base angular velocity, joint positions/velocities, last action), explicit privileged state $\boldsymbol{e}_t$ (base linear/angular velocity), implicit privileged state $\boldsymbol{i}_t$ (per-link contact forces, encoded to low-dimensional latent to reduce sim-to-real gap), and command $\boldsymbol{c}_t$ (velocity command $V=(v_x,v_y,v_{\text{yaw}})$ and gait one-hot $g$). Action space $\boldsymbol{a}_t\in\mathbb{R}^{12}$ is desired joint positions for 12 joints.

Reward: quadrupedal $r^{\text{quad}}=r^{\text{quad}}_{\text{track}}+r^{\text{quad}}_{\text{reg}}$; bipedal $r^{\text{bip}}=r^{\text{bip}}_{\text{track}}+r^{\text{bip}}_{\text{stand}}+r^{\text{bip}}_{\text{reg}}$. Termination: quadrupedal terminates when $\theta_{\text{roll}}>1.0$ or $\theta_{\text{pitch}}>1.6$; bipedal terminates when any link except rear feet and calf contacts ground after 1 second.

MoE module is the core innovation. In stage 1 (Oracle policy), all observations $[\boldsymbol{p}_t,\boldsymbol{e}_t,\boldsymbol{i}_t,\boldsymbol{c}_t]$ are available. The implicit state is encoded and concatenated with explicit privileged and proprioception to form dual-state representation $\boldsymbol{l}_t=[\text{Enc}(\boldsymbol{i}_t),\boldsymbol{e}_t,\boldsymbol{p}_t]$, which an LSTM processes into $\boldsymbol{h}_t$. The MoE module is introduced into both actor and critic:

$$\hat{g}_{i}=\text{softmax}(g(\boldsymbol{h}_{t}))[i]$$

$$\boldsymbol{a}_{t}=\sum_{i=1}^{N}\hat{g}_{i}\cdot f_{i}(\boldsymbol{h}_{t})$$

where $g$ is the gating network and $f_i$ is expert $i$. Actor and critic MoE share the same gating network, ensuring consistency between policy evaluation and action generation. MoE dynamically routes tasks to appropriate experts, promoting coordination of similar skills while minimizing conflicts between heterogeneous tasks. Total optimization: $L_{\text{surro}}+L_{\text{value}}+L_{\text{recon}}$.

MoE-Loco pipeline overview

Figure 3: MoE-Loco pipeline overview. The MoE architecture enables robust multitask locomotion across challenging terrains and multiple gaits.

3. Skill Decomposition and Composition

The MoE framework provides a natural solution for skill composition by dynamically decomposing tasks into different skill experts. Using pretrained experts with modified gating weights:

$$\hat{g}_{i}=w[i]\cdot\text{softmax}(g(\boldsymbol{h}_{t}))[i]$$

where $w[i]$ can be manually defined or dynamically adjusted by a neural network. This enables constructing new locomotion strategies without additional training—each expert naturally specializes in different movement aspects (balancing, crawling, obstacle crossing), and selective weight adjustment synthesizes new gait patterns.

graph TD
  A[Obs: p_t + e_t + i_t + c_t] --> B[Encoder Enc]
  B --> C[Dual-state l_t]
  C --> D[LSTM]
  D --> E[Hidden state h_t]
  E --> F[Gating Network g]
  F --> G[Gate Scores softmax]
  E --> H[Expert 1 f_1]
  E --> I[Expert 2 f_2]
  E --> J[Expert N f_N]
  G --> K[Weighted Sum]
  H --> K
  I --> K
  J --> K
  K --> L[Action a_t]
  style F fill:#f5a623,stroke:#b97316,color:#fff
  style K fill:#4a90d9,stroke:#2c5f8a,color:#fff
  style L fill:#7ed321,stroke:#4a8a14,color:#fff

4. Two-Stage Training

Stage 1 trains the Oracle policy (with privileged info) and pretrains the estimator. Stage 2 accesses only $[\boldsymbol{p}_t,\boldsymbol{c}_t]$; estimator/LSTM/MoE weights are copied from stage 1, and Probability Annealing Selection (PAS) gradually adapts to imprecise estimates. Simulation in IsaacGym with 4096 parallel robots on RTX 3090: 40,000 iterations plane walking, 80,000 challenging terrain, 10,000 PAS adaptation. 50Hz control, deployed on Unitree Go2 (Jetson Orin), PD control $K_p=40.0, K_d=0.5$, $N_{\text{exp}}=6$.

Experimental Results

Simulation Comparison

Baselines: Ours w/o MoE (same framework, MoE replaced by equal-parameter MLP) and RMA (1D-CNN adaptation module). The mixed-task runway contains bars, pits, baffles, up/down stairs, up/down slopes.

MethodMix SRBar(q)Baffle(q)Stair(q)Pit(q)Slope(q)Walk(b)Slope(b)Stair(b)
MoE-Loco0.8790.8860.9240.6840.9020.9560.9320.9610.964
w/o MoE0.5710.8480.2640.5680.6980.9880.8260.5040.453
RMA0.0000.8710.0580.0170.0170.4370.0000.0000.000

MoE policy achieves the best across all three metrics. Mixed-task success rate rises from 57.1% (w/o MoE) to 87.9%. Except quadrupedal slope walking (relatively simple), MoE outperforms baselines on all single tasks. RMA completely fails in mixed scenarios (0% success) due to its MLP+CNN architecture's inability to handle multi-gait multi-terrain.

Terrain settings

Figure 2: Terrain settings. Left to right: bar, pit, baffle, slope, stairs.

Training StageIterationsObjective
Plane walking40,000Basic quadrupedal + bipedal gaits
Challenging terrain80,000Bars/pits/baffles/stairs/slopes
PAS adaptation10,000Pure proprioception adaptation

Real-World Experiments

Zero-shot deployment on real Go2. Mixed terrain: 20cm bar, 22cm baffle, 15cm stairs, 20cm pit, 30° slope (quadrupedal), then bipedal command to stand, walk 30° slope, turn and descend. 20 trials per test for average success rate.

Real-world success rate

Figure 4: Real-world success rate across multiple terrains and gaits.

Expert Specialization and Skill Composition

Quantitative analysis shows different experts naturally specialize in distinct locomotion behaviors. By adjusting gating weights $w[i]$, new skills can be composed without additional training—e.g., blending balancing and obstacle-crossing experts generates gait patterns adapted to specific new terrains.

Multi-robot objective

$$ J(\pi)=\mathbb{E}\left[\sum_{\tau}\sum_{t=0}^{\infty}\gamma^{t}R_{\tau}(s_{t},a_{t},s_{t+1})\right] $$

Limitations

Author-stated: On quadrupedal slope walking, MoE slightly underperforms the w/o MoE variant, likely because this task is relatively simple and MoE routing overhead introduces a minor negative effect.

Analysis: Expert count is fixed at 6; how to optimally choose expert count for different task complexities is not discussed. Skill composition relies on manual weight adjustment $w[i]$, lacking automated new-skill discovery. Only proprioception is used (blind locomotion); no visual perception integration, limiting scenarios requiring distant terrain sensing. Bipedal gaits only cover standing, plane, slope, and stair descending—not more challenging bipedal stair climbing or obstacle crossing. Real-world mixed-terrain success rates are not fully reported in the main text.

Conclusion and Future Work

MoE-Loco demonstrates the effectiveness of the Mixture of Experts architecture in structurally resolving gradient conflicts in multitask locomotion. Six experts naturally specialize into different locomotion behaviors within a single policy, handling 9 tasks across 5 terrains × 2 gaits. The shared gating network ensures actor-critic routing consistency. The skill composition formula $\hat{g}_i=w[i]\cdot\text{softmax}(g(\boldsymbol{h}_t))[i]$ provides a no-retraining interface for new skill synthesis. Two-stage teacher-student training with PAS enables zero-shot sim-to-real transfer.

The key to one policy managing nine tasks isn't making the network bigger—it's letting different experts handle their own specialties while the gating network learns who to listen to and when.

Related Papers

ID Balancing: Stable Training of Extremely Sparse MoE via PID-Based Load Control

ID Balancing: Stable Training of Extremely Sparse MoE via PID-Based Load Control

Scaling Large Language Models (LLMs) via Mixture-of-Experts (MoE) enables massive parameter growth with nearly constant per-token computation. However, further scaling the parameter count requires increasingly sparse routing, where expert load imbalance becomes more severe. This imbalance reduces parameter utilization and training efficiency, and can undermine training stability, becoming a bottleneck to reliable scaling. In this work, we unify two representative auxiliary-loss-free methods as incomplete Proportional-Integral-Derivative (PID) controllers: DeepSeek's loss-free method acts as a fixed-step integral controller, while Kimi K3's Quantile Balancing functions as a generalized proportional controller. Building on this control perspective, we propose ID Balancing, an Integral-Derivative controller. It scales its integral term with load error and activates its derivative term only when imbalance worsens, enabling stronger corrections for large or worsening errors and smaller updates near balance. Evaluated across Top-$10$, Top-$5$, and Top-$3$ routing over $768$ experts, ID Balancing reduces worst-case backbone MaxVio and training-average backbone MinVio by over 50% and 12%, respectively, relative to the best baselines in the Top-$3$ setting. When the total parameter count increases from $18.9$B to $69.9$B (Top-$10$-of-$768$), ID Balancing's worst-case backbone MaxVio remains nearly unchanged and is approximately 89.6% lower than that of the auxiliary-loss baseline. ID Balancing also maintains competitive language-modeling and downstream performance. The advantages of ID Balancing grow as sparsity increases, making it a promising solution for scaling larger, sparser MoE models.

Peng Jin, Zihan Qiu, Zekun Wang · 161 views · Sep 30, 2026
Mixture-of-ExpertsLoad Balancing
Sep 30, 2026
Emergent Compositional Skills in Mixture-of-Experts VLAs

Emergent Compositional Skills in Mixture-of-Experts VLAs

We consider the problem of learning compositional robot policies end-to-end from expert demonstrations, without any pre-specified notion of task decomposition or hierarchy. We ask whether a VLA trained with a simplified Mixture-of-Experts (MoE) action head can emergently learn to decompose tasks into reusable, interpretable primitives. We find that learned experts are heavily reused across tasks and consistently correspond to qualitatively distinct low-level behaviors, suggesting that the router implicitly learns to perform high-level sequencing while experts serve as compositional primitives. Our MoE matches the task performance of a monolithic baseline while demonstrating meaningful expert specialization, a step toward modular, interpretable robot policies that emerge from data alone.

Shlok Shah, Rhiaan Jhaveri, Tharun Kumar Tiruppali Kalidoss · 586 views · Jul 22, 2026
VLAHybrid Expert
Jul 22, 2026
Learning a Unified Policy for Position and Force Control in Legged Loco-Manipulation

Learning a Unified Policy for Position and Force Control in Legged Loco-Manipulation

Robotic loco-manipulation tasks often involve contact-rich interactions, requiring joint modeling of contact force and robot position. We propose a unified policy for legged robots that jointly models force and position control without relying on force sensors, and improves contact-rich imitation learning by approximately 39.5%.

Peiyuan Zhi, Peiyang Li, Jianqin Yin · 497 views · May 27, 2025
UnitreeCoRL 2025
May 27, 2025
Optimizing Grasping in Legged Robots: A Deep Learning Approach to Loco-Manipulation

Optimizing Grasping in Legged Robots: A Deep Learning Approach to Loco-Manipulation

This paper presents a deep learning framework designed to enhance the grasping capabilities of quadrupeds equipped with arms, with a focus on improving precision and adaptability. Our approach centers on a sim-to-real methodology that minimizes reliance on physical data collection. We developed a pipeline within the Genesis simulation environment to generate a synthetic dataset of grasp attempts on common objects. By simulating thousands of interactions from various perspectives, we created pixel-wise annotated grasp-quality maps to serve as the ground truth for our model. This dataset was used to train a custom CNN with a U-Net-like architecture that processes multi-modal input from an onboard RGB and depth cameras, including RGB images, depth maps, segmentation masks, and surface normal maps. The trained model outputs a grasp-quality heatmap to identify the optimal grasp point. We validated the complete framework on a four-legged robot. The system successfully executed a full loco-manipulation task: autonomously navigating to a target object, perceiving it with its sensors, predicting the optimal grasp pose using our model, and performing a precise grasp. This work proves that leveraging simulated training with advanced sensing offers a scalable and effective solution for object handling.

Dilermando Almeida, Guilherme Lazzarini, Juliano Negri · 462 views · Aug 24, 2025
Legged robotsDeep Learning
Aug 24, 2025

As an Amazon Associate, we earn from qualifying purchases.