BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion

Authors: Qiayuan Liao, Takara E. Truong, Xiaoyu Huang, Yuman Gao, Guy Tevet, Koushil Sreenath, and C. Karen Liu.

Paper: arXiv:2508.08241. Project page: beyondmimic.github.io.

Code state: the original training repository, HybridRobotics/whole_body_tracking, was retired on 15 October 2026 and its tracking work was moved to mujocolab/mjlab. The real-time controller remains available in HybridRobotics/motion_tracking_controller. This analysis checked commits 9db9158 and 742924f, respectively.

One-Sentence Summary

BeyondMimic builds a reusable library of agile humanoid skills with one motion-tracking recipe, compresses the resulting state-action trajectories into a latent diffusion prior, and guides that prior at inference time with task costs, allowing a Unitree G1 to perform unseen joystick, waypoint, inpainting, and obstacle-avoidance tasks without task-specific policy retraining.

The Problem Is Not Simply "Imitate a Clip"

A humanoid is mechanically close to a human, but that similarity does not make human motion directly executable. A floating base, roughly thirty actuated joints, changing contacts, limited foot area, actuator bandwidth, and impact absorption all participate in every agile movement. A reference clip specifies where body parts should be, yet a controller must infer how to create the forces and contact transitions that make those poses physically possible.

Model-based controllers manage this complexity by planning a smaller representation, such as center of mass or foot placement, and then tracking it with a whole-body controller. This decomposition is useful for stability, but it can erase the coordination that makes human motion look natural. A crouched, repeatedly stepping controller may be robust while remaining visibly unlike the demonstrated movement. Acrobatic contact changes also stress the reduced model precisely where it is least reliable.

Reinforcement learning can learn the missing controller directly, but motion-specific reward engineering recreates the scaling problem in another form. Walking rewards favor velocity and foot timing, while a cartwheel needs torso rotation and may permit contact on hands or other non-foot bodies. Combining such objectives into one hand-tuned function becomes increasingly difficult as the motion library grows. The system may learn each clip in isolation yet still lack a common representation for switching and composition.

Motion tracking offers a better shared supervision signal: compare the robot with a retargeted reference throughout the episode. DeepMimic-style methods established that this signal can produce convincing balance and acrobatics, while adversarial priors can improve style. Both lines, however, still tend to bind the learned controller to a motion or task. A reusable low-level tracker helps, but separately trained high-level planners may emit trajectories outside the tracker's executable distribution, especially during flight or abrupt contact changes.

The paper therefore separates two forms of generalization. The first stage asks how one training recipe can master many radically different motions. The second asks how the resulting skills can be recombined after deployment for objectives that were never encoded as training conditions. This separation is the organizing idea of BeyondMimic: make the skill foundation broad, then expose it to differentiable task costs through a generative trajectory model.

Stage One: A Compact Tracking MDP

The skill layer is a reinforcement-learning policy that observes the current robot state and reference phase and outputs joint-position setpoints. The reward compares selected body positions, orientations, linear velocities, and angular velocities with their reference values. Each error is first averaged over the relevant bodies, then converted into a bounded exponential score. This keeps position, rotation, and velocity scales from being added as raw quantities with incompatible units.

$$r(\bar e_s,\sigma_s)=\exp\left(-\bar e_s/\sigma_s^2\right),\qquad r_{\mathrm{task}}=\sum_{s\in\{p,R,v,\omega\}}r(\bar e_s,\sigma_s).$$

There are only three regularizers around the task reward: soft joint limits, action smoothness, and non-foot self-collision. In the released configuration, the limit penalty has weight 10, while smoothness and contact have weight 0.1. The sparse compensation matters. Most of the learning signal still comes from resembling the reference, while the additional terms protect hardware and remove obviously unusable actions.

$$r=r_{\mathrm{task}}-\lambda_l r_{\mathrm{limit}}-\lambda_s r_{\mathrm{smooth}}-\lambda_c r_{\mathrm{contact}}.$$

The source code follows this formulation closely. In tracking/mdp/rewards.py, squared body-position error is reduced across bodies and passed through an exponential. The configuration in tracking_env_cfg.py then mixes several such terms with fixed weights. This is not a generic reward hidden behind a learned objective: the correspondence between paper equation, environment configuration, and the scalar returned to PPO is visible in the repository.

# whole_body_tracking: tracking/mdp/rewards.py
error = torch.sum(
    torch.square(command.body_pos_relative_w[:, body_indexes]
                 - command.robot_body_pos_w[:, body_indexes]),
    dim=-1,
)
return torch.exp(-error.mean(-1) / std**2)

Reference-Relative Control Preserves Style Without Fighting Drift

A literal world-frame comparison would treat every accumulated translation error as a failure. On a real robot, small slips and state-estimation offsets are unavoidable, so a policy optimized that way may spend its authority returning to an arbitrary global point instead of maintaining the relationship among torso, limbs, and support contacts. BeyondMimic instead anchors the target pose to the robot's current anchor pose while preserving the reference motion's internal geometry.

$$R_b^{\mathrm{des}}=R_\Delta R_b^{\mathrm{ref}},\qquad p_b^{\mathrm{des}}=p_\Delta+R_\Delta(p_b^{\mathrm{ref}}-p_{\mathrm{anchor}}^{\mathrm{ref}}).$$

The translation copies the current anchor's horizontal position, but keeps the reference anchor height. The rotation compensates only yaw between the robot anchor and reference anchor. As a result, the robot can shift across the ground without the whole motion being pulled downward when the pelvis temporarily lowers during a crouch or landing.

$$p_\Delta=[p_{\mathrm{anchor},x},p_{\mathrm{anchor},y},p_{\mathrm{anchor},z}^{\mathrm{ref}}],\qquad R_\Delta=R_z(\operatorname{yaw}(R_{\mathrm{anchor}}{R_{\mathrm{anchor}}^{\mathrm{ref}}}^{\top})).$$

This relative coordinate convention is also reflected in the training command implementation. The code computes the delta between robot and reference anchors, replaces the delta height with the reference value, applies the yaw delta to orientation, and transforms all body targets into that local frame. Deployment code repeats the same alignment when it initializes a motion. The paper is therefore relying on a shared geometric contract, not merely a similar reward number.

# whole_body_tracking: tracking/mdp/commands.py
delta_pos_w = robot_anchor_pos_w_repeat
delta_pos_w[..., 2] = anchor_pos_w_repeat[..., 2]
delta_ori_w = yaw_quat(
    quat_mul(robot_anchor_quat_w_repeat, quat_inv(anchor_quat_w_repeat))
)
self.body_quat_relative_w = quat_mul(delta_ori_w, self.body_quat_w)
self.body_pos_relative_w = (
    delta_pos_w
    + quat_apply(delta_ori_w, self.body_pos_w - anchor_pos_w_repeat)
)

Adaptive Sampling Concentrates Training on Failure Segments

Long recordings are highly uneven. Most frames of a dance or locomotion sequence may be easy while a short takeoff, inversion, or recovery segment determines whether the whole motion succeeds. Uniform frame sampling gives every time bucket the same expected number of resets, so rare hard segments receive too few attempts. The paper bins the timeline and adapts reset probability from observed terminations, with a kernel that also samples neighboring bins.

The estimator is deliberately slow and bounded. Failure statistics are smoothed, and a uniform component keeps nonzero coverage even after a bin becomes difficult. Once the policy improves, the adaptive bias can recede rather than permanently starving easy regions. This matters because a curriculum that freezes forever on early failures may overfit one transition and lose continuity with the surrounding motion.

$$p_{\mathrm{bin}}\propto \bar f_{\mathrm{bin}}+\epsilon,\qquad \bar f_s\leftarrow0.999\bar f_s+0.001f_s,\qquad \bar f_s\leftarrow\bar f_s+\frac{0.1}{S}.$$

The implementation confirms the mechanism. Terminated environments contribute counts to the current time bucket; the counts are convolved across neighboring buckets and normalized into a sampling distribution. The policy is not told which semantic skill is hard. Difficulty is inferred from where it actually falls.

Sim-to-Real Depends on Matching the Actuator, Not Randomizing Everything

BeyondMimic treats the G1's actuator response as part of the model rather than a nuisance to hide with broad randomization. PD gains are derived from rotor and link inertia plus the desired natural frequency. The setup uses 10 Hz natural frequency, a damping ratio of 2, and per-joint armature estimates. Action scale is chosen to match the hardware interface. The policy then sees limited randomization over friction, restitution, joint offsets, torso center of mass, and root velocity.

This design has a clear hypothesis: if the nominal actuator is modeled badly, no amount of noise will teach the right timing. A policy trained with excessively stiff simulation joints can learn a strategy that depends on instantaneous position servoing, only to discover a more compliant real actuator and communication delay. BeyondMimic instead keeps the nominal response close and randomizes the uncertainties that remain credible.

The observation is similarly compact. It contains reference phase, anchor-relative pose errors, IMU velocity, joint offsets, joint velocities, and the previous action. The policy does not stack a long observation history at this stage. Fast feedback and precise synchronization are preferred over making the network infer delay from a long window.

$$o=[\psi,e_{\mathrm{anchor}},\mathcal V_{\mathrm{imu}},\theta-\theta^0,\dot\theta,a_{\mathrm{last}}].$$ $$a_{\mathrm{sp}}=\theta^0+\alpha\odot a.$$

The released PPO configuration uses 24 environment steps per update, 30,000 iterations, a three-layer MLP with widths 512, 256, and 128, a clipping range of 0.2, entropy coefficient 0.005, learning rate 0.001, and an adaptive KL target of 0.01. These details matter to a reproduction attempt because the contribution is a complete recipe, not only an architectural diagram.

Stage Two: Turning a Policy Collection into a Generative Prior

A collection of tracking policies is not yet a task composer. The second stage samples trajectories from those policies, adds temporally correlated Ornstein-Uhlenbeck perturbations, and records how the controller departs from the reference and recovers. This enriches the dataset around realistic error states. Recording only perfect rollouts would train a trajectory model for an environment that deployment never provides.

A conditional VAE compresses reference-motion intent. Its encoder reads only the reference phase ψ and anchor error e_anchor, producing a latent representation; the decoder combines that latent with current proprioception to reconstruct the action. The reconstruction term preserves behavior, while the KL term makes the latent continuous enough for diffusion. The design is motivated by the irregularity of impedance setpoints, which contain sharp changes that are poorly matched to a smooth trajectory prior.

$$\mathcal L_{\mathrm{VAE}} =\mathbb E_{q_\mathcal E(z\mid[\psi,e_{\mathrm{anchor}}])} [\|\hat a-a\|^2] +\beta D_{\mathrm{KL}} (q_\mathcal E(z\mid\psi,e_{\mathrm{anchor}})\|\mathcal N(0,I)).$$

The diffusion model operates on a state-latent trajectory rather than on actions alone. This joint representation is what turns a behavior prior into predictive control. Future states expose position, velocity, orientation, and contact-relevant body motion to a cost function; the corresponding latents decode into executable actions. A state-only model would not command the robot, and an action-only model would force task costs to reason through an unknown forward dynamic.

$$q(z^k\mid z^0)=\mathcal N\left(\sqrt{\bar\alpha_k}z^0,(1-\bar\alpha_k)I\right),\qquad \bar\alpha_k=\prod_{i=1}^k\alpha_i.$$ $$\mathcal L_{\mathrm{Diffusion}} =\mathbb E\left[\|z_\phi(\tau^{\mathbf k},\mathbf k)-\tau\|^2\right].$$

Stage-one trackers run at 50 Hz. For diffusion, the tracking policies are retrained and deployed at 25 Hz under the same formulation, leaving time for inference; the model predicts a 0.64 second horizon. The diffusion transformer has about 19.8 million parameters and uses 20 denoising steps. On a Mini PC with an RTX 4060 Mobile GPU, each denoising step takes about 20 ms under TensorRT. The lightweight VAE decoder runs on CPU against current observations. Diffusion inference is asynchronous with the 25 Hz control loop.

flowchart LR
    A[Retargeted human clips] --> B[Shared PPO tracking MDP]
    B --> C[Stage-1 tracking policies: 50 Hz]
    C --> D[Retrained 25 Hz trackers + noisy rollouts]
    D --> E[Conditional action VAE]
    E --> F[State-latent trajectories]
    F --> G[Transformer denoiser]
    H[Task cost or keyframes] --> I[Classifier guidance]
    G --> I
    I --> J[Future states and latents]
    J --> K[VAE decoder]
    K --> L[25 Hz G1 setpoints]
  

Guidance Reuses the Prior Instead of Retraining a Task Policy

At sampling time, classifier guidance adds the gradient of a task log-likelihood to the unconditional score. For a differentiable cost, the update simply descends the cost while the learned score keeps the trajectory inside the distribution of feasible, natural movement. The formula below is the key bridge between a generative model and a controller.

$$\nabla_\tau\log p(\tau\mid\tau^*) =\nabla_\tau\log p(\tau)+\nabla_\tau\log p(\tau^*\mid\tau) =-\nabla_\tau G(\tau).$$

Joystick control is the clearest example. The cost compares planar root velocity at each predicted horizon point with the commanded velocity. Because the model predicts a sequence, guidance can adjust the future body lean and stepping pattern before the current pose has reached the requested speed. This is predictive control inside the denoising loop, not a one-step reactive policy with a new output layer.

$$G_{\mathrm{js}}(\hat\tau_t) =\frac12\sum_{i=0}^{H}\left\|V_{xy,i}(\hat\tau_t)-g_v\right\|^2.$$

Waypoint control switches emphasis according to predicted distance. Far from the target, position error dominates. Near the target, the velocity penalty grows so that the robot decelerates instead of crossing the point at full speed. The transition is continuous rather than an explicit mode switch.

$$G_{\mathrm{wp}}(\hat\tau_t) =\sum_{i=0}^{H} (1-e^{-2d_i})\left\|P_{xy,i}(\hat\tau_t)-g_p\right\|^2 +e^{-2d_i}\left\|V_{xy,i}(\hat\tau_t)\right\|^2.$$

Motion inpainting uses known future keyframes as constraints. The controller can start from walking, insert cartwheel keyframes at 0.2 second intervals, synthesize the transition and intermediate movement, and return to locomotion. Obstacle avoidance composes a signed-distance-field cost with waypoint guidance. These are different inference-time interfaces over the same learned library, which is a stronger claim than saying the system can replay several known clips.

What Was Actually Tested

The first stage used roughly 2.5 hours of motion data. The authors validated 30 clips on the physical G1 for about 15 minutes in total. The repertoire spans standing, jumping, spinning kicks, aerial cartwheels, consecutive cartwheels, ground crawling, getting up, elderly-inspired walking, dance, and sports celebrations. Figure 2 shows the breadth of tracked behavior; Figure 1 summarizes how the stages connect.

The aerial cartwheel provides a useful stress test because success requires explosive takeoff, fast rotation, and a controlled landing. During flight, the G1 reached a peak acceleration of 31 m/s² and a pelvic angular velocity of up to 20 rad/s, with a mean of 7.01 rad/s. The paper compares that mean with 7.75 rad/s for skilled human aerial motion. Figure 3 shows the outdoor executions on soft soil, leaves, and uneven ground.

Validation scope	Reported setting	What it establishes
Robot	Unitree G1	One physical humanoid platform
Training motion	About 2.5 hours	Breadth comes from human motion data
Hardware repertoire	30 clips, about 15 minutes	Multiple agile skills transfer zero-shot
Stage-one tracking	50 Hz, ONNX inference below 1 ms	Low-latency standalone skill execution
Diffusion deployment	25 Hz tracker and controller, horizon 0.64 s	Time for guided trajectory inference
Guidance compute	RTX 4060 Mobile, 20 steps at about 20 ms each	Real-time inference on an external Mini PC

Naturalness Evidence Is Strongest for Running

The evaluation combines biomechanics and human judgment. Vertical ground reaction forces for walking retain heel-strike and push-off features, while running approaches a single impact peak. The timing is broadly human-like, although the robot lacks an actuated toe and produces sharper force profiles in some conditions. This is a meaningful check because visual similarity alone could hide a controller that loads the feet unrealistically.

For the user study, 77 participants viewed 20 paired five-second clips and selected the motion that appeared more human-like and natural. BeyondMimic was preferred overall at 70.8 percent versus 29.2 percent for Unitree's native controller. The advantage was much larger for running than walking, consistent with the visible arm swing, torso lean, and flight rhythm that separate the two running styles.

Comparison	BeyondMimic	Unitree native	Statistical result	Effect size
Overall naturalness, N = 77	70.8%	29.2%	p < .001	Cohen's h = 0.859
Running	84.7%	15.3%	p < .001	h = 1.532
Walking	57.0%	43.0%	p < .001	h = 0.281

The walking result is statistically significant but has a small effect size. That distinction is important. A method can improve naturalness without solving every platform limitation, and a missing toe joint is a plausible source of residual mismatch. The result also shows why preference should be reported separately from tracking error: a policy may reproduce body poses while still giving observers a rigid or unbalanced impression.

Task Composition Is More Informative Than a Single Success Video

The joystick experiment demonstrates omnidirectional velocity following and walking-to-running transitions. A longer-horizon trial sustained running for more than 50 m. In simulation, average velocity tracking errors were 12.14 percent for walking and 13.65 percent for running. Those errors are not negligible, but the important qualitative result is that the speed change comes from the same guided trajectory model rather than a hand-coded gait switch.

The inpainting test places sparse future keyframes around a cartwheel. The model must generate not only the acrobatic segment but also the transition from walking and the return afterward. The paper also reports sequences of walk or run segments interleaved with four keyframed cartwheels, plus transitions into lying down, standing up, spin-kicks, and flip-kicks. Figure 6 makes clear why this is a composition claim: the task interface changes the motion while the low-level style comes from the prior.

Waypoint navigation and obstacle avoidance add scene-level costs. A signed distance field provides distance gradients to obstacles, and the waypoint cost pulls toward the target. The robot detours rather than relying on a fixed set of trajectories. The same composition is demonstrated with joystick guidance, which keeps collision avoidance active even when the user command is imperfect or partly off-goal.

Ablations Show Where the Method Is Fragile

The adaptive-sampling ablation changes convergence at the level of the whole recipe. Without it, three of four selected motions still fail in challenging segments after 30,000 iterations. For an easier motion, adaptive sampling reaches success in roughly 2,000 iterations instead of 4,000. The gain is not cosmetic: the shared tracker only scales to a broad library if rare difficult intervals receive enough training resets.

The latent-space ablation is equally direct. For a cartwheel in sim-to-sim transfer, latent diffusion succeeds 95 percent of the time, while the no-latent baseline reaches 5 percent. The exact number is tied to this transfer setup and one motion family, but the contrast supports the paper's argument that action-space diffusion is a poor match for sharp impedance setpoints.

Latency is the least forgiving variable. An extra 2 ms increases velocity error; 5 ms causes one failure in three trials; 10 ms causes failures in two of three. These are small software delays by ordinary application standards, yet they are large relative to an airborne contact phase. The experiment turns timing into part of the method's validity boundary.

Ablation or stress	Reported outcome	Interpretation
No adaptive sampling	3 of 4 selected motions fail after 30k iterations	Uniform resets undersample difficult segments
Easy-motion convergence	About 2k iterations with adaptive sampling vs. 4k without	Failure weighting accelerates learning
No latent trajectory	5% cartwheel success vs. 95% with latent diffusion	Latent smoothness is central to generation
Additional 2 ms latency	Velocity error increases	Reference and execution begin to desynchronize
Additional 5 ms / 10 ms	1 of 3 failures / 2 of 3 failures	Agile closed-loop control is highly timing-sensitive

Figure 8 also includes ablations of continuous orientation, observation history, armature, and delay. The combination shows that the transfer result is not explained by scale alone. Geometry, actuator modeling, temporal alignment, and the reinforcement-learning curriculum all contribute. Removing one apparently small convention can move the policy outside the high-quality transfer regime.

What BeyondMimic Changes in Practice

The most reusable engineering idea is the interface between stages. A task does not need a new reward function if it can be expressed as a differentiable cost over a predicted trajectory. The expensive task-specific work moves to cost design, while the skill prior remains stable. For a growing humanoid motion library, this is easier to audit than a single monolithic instruction-conditioned policy because the assumptions at each layer remain visible.

The interface is also narrower than the phrase "general humanoid control" may suggest. The cost must be differentiable or otherwise convertible into a usable gradient, the trajectory must expose the state variables needed by that cost, and the horizon must contain enough information for the decision. A door-opening task with hidden contact modes or a long semantic route cannot be solved merely by replacing one quadratic cost with another.

The system view is equally important. Success depends on exact actuator parameters, action scales, ONNX policy deployment, thread synchronization, TensorRT diffusion inference, a lightweight CPU decoder, and state-estimation timing. Sharing hyperparameters across motions reduces task-level tuning, but it does not remove deployment engineering. Reproducing only the network while ignoring the real-time contract would test a different system.

Limitations

The authors identify state-estimation error as a direct limitation. The diffusion model conditions on anchor position, orientation, and velocity; an inaccurate estimate becomes part of the future it predicts. Sensor fusion or learned state estimation could improve this, but the paper does not provide a systematic degradation curve across common estimator failures.

They also identify the 0.64 second horizon as a planning limit. It is appropriate for reactive balance and local obstacle avoidance, not for distant goals or decisions that require anticipating events beyond the current window. Rolling the controller forward does not by itself create a long-horizon planner.

History creates a third reported tension. It stabilizes prediction, yet can trap the model in repetitive motion. Stronger guidance can break repetition, but may make the beginning and end of a mode transition less stable. The authors observe that gait orbits are robust once established while starts and endings are more prone to stumbling. A useful next step would be an interface that controls transient behavior without depending on a single global guidance weight.

Independent of the authors' list, the hardware evidence covers one robot model and 30 selected clips. The paper demonstrates that those clips transfer, but it does not provide a stratified estimate of repeated-run success, recovery time, energy cost, or wear by motion class. Such reporting would be more informative than treating every successful deployment as an equally difficult reliability observation.

The perception conditions also differ by task. Joystick and inpainting use proprioceptive state estimation, while waypoint navigation and obstacle avoidance use motion capture for improved localization and environmental context. Diffusion runs on a Mini PC with a discrete RTX 4060 Mobile GPU rather than on the G1's onboard computer. These details do not invalidate the demonstrations, but they define the actual system being demonstrated.

Conclusion

BeyondMimic's strongest contribution is not simply adding diffusion to a humanoid controller. It establishes a division of labor: reinforcement learning turns heterogeneous human clips into executable skills under one tracking formulation, while a state-latent diffusion model provides a compact multi-skill prior that can be optimized after deployment. Classifier guidance supplies the missing task interface.

The evidence shows that this decomposition can preserve agile, human-like movement while supporting commands and constraints absent from stage-two training. It also exposes the next technical bottlenecks clearly: reliable state estimation, longer prediction horizons, more stable guided transitions, and perception that does not depend on room-scale motion capture. The paper is therefore both a capable system result and a concrete research agenda for turning motion imitation into reusable humanoid control.

Golden insight: The useful definition of versatility is not how many motions a robot can replay, but whether its learned skills can be reorganized around a new objective without losing the physical and stylistic structure that makes them executable.