Skip to content
←Back to Papers

PAPER DEEP DIVE

HumanoidMotion retargetingSampling-based MPC

Dense Temporal Motion Retargeting for Legged Robots

DTMR jointly optimizes reference phase increments and PD joint targets, deforming motion only where robot dynamics require it. Across 999 clips it improves retargeting precision while running about 19–21 times faster than STMR.

Authors: Jaeryeong Kim, Taerim Yoon, Jin Cheng, Sungjoon Choi, and Stelian Coros. The paper is available as arXiv 2609.38617v1, classified under robotics, and distributed under CC BY 4.0. The authors released code at github.com/nicolejrkim/DTMR together with a project page and real-robot demonstrations.

One-Sentence Summary

DTMR treats the increment of reference-motion phase as an additional control input, optimizes it together with PD joint targets through GPU-parallel annealed MPPI, and therefore changes timing only where the robot cannot physically follow the source motion.

Why This Problem Matters

Human motion capture provides a broad and semantically rich source of demonstrations for humanoid robots. A captured walk, jump, or dance contains coordination patterns that would be difficult to author by hand. The source human and the target robot nevertheless differ in limb length, mass distribution, joint limits, actuator strength, and contact dynamics.

A conventional pipeline first retargets body keypoints or joint angles to the robot and then asks a whole-body controller to track the resulting trajectory. Geometry is handled in the first stage, while dynamics are handled in the second. The source time axis is usually left unchanged throughout this process.

That fixed time axis creates a specific failure mode. If a robot needs extra time to crouch before a high jump, a tracker cannot manufacture the missing preparation by increasing gains. It either falls behind the reference or sacrifices another part of the body. Many failures are therefore not errors in target pose, but errors in when the target becomes reachable.

Existing temporal retargeting methods can rescale a complete clip or treat static and dynamic segments differently. Such changes improve feasibility, but they distribute the modification too broadly. A single difficult takeoff may force the entire clip to become slower, and a dance may lose the beat even though only a few frames required adjustment.

DTMR starts from a different premise: timing should be resolved densely over the trajectory and jointly with motor control. The phase increment can vary at every control step, so a difficult local transition receives more time while an easy section can retain its original pace. Timing becomes part of the physical search rather than a preprocessing label.

Method Preliminaries

The controller builds on model predictive control with path-integral-style sampling. At each control step, it samples perturbations around a nominal future plan, rolls the candidates through a dynamics model, scores them, and updates the nominal plan using a cost-weighted average.

The core update is the MPPI rule below. Low-cost trajectories receive larger weights, but the exponential transform keeps several plausible candidates in the update instead of committing to one sampled extremum.

$$\mathbf{U}\leftarrow\mathbf{U}+\sum_{n=1}^{N_W}w_n[W]_n,\qquad w_n=\frac{e^{-J_n/\lambda}}{\sum_m e^{-J_m/\lambda}}$$

Here $\mathbf{U}$ is the nominal plan, $[W]_n$ is one sampled noise sequence, $J_n$ is its rollout cost, and $\lambda$ controls the softness of the weighting. GPU execution is essential because thousands of independent physics rollouts are evaluated for every control step.

The reference motion is represented by body keypoints $\bar{\mathbf{p}}$, and a phase variable selects the portion of that motion currently being requested. Advancing the phase at the source rate reproduces source timing. Advancing it faster or slower changes local tempo while preserving the same underlying motion path.

How DTMR Works

1. The original problem searches controls and phase separately

The robot state is $\mathbf{x}_k=(\mathbf{q}_k,\dot{\mathbf{q}}_k)$, where $\mathbf{q}_k$ contains the root pose and joint angles. The state evolves under dynamics $f$ and motor control $\mathbf{u}_k$. A phase trajectory determines which reference frame the robot should match at each step.

A direct temporal retargeting objective can be written as follows:

$$\min_{\mathbf{u},\boldsymbol{\phi}}\frac{1}{2T_\phi}\sum_{k=0}^{T_\phi}\left\|\mathrm{FK}(\mathbf{x}_k)-\mathrm{LI}(\phi_k;\bar{\mathbf{p}})\right\|_Q^2$$

The problem is constrained by robot dynamics and a monotone phase path:

$$\mathbf{x}_{k+1}=f(\mathbf{x}_k,\mathbf{u}_k),\quad\phi_0=0,\quad\phi_k\leq\phi_{k+1},\quad\phi_{T_\phi}=1$$

$\mathrm{FK}$ denotes forward kinematics, $\mathrm{LI}$ interpolates reference keypoints at the selected phase, $Q$ weights the tracking error, and $T_\phi$ is the number of control steps needed to reach the end. This formulation is conceptually clean, but it leaves phase as a separate trajectory that is awkward to sample inside a standard motor controller.

2. Phase increments become part of the control vector

DTMR converts the bilevel-looking problem into a single augmented control problem:

$$\tilde{\mathbf{x}}_k=(\mathbf{x}_k,\phi_k),\qquad\tilde{\mathbf{u}}_k=(\mathbf{u}_k,d\phi_k),\qquad\tilde{\mathbf{x}}_{k+1}=\tilde{f}(\tilde{\mathbf{x}}_k,\tilde{\mathbf{u}}_k)$$

The robot part of the state follows the original dynamics, while the phase advances according to $\phi_{k+1}=\phi_k+d\phi_k$. Each sampled control now answers two questions at once: which joint targets should the robot execute, and how much of the reference should it consume during the next interval?

To make tempo changes comparable across clips, the paper defines a deformation rate relative to source speed:

$$r_k=\log_2(d\phi_k/d\phi^{\mathrm{src}}),\qquad\langle|r|\rangle=\frac{1}{T_\phi}\sum_k|r_k|$$

A value of $r_k=-1$ means the local motion is twice as slow as the source, while $r_k=1$ means it is twice as fast. The average absolute rate $\langle|r|\rangle$ summarizes how much timing was changed over a complete motion.

3. Tracking and temporal deformation share one running cost

After state augmentation, the trajectory objective becomes:

$$\min_{\tilde{\mathbf{u}}}\frac{1}{T_\phi}\sum_{k=0}^{T_\phi}c(\tilde{\mathbf{x}}_k,\tilde{\mathbf{u}}_k)$$

The cost separates tracking, timing, and regularization:

$$c(\tilde{\mathbf{x}}_k,\tilde{\mathbf{u}}_k)=c_{\mathrm{track}}+w_{d\phi}c_\phi+c_{\mathrm{reg}}$$

The tracking term compares root position, link position, link orientation, joint velocity, and torque behavior against the reference selected by the current phase. The phase term penalizes temporal deformation, and the regularizer discourages abrupt or high-energy controls. The scalar $w_{d\phi}$ determines how strongly source tempo is protected.

This cost has an important practical consequence. The controller does not need a classifier that labels an action as static or dynamic. At each step it compares the physical tracking benefit of slowing down against the timing cost. A high jump may retain speed during flight and slow down only before takeoff, when the robot needs more time to load its legs.

4. Sampling is compressed to control nodes

Independently sampling every joint target and phase increment at every horizon step would create a very high-dimensional search space. DTMR instead samples a small set of control nodes and expands them into the full horizon. The main experiment uses a horizon of 40 control steps and only 8 nodes, while evaluating 4096 parallel rollouts per annealed iteration.

The sampling variance follows a schedule inherited from DIAL-MPC:

$$\sigma_{i,h}=\sigma_0\beta^{i-1}\beta_h^{H-h}$$

Later iterations search closer to the current plan, and near-term controls are refined more aggressively than distant ones. The work also applies robust norms to tracking residuals so that one large component does not dominate the ranking of an otherwise useful rollout:

$$\rho(\mathbf{e})=\sum_i\left[(|e_i|^q+\delta^q)^{1/q}-\delta\right]$$

Root and link errors use different values of $\delta$ and $q$. The root is treated more like a direct spatial constraint, while link orientation and position receive a smoother robust penalty. This distinction matters because a small root drift often changes the global task more than a local limb deviation.

5. Rollouts advance phase before physics

The initial plan is built from the reference joint trajectory and the source phase speed. For each sampled control sequence, the controller advances phase first, sets the selected PD target, performs the physics step, and evaluates the tracking cost at the newly selected reference frame. It then updates the plan with annealed MPPI, executes only the first control, and shifts the warm-start plan forward.

The public implementation exposes the same ordering. The controller extracts the final action dimension as the phase-rate command, updates phase, launches control and physics, and evaluates plan cost:

for t in range(H):
    extract_rdphi(actions, rdphi, t)
    phase_update(rdphi, phi, dphi, frame_idx, frame_alpha)
    launch_ctrl(state, actions, t)
    step_physics(state, actions, t)
    launch_plan_cost(state, t)

This logic is implemented in src/dtmr/tracker/controller.py:327-395. Getting the ordering wrong would compare a robot state to a reference frame selected one step later, which is especially harmful around contact events.

6. The released implementation has two reproducibility details

The paper describes linear interpolation between control nodes, but the current repository builds the interpolation matrix with a quadratic spline: make_interp_spline(..., k=2) in src/dtmr/tracker/splines.py:10-17. A quadratic spline can produce smoother transitions between nodes, but the paper does not include an ablation isolating this implementation choice.

The repository also contains debug defaults that are smaller than the paper's main evaluation setting of 4096 samples, 40 horizon steps, and 8 nodes. Results reported in the paper should therefore be reproduced with the release configuration and table values, not by assuming that every code default corresponds to the final experiment.

7. Dense timing is produced by receding-horizon feedback

Only the first action of each optimized plan is executed. The next state is simulated or measured, and the entire phase-control problem is solved again. This receding-horizon loop lets timing respond to deviations: if a landing impact changes the robot's vertical velocity more than expected, the next phase increment can allocate more time to recovery.

The cost also determines where deformation is concentrated. Spreading a small slowdown over many easy frames may be cheaper than making one difficult frame impossible to track, but the reverse can occur when a local transition dominates the failure. The paper's high-jump example demonstrates the second case: the robot delays takeoff and shortens flight instead of uniformly stretching the entire approach.

The following diagram summarizes the paper-specific rollout loop rather than a generic model pipeline.

flowchart LR
    A[Source keypoint motion] --> B[Phase phi in 0 to 1]
    B --> C[Plan nodes: PD targets + dphi]
    C --> D[Sample 4096 controls]
    D --> E[Advance phase before physics]
    E --> F[MuJoCo Warp rollout]
    F --> G[Tracking + timing + torque cost]
    G --> H[Annealed MPPI elite update]
    H --> I[Execute first control]
    I --> J[Shift warm start plan]
    J --> D
  

8. Why joint optimization changes the result

If timing is fixed first, a controller can only report that a rapid motion is infeasible. If timing is optimized separately, the resulting curve may not match what the motor controller can realize. DTMR compares timing and joint targets inside the same physics rollout, so the selected phase rate reflects the actual cost of following it.

This explains why the method can improve both feasibility and semantic tracking. A dynamic segment receives the extra time needed to reduce large pose errors, while a static segment can remain close to source speed. Timing is used as a continuous resource that lowers physical cost, not as an instruction to slow down uniformly.

Experiment Design

The authors retarget a two-hour human-motion dataset into 999 clips, including 496 locomotion clips. Humanoid experiments cover Unitree G1, R1, and H1-2 as well as Booster T1. A smaller quadruped evaluation uses Unitree Go1. The resulting references are evaluated in simulation and then used to train policies for real-robot deployment.

The paper reports two complementary errors. Phase-aligned retargeting error compares the robot and reference at the same normalized time, so it exposes failures caused by physical infeasibility. Dynamic time warping error first allows a nonlinear temporal alignment, then measures shape mismatch. The first metric asks whether the motion happens on schedule; the second asks whether its semantic structure survives.

The tracking-cost weights are listed below. They are not a single global scale factor: root position, links, orientation, velocity, phase, torque, and action smoothness have separate roles.

Cost termWeightPurpose
Root position28Preserve global translation
Link position14Track whole-body keypoints
Link orientation5Preserve limb orientation
Joint velocity0.01Light smoothness constraint
Phase deformation5Control timing changes
Torque energy0.05Discourage excessive effort
Action rate1.0Reduce target jumps

The optimization settings below come from the paper's principal experiments. They are important because runtime and success rates change substantially with the number of rollouts, horizon length, and annealing iterations.

ParameterValueParameterValue
Samples4096Horizon40 steps
Control nodes8Cold / warm annealing5 / 2 iterations
Initial noise0.15Iteration decay0.6
Horizon envelope0.95Sampling temperature0.3
Elite fraction0.1Phase noise0.3
Root robust norm0.1 m, q=2Link robust norm0.2 m, q=4
Control period20 msPhysics step5 ms

Results

1. Dynamic feasibility improves on all four humanoids

DTMR is applied on top of SoMa-RT and compared with GMR, PHUMA, OmniRetarget, and the original SoMa-RT output. The median phase-aligned errors improve on every humanoid: G1 falls from 103.7 mm to 81.4 mm, R1 from 83.9 mm to 77.6 mm, H1-2 from 151.8 mm to 136.3 mm, and T1 from 132.0 mm to 112.8 mm.

PlatformSoMa-RT median errorDTMR median errorRelative improvement
Unitree G1103.7 mm81.4 mm21.5%
Unitree R183.9 mm77.6 mm7.5%
Unitree H1-2151.8 mm136.3 mm10.2%
Booster T1132.0 mm112.8 mm14.5%
Four-platform average13.4%

The variation across robots is informative. R1 already has relatively low baseline error, so there is less room to improve. G1 and T1 gain more from local timing changes, suggesting that actuator limits and morphology alter where physical infeasibility appears.

2. Most gains come from the hardest quartile

The lower quartile improves by only 1.8% on average, while the median improves by 13.4% and the upper quartile by 14.9%. Easy clips therefore remain close to their original timing, while difficult clips receive the largest benefit. This pattern is inconsistent with a method that simply slows every motion by a constant factor.

Dynamic time warping error shows the same direction. Median warped error improves by 12.9% across the four humanoids. Since this metric already tolerates nonlinear time alignment, the improvement indicates that DTMR preserves motion shape while changing local tempo rather than replacing semantic structure with a smoother but different action.

The dance study exposes the tuning tradeoff. A larger phase weight protects the source rhythm, while a smaller weight allows more deformation. When $w_{d\phi}$ decreases from 5 to 0.5, average absolute deformation increases from 0.20 to 0.23 and error drops by about 30 mm. At weight 10, error is about 202 mm; at 0.5, it is about 170 mm, with the range 164 to 170 mm for values at or below 0.5.

Dance timing and duration under different phase weights

Figure 1: Paper Figure 4. Larger phase weights preserve source tempo, while smaller weights permit more timing deformation. Image from the paper.

3. DTMR is more precise under the same deformation budget

To separate the benefit of dense timing from the benefit of unrestricted deformation, the authors compare against STMR using the same deformation budget. DTMR is about 22% more precise. On ten G1 clips, total runtime falls from 45.4 hours to 2.4 hours, about 19.2 times faster; average time per clip falls from 4.5 hours to 14 minutes, about 18.9 times faster. Two Go1 clips fall from 2.4 hours to 7 minutes, about 20.8 times faster.

SettingBaselineDTMROutcome
Ten G1 clips, total45.4 h2.4 hAbout 19.2x faster
Average G1 clip4.5 h14 minAbout 18.9x faster
Two Go1 clips2.4 h7 minAbout 20.8x faster
Same deformation budgetSTMRDTMRAbout 22% more precise

The speedup comes from solving a single sampling problem instead of repeatedly optimizing a separate temporal trajectory. The complete two-hour dataset is estimated to require about 200 GPU hours, or 80 to 100 times the motion duration. This remains a substantial compute cost, but it shifts large-scale dataset generation from impractical to feasible on a well-equipped machine.

4. Jump and flip clips reveal the effect of localized timing

Table V gives the clearest failure cases. The third jump clips improve from 556.0 mm to 82.2 mm, the flip from 1213.6 mm to 111.6 mm, and the first catch from 165.3 mm to 55.3 mm. These motions require large center-of-mass changes and brief flight phases; when timing is fixed, tracking error accumulates before landing and cannot be recovered.

MotionSTMR errorDTMR errorChange
Jump-3556.0 mm82.2 mm85.2% lower
Flip1213.6 mm111.6 mm90.8% lower
Catch-1165.3 mm55.3 mm66.5% lower
Side steps83.2 mm76.7 mm7.8% lower
Brushing11.6 mm12.3 mm6.0% higher

Static actions do not improve monotonically. Brushing increases slightly, showing that additional timing freedom can introduce small deviations when the original timing was already suitable. The result supports a nuanced conclusion: DTMR is valuable for difficult dynamics, but not every clip should be expected to benefit.

High jump motion with delayed crouch and shortened flight

Figure 2: Paper Figure 1. DTMR delays the jump preparation and reduces flight time while preserving the recognizable high-jump structure. Image from the paper.

STMR fails the high jump while DTMR succeeds

Figure 3: Paper Figure 3(a). STMR fails during the high jump, while DTMR succeeds after changing takeoff timing. Image from the paper.

STMR fails the flip landing while DTMR remains stable

Figure 4: Paper Figure 3(b). STMR loses the landing, while DTMR redistributes flight and landing timing. Image from the paper.

5. Pelvis timelines show that timing is changed locally

Figure 5 uses pelvis height over time for a static catch. STMR applies a broader timing deformation, while DTMR remains closer to the source rhythm and deviates mainly when balance requires it. Figure 6 shows the opposite case: the high jump is delayed before takeoff and then compressed during flight.

Pelvis height timelines for source, STMR, and DTMR in a static catch

Figure 5: Paper Figure 6(a). Pelvis timelines for a static catch show that DTMR does not uniformly stretch the easy portion. Image from the paper.

Pelvis height timelines for source, STMR, and DTMR in a high jump

Figure 6: Paper Figure 6(b). The high-jump timeline changes takeoff and landing timing while preserving local flight structure. Image from the paper.

These plots matter because they connect the optimization objective to an observable behavior. The controller returns toward source speed when tracking is affordable and changes phase more aggressively when the physical constraints require it. The average error reduction is therefore produced by a sequence of local timing decisions.

6. Real-robot deployment provides an initial execution check

The authors train policies on DTMR references and deploy them on a Unitree G1. Tracking reward improves from 40.8 to 42.5 for a foot-shuffle dance and from 38.9 to 41.7 for a weight-shift dance. In the second motion, the arm is slowed to 0.84 times the source speed, allowing the robot to maintain the broader rhythm while avoiding an arm motion it cannot execute comfortably.

The real-robot evidence is encouraging but not exhaustive. The paper does not establish long-term reliability across many robots or environments. It does show that the timing changes survive the transition from simulation to actuated hardware and do not only fit a narrow simulator state distribution.

Implementation and Reproducibility

The repository supports several humanoids and quadrupeds and provides an end-to-end path from imported motion to tracking, scoring, replay, and video rendering. The code's phase exponent base is set to 2.0, matching the paper's definition of deformation rate in log base two.

dphi = (dphi_exp_base ** rdphi) * dt / motion_duration

This corresponds to src/dtmr/env/kernels.py:745-764, where the last action dimension is restored to a phase increment. The controller then advances phase before the physics step. A reimplementation that updates phase after physics will evaluate the tracking cost against the wrong source frame.

The code also captures the rollout sequence in a CUDA graph to reduce launch overhead. That engineering choice is part of why thousands of parallel rollouts are usable in practice. At the same time, the quadratic interpolation and debug defaults should be documented when comparing runtimes or reproducing a particular table entry.

Discussion

The main methodological contribution is a change in what counts as a control resource. Timing is not a fixed label attached to a reference clip; it is a variable that determines whether the robot can satisfy the next physical state. This is especially appropriate for legged systems, where contact timing and actuator saturation are central to feasibility.

The work also changes how robot motion datasets can be produced. Instead of copying human timing and hoping a tracker can follow it, the pipeline can generate references that already contain physically informed local re-timing. That may improve downstream imitation learning because the learner receives trajectories closer to the controller's reachable set.

Physical feasibility is still not semantic correctness. A motion can be dynamically stable and still be wrong for the task. Music-driven dance, human-robot collaboration, and object manipulation all contain event constraints that cannot be inferred from tracking cost alone.

Limitations

The authors explicitly note that DTMR does not know which parts of a motion must remain semantically fixed. A dance can become physically easier while losing the musical beat. The phase weight limits the amount of deformation, but it does not encode that a specific footfall, gesture, or accent must occur at a specific time. Event or musical constraints would be needed for those tasks.

The authors also leave object interaction open. Catching, placing, or throwing requires the robot and object to reach a contact state simultaneously. Retiming the robot alone can break the object's trajectory or contact schedule. A future formulation should optimize robot phase, controller, object state, and contact events together.

The method is compute-intensive. A complete two-hour dataset requires roughly 200 GPU hours, and each robot or contact model may require new tuning. This is much better than the baseline pipeline, but it is still a barrier for groups without parallel physics simulation.

Tracking weights determine which body parts are allowed to absorb error. A high root weight preserves global translation but may push error into the arms or joints. The paper studies several weight settings, yet real tasks still require a human decision about which errors are acceptable.

Finally, the public code and paper differ in interpolation and some default parameters. The quadratic spline may be preferable, but it is not isolated by an ablation. Exact reproduction should record the repository commit, release configuration, and whether CUDA graph execution is active.

Conclusion

DTMR makes reference phase an explicit control input and optimizes it with PD targets inside one GPU-parallel physics rollout. This allows the controller to slow down only the segments that require more time, speed up easy segments, and preserve the overall motion structure. The method is not merely a post-hoc smoother; it changes the formulation of temporal motion retargeting.

Across four humanoids, the paper reports lower phase-aligned and time-warped error, particularly on difficult dynamic clips. Under the same deformation budget, DTMR is more precise than a separate temporal baseline and roughly nineteen times faster. Real-robot experiments provide an initial execution check.

The next step is to make timing semantically aware. Music beats, object contacts, and task events should become explicit constraints around the physically optimized phase. If that happens, dense temporal retargeting can move from making a motion physically possible to making it happen at the moment the task requires.

Golden line: A robot often does not need a different pose; it needs the correct pose placed on a physically reachable timeline.

Related Papers

XBot-Omni: An Omni-Modal Model for Embodied Understanding and Interaction

XBot-Omni: An Omni-Modal Model for Embodied Understanding and Interaction

Embodied interaction requires an agent to integrate multimodal perception with verbal and nonverbal behaviors to effectively communicate its intentions, including speech, body motion, and facial expressions. We present XBot-Omni, a Thinker–Talker–Actor architecture that extends the Omni paradigm to robotic modalities, including body motion and facial expressions, with ``X'' representing the extensibility to robotic-specific modalities. The Thinker processes text, images, audio, and video to formulate contextually appropriate responses, which are then realized by the Talker and Actor through speech and coordinated bodily expressions, respectively. In this report, we innovatively introduce the concept of ``Actor'' as a dedicated component for embodied agents: a coordinated, streaming module that leverages the native representations of the Thinker and Talker within an end-to-end pipeline, introducing only modest latency overhead. This unified architecture enables multimodal generation conditioned on omni-modal inputs directly to prevent information loss in staged designs. We comprehensively evaluate XBot-Omni across seven benchmarks, demonstrating strong multimodal understanding capabilities on both general-domain and robot-centric videos, as well as high-quality, natural, and well-coordinated motion generation. Furthermore, our results validate its capability to generate embodied multimodal responses directly from omni inputs, meeting the diverse requirements of embodied interaction. XBot-Omni enables robots not only to perceive the omni world but also to express themselves through speech and bodily behaviors, extending Omni models from conversational interaction to embodied communication in physical environments.

Kerui Gu, Xing Fu, Shuoyang Sun · 217 views · Nov 12, 2026
Embodied AIOmni-Modal Model
Nov 12, 2026
A Unified Inertial Encoder for Human and Humanoid Motion

A Unified Inertial Encoder for Human and Humanoid Motion

A shared kinematic-IMU interface jointly learns velocity estimation and motion-text alignment from human and retargeted humanoid motion, enabling zero-shot Unitree G1 deployment without robot-specific training.

Yutian Chen, Shibo Zhao, Rocky Duan · 240 views · Nov 6, 2026
Inertial OdometryHumanoid robots
Nov 6, 2026
DyCraft: An Agentic System for Long-Horizon 4D Scene Generation

DyCraft: An Agentic System for Long-Horizon 4D Scene Generation

While frontier coding agents can generate high-quality 3D scenes and articulated assets, they struggle to create coherent long-horizon animations. When equipped with general-purpose harnesses, they frequently produce physically implausible outputs, generating objects that interpenetrate, lack physical support, or move unnaturally. Such bare agents struggle to repair these errors because they default to scripting the full animation in one pass, making it difficult to verify their work and disentangle complex spatiotemporal constraints. Drawing on the practice of human animators, we introduce DyCraft, a system for animated 3D scene generation that breaks animation into steps that coding agents can more reliably solve. DyCraft first expands the user's prompt into a detailed plan and builds a static scene. It then creates a 3D storyboard of key animation poses, where each frame is a static 3D scene that the agent can build and debug in isolation. Storyboarding decomposes the problem into pairs of verified start and end states, allowing the agent to author local motion between them with a collection of motion authoring tools. We evaluate DyCraft on 24 long-horizon, multi-shot animation tasks covering humanoid robots, human and animal characters, and physical mechanisms. Compared with a native GPT-6 agent in Codex and prior agentic baselines, DyCraft substantially reduces physical violations and generates higher-quality animated 3D scenes that are more often preferred by human raters.

Jeff Tan, Hang Dai, Zeqian Long · 231 views · Oct 29, 2026
4D GenerationAgentic AI
Oct 29, 2026
KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills

KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills

A physics-based whole-body control framework that combines contact-aware motion filtering, differentiable IK retargeting, and adaptive RL tracking to teach Unitree G1 highly dynamic skills from human motion.

Weiji Xie, Jinrui Han, Jiakun Zheng · 222 views · Jun 15, 2025
HumanoidWhole-Body Control
Jun 15, 2025

As an Amazon Associate, we earn from qualifying purchases.