PAPER DEEP DIVE
Movens-Zero: Scaling Humanoid Motion Foundation Models with Large-Scale Human Videos
Movens-Zero builds five million executable humanoid motions from Internet human videos, unifies text, video, speech, and music conditioning, and drives Unitree G1 in real time through chunked closed-loop control.
One-Sentence Summary
Movens-Zero converts Internet human videos into more than five million physically repaired and tracking-verified Unitree G1 motions, then couples a frozen multimodal encoder to a scalable flow-matching action expert that can generate executable reference trajectories in a real-time closed loop. Its central claim is that humanoid motion scaling depends on the executability of the training distribution, not merely on the number of action sequences collected.
The Core Problem Is the Embodiment Gap
Human video is an unusually broad source of behavior. It contains cooking, sports, dance, gestures, recovery movements, and improvised interactions that are difficult to reproduce with motion capture or robot teleoperation. Yet a video is not a motion record. The camera moves, people occlude one another, scale is ambiguous, gravity may be only visually implied, and markerless pose estimators can jump at shot boundaries or during fast contact. Treating these artifacts as ordinary noise forces one model to learn vision, geometry, and robot dynamics at once.
The harder issue appears after a plausible human motion has been recovered. Human and humanoid bodies do not share the same limb lengths, joint ranges, foot geometry, mass distribution, or balancing strategy. A trajectory can look correct in SMPL-X space and still produce knee-limit violations, foot skating, self-penetration, or unsupported centers of mass after retargeting to G1. Kinematic plausibility also does not prove that a closed-loop controller can track the reference without falling.
Movens-Zero therefore treats executability as a data problem before it treats it as a model problem. Every retained sample keeps its source video, temporal span, text and audio conditions, recovered human body, retargeted robot trajectory, selected rollout, and quality evidence. This provenance is what allows the paper to compare raw recovered motion with filtered, repaired, and execution-verified motion under a fixed training budget.
The System Has Three Separated Responsibilities
The first component is a frozen Qwen2.5-Omni-3B encoder. It consumes text, sampled video frames, or audio and produces 2,048-dimensional context tokens with an attention mask. The second is a flow-based transformer called the action expert. It receives those context tokens, the latest 40-dimensional robot state, the unexecuted tail of the previous prediction, and temporal progress features, then generates a short motion chunk at 50 Hz. The third is a high-frequency whole-body tracker that converts the reference into simulation or real-G1 control.
Figure 1: A frozen multimodal understanding backbone, a scalable flow-matching action expert, and a whole-body tracker share a robot-centric motion interface; Movens-5M supplies execution-verified supervision.
flowchart LR U["Text / speech / music / video"] --> Q["Frozen Qwen2.5-Omni-3B"] Q --> C["Context tokens
2048 dimensions"] S["Latest G1 state
40 dimensions"] --> A["Flow action expert"] C --> A P["Unexecuted tail
clean prefix"] --> A A --> M["30-frame prediction
50 Hz"] M --> E["Commit first 20 frames"] E --> T["Whole-body tracker"] T --> R["Simulation or real G1"] R --> S
This decomposition avoids two brittle extremes. The semantic backbone does not regress low-level torques directly, so it can retain broad multimodal knowledge from pretraining. The action expert does not need to solve raw perception from scratch; it learns the mapping from context tokens to a robot motion space. The tracker owns high-frequency stability and hardware limits. The interfaces are explicit enough that a research team can replace one component without redefining the entire stack.
Building Movens-5M
The data engine begins with media validation and shot-aware segmentation. Long recordings are checked for readable streams, duration, resolution, and frame-rate consistency, then divided at detected shot boundaries. Train and evaluation splits are formed at the original-video level before temporal slicing, and near-duplicate clips are removed across splits. Each descendant clip keeps a stable source identity and frame offset. These choices prevent adjacent windows or reuploaded videos from inflating the apparent training set.
Figure 2: Raw web video passes through clip generation, visual and motion filtering, semantic annotation, motion recovery, physical refinement, retargeting, and tracking verification.
The visual branch rejects degraded footage and checks whether a real, clearly visible person is central to the scene. Qwen3-VL estimates human relevance, and a clip is retained only when the normalized yes/no probability crosses a threshold:
$$s_{\mathrm{human}}\geq 0.90$$Visual quality alone cannot distinguish articulated motion from a moving camera or a hard cut. RAFT optical flow and person masks are therefore used to separate motion inside the human region from background motion. The diagnostics look for spikes, temporal isolation, subject disappearance, and abrupt scene changes. The design avoids rejecting genuine fast limb movement simply because its flow magnitude is high.
The semantic stage produces a factual video caption, a temporally localized motion caption, structured motion and scene labels, and synchronized speech or music conditions. Single-person clips are reconstructed with GVHMR; crowded, occluded, or interaction-heavy clips use PromptHMR. The outputs are canonicalized to SMPL-X. To reject footage whose apparent action is mostly camera movement, the pipeline removes global root translation and orientation and measures distal-joint movement after body-scale normalization:
$$A_{\mathrm{limb}}\geq 0.08$$This test preserves in-place gestures and squats while removing clips dominated by camera dolly or root translation. It is a better fit for motion learning than a simple requirement that the person occupy more pixels or travel farther in the frame.
Physical refinement has two loops. First, four-second SMPL-X chunks with half-second overlap are tested for ground contact, temporal jitter, fall risk, self-penetration, and foot sliding. A failed interval is regenerated conditionally with trusted boundary context and then rechecked together with its neighbors. Second, GMR provides a morphology-aware robot initialization, followed by sequence-level optimization over root translation and actuated joints with contact, velocity, and position constraints. A retargeted trajectory is retained only after simulation rollout by at least one tracking policy achieves a sufficient score:
$$q\geq 0.20$$The policy bank may contain SONIC, Humanoid-GPT, HoloMotion, and OmniXtreme, but success by one evaluated policy is enough. This is not a hardware success-rate study. It is a dataset gate intended to remove trajectories that look plausible but cannot be tracked. The gate also creates a known selection bias: extreme, airborne, or unstable behaviors are underrepresented because they are harder to verify.
Augmentation Includes a Second Acceptance Test
Web-scale collection still overrepresents short, common, easy-to-film activities. The augmentation pipeline expands three axes: underrepresented fine-grained categories, long-horizon compositions of three to six sub-actions, and targeted extreme movements such as rapid support changes, inverted poses, fast rotations, and large-range balance maneuvers. A new caption is not accepted merely because it paraphrases an old one; it must instantiate a meaningfully different motion.
The paper encodes original and rewritten captions with Qwen2.5-Omni-3B and measures the rewritten sample's nearest distance in the shared representation:
$$d_{\mathrm{nov}}(\tilde c)=\min_{c\in\mathcal{C}}\left\|\pi(\tilde c)-\pi(c)\right\|_{2}$$The threshold is calibrated from nearest-neighbor distances inside the original caption set. For long compositions, the model generates each segment while carrying the terminal state, state history, and motion prefix into the next segment. The robot is not reset between sub-actions, so the concatenated result is a continuous trajectory rather than several independent clips placed side by side.
Figure 3: Caption design expands semantic and difficulty coverage; generated trajectories are retained only after tracking and semantic-consistency checks.
Synthetic motion can still be unstable or drift from its instruction. Each candidate is replayed with a tracking policy and rendered. A vision-language model compares the observable body motion with the conditioning caption, ignoring objects, background, and appearance that a robot-only render cannot preserve. With logits for constrained yes and no answers, semantic consistency is:
$$s_{\mathrm{sem}}(v,\tilde c)=\frac{\exp(z_{\mathrm{yes}})}{\exp(z_{\mathrm{yes}})+\exp(z_{\mathrm{no}})}$$A sample is kept only if it passes both the tracking criterion and the semantic threshold. This closes an important loophole in synthetic data generation: a motion can be executable but express the wrong action, or express the action in a visually convincing animation that the controller cannot follow. The remaining limitation is explicit: rendered bodies do not validate object affordances or physical contact with the scene.
What Five Million Executable Motions Represent
The finished dataset contains roughly 15 million human-motion sequences and five million verified G1 sequences, exceeding 10,000 hours and 1.08 billion frames. It covers 297 fine-grained motion types and 16 scene groups. The two sequence counts are deliberately separate: the larger number represents recovered human motion, while the smaller number represents the subset that completed robot retargeting and closed-loop verification.
| Property | Movens-5M | Meaning |
|---|---|---|
| Human motions | 15 million | Recovered and canonicalized to SMPL-X |
| Executable G1 motions | 5 million | Retargeted and successful under at least one tracker |
| Duration and frames | Over 10,000 hours / 1.08B frames | Totals from the complete pipeline |
| Coverage | 297 motion types / 16 scenes | Daily activity, fitness, sport, dance, locomotion, performance, and gesture |
| Verification bank | Up to 4 policies | SONIC, Humanoid-GPT, HoloMotion, and OmniXtreme |
Figure 4: Duration density, hierarchical category composition, scene composition, and motion-scene association show coverage beyond a single locomotion or dance corpus.
Figure 5 compares eight motion datasets in one shared G1 space. The descriptor uses 61 frames of mean-centered joint angles and velocities over all 29 actuated joints. Root pose, text, video appearance, and dataset identity are excluded. This prevents the projection from clustering visual or source artifacts instead of motion structure.
Figure 5: Movens-5M is projected against motion-capture, video-recovered, and robot-motion datasets using only G1 joint kinematics; the panels on the right show fine-grained actions and source videos.
Scale and diversity still do not prove quality. The paper evaluates reference trajectories and SONIC rollouts under a common protocol. Movens-5M reaches 100% joint feasibility, 100% non-penetration, a 0% fall rate, and 5.71% joint-limit violation, with a tracking score of 0.325. Its non-floating score is only 90.05, below PHUMA at 92.05, making foot-ground support the clearest remaining quality weakness. The paper reports unresolved tradeoffs rather than hiding them inside a single aggregate.
Movens-Intent Makes Four Interfaces Comparable
A model can look strong on text, audio, and video benchmarks without possessing a unified intent-to-motion capability if each benchmark uses a different robot, representation, horizon, and evaluator. Movens-Intent contains four tracks of 2,048 condition-motion pairs, for 8,192 examples total. All target Unitree G1, while each condition keeps its natural format and duration. Source identities and near-duplicate motions are disjoint from Movens-5M, and every final example received expert review.
Figure 6: A human can specify the same G1 behavior through language, a spoken command, music, or a video demonstration; the benchmark preserves the natural temporal form of each condition.
The tracks use track-appropriate metrics. Text reports FID, matching distance, retrieval precision, and diversity. Speech reports semantic retrieval and execution quality. Music uses beat alignment, two FID variants, and physical-contact measures. Video reports camera-space and world-space reconstruction errors, root trajectory error, jitter, and foot sliding. Shared execution metrics come from G1 tracking rollouts. The benchmark does not collapse a beat score and a world-space pose error into a fictional single capability number.
| Track | Samples | Condition | Representative metrics |
|---|---|---|---|
| Text | 2,048 | Concrete visible-motion sentence | FID, matching, R@3, diversity, tracking |
| Speech | 2,048 | Full English command from 16 voices | Semantic retrieval, FID, MPJPE, falls, joint limits |
| Music | 2,048 | Complete synchronized music-motion pair | BeatAlign, FID-k, FID-g, physical contact |
| Video | 2,048 | RGB demonstration with complete motion | WA-MPJPE, W-MPJPE, root error, jitter, sliding |
The 40D Interface Connects Generation to Control
Both generated references and observed robot states use the same 40-dimensional frame. The first 29 values are absolute joint positions, followed by two planar root displacements in a gravity-aligned chunk anchor, six root-orientation values, one world height, and two foot-contact indicators. The anchor comes from observed history and stays fixed inside a prediction chunk.
$$\mathbf{x}_{j}=\big[\mathbf{q}_{j},\Delta\mathbf{r}^{\mathrm{anc},xy}_{j},\bm{\rho}^{\mathrm{anc}}_{j},h_{j},c_{j}^{\mathrm{L}},c_{j}^{\mathrm{R}}\big]\in\mathbb{R}^{40}$$The open-source implementation makes the paper's layout unusually concrete. movens_zero/root40.py assigns dimensions 0-28 to IsaacLab-ordered joints, 29-30 to anchored frame displacement, 31-36 to the first two rotation-matrix columns, dimension 37 to world height, and 38-39 to contacts. The heading anchor is constrained to a yaw-only rotation and uses the latest stable projected forward axis, falling back to the lateral axis when no stable forward direction exists.
A six-dimensional rotation avoids the discontinuities of Euler angles while giving the network a representation that can be orthogonalized at decode time. The contact channels expose support transitions directly instead of forcing the low-level tracker to infer every foot event from joint trajectories. They also allow MM-RTC to preserve predicted contacts when the unexecuted tail is rebased into the next anchor.
Generate Thirty Frames, Execute Twenty
Each model call predicts 30 frames but commits only the first 20 to execution. The final ten frames become the overlap for the next prediction. If the full horizon is $H$ and the committed horizon is $H_{\mathrm c}$, the retained prefix is:
$$K=H-H_{\mathrm c}$$The released configuration uses 50 Hz, so this is 0.6 seconds of prediction, 0.4 seconds of commitment, and 0.2 seconds of overlap. Too little overlap creates discontinuities when conditions and controller errors change; too much overlap leaves little room for new input to affect the future. The 20/10 split is an explicit compromise between continuity and responsiveness.
The generation map at step $n$ is:
$$\hat{\mathbf X}_{n}=G_{\theta}\!\left(E_{\psi}(\mathbf{u}_n),\mathbf{s}_n,\mathbf{P}_n,\mathbf{g}_n\right)$$Here $\mathbf{u}_n$ is the multimodal condition, $\mathbf{s}_n$ is the latest robot state, $\mathbf{P}_n$ is the clean motion tail from the preceding chunk, and $\mathbf{g}_n$ is a temporal progress encoding. At the next step, the previous tail is re-expressed using the current anchor and observed state. Predicted foot contacts are retained, the prefix is held clean, and only the unknown suffix is regenerated.
Progress is represented with more than a single normalized index. Each action token receives start position, remaining progress, and trigonometric phase:
$$\mathbf g_{n,j}=[p_{n,j},1-p_{n,j},p_n^{\mathrm{start}},\sin(2\pi p_{n,j}),\cos(2\pi p_{n,j})]$$This helps distinguish identical poses at different points in a long sequence. It also gives the action expert enough information to preserve rhythmic structure while the window advances, which would be difficult with a chunk-local positional embedding alone.
flowchart LR I["Prediction n
30 frames"] --> X["Execute first 20"] X --> O["Controller and robot state"] I --> P["Retain last 10"] P --> R["Rebase in current anchor
preserve contacts"] O --> N["Prediction n+1"] R --> N N --> M["Regenerate unknown suffix"] M --> I
Real-time deployment turns the diagram into a hard systems deadline. sim2real/server.py performs two warm-up inferences and refuses to start if the measured chunk latency exceeds 180 milliseconds. The running loop also assigns 180 milliseconds to each new 20-frame chunk while receiving robot state and control acknowledgements. This is much stricter than reporting an average offline generation time: the server must meet the deadline under live state updates and variable GPU contention.
Flow Matching and Prefix-Aware Training
The action expert learns a velocity field rather than directly regressing a single motion. For target chunk $\mathbf X$, Gaussian noise $\bm\epsilon$, and flow time $\sigma\in[0.05,1)$, a linear probability path is:
$$\mathbf X^{\sigma}=(1-\sigma)\mathbf X+\sigma\bm\epsilon$$ $$\mathbf V^{\star}=\bm\epsilon-\mathbf X$$The network predicts the target velocity from the noisy action, flow time, context tokens, state token, and progress features. The masked, channel-weighted objective is:
$$\mathcal L_{\mathrm{flow}}=\mathbb E\left[\sum_{j,k}M_{j,k}w_k(V_{\theta,j,k}-V^{\star}_{j,k})^2\right]$$The appendix reports relative loss weights of 1, 3, 10, 15, and 0.1 for joints, root displacement, root rotation, root height, and foot contact. The base weight for every channel is $1/40$. Joint height and orientation receive stronger emphasis because small root errors accumulate into unsupported motion, whereas the binary contact channels need a different gradient scale from continuous joints.
MM-RTC requires the action expert to accept any clean prefix length that can arise when a previous prediction is partially executed. During training, a prefix length $\ell$ is sampled for every target. The first $\ell$ frames remain clean, and only the noisy suffix contributes to the loss:
$$\widetilde{\mathbf x}^{\sigma}_{j}= \begin{cases} \mathbf{x}_{j}, & j\lt \ell,\\ (1-\sigma)\mathbf{x}_{j}+\sigma\bm\epsilon_{j}, & j\geq \ell. \end{cases}$$The repository implementation in training/objective.py matches this idea at frame resolution. It samples flow times, random delays up to 11, and rank-independent Gaussian noise, then masks the clean prefix out of the error. If a chosen prefix covers every valid target frame, the code raises an error instead of silently training on an empty suffix. Randomizing the prefix exposes the model to both fresh generation and continuation with different amounts of committed motion.
| Task | Condition | Shared target | Sampling |
|---|---|---|---|
| Text to motion | Language and optional prompt | 40D G1 action chunk | Data-proportional mixture |
| Music to motion | 0.6 s audio window | Motion, root, and contacts | With and without text |
| Speech to motion | Full spoken command | Motion, root, and contacts | With and without text |
| Video to motion | RGB sampled at 8 FPS | Motion, root, and contacts | With and without text |
The seven tasks share one action expert and are sampled in proportion to their training-set sizes. This gives the largest modalities more gradient, but it also forces all conditions through one motion prior. Leave-one-modality-out ablations show that removing text, video, or audio alignment changes performance on the remaining tasks, which is evidence for cross-modal transfer. Because the study is a single leave-one-out design, it does not fully isolate every causal path behind those changes.
Generation Results
On the public OMG text benchmark, Movens-Zero reports an FID of 10.13, R@3 of 85.06%, a 0.29% fall rate, and 10.45% joint-limit violation. On the source-disjoint Movens-Intent text track, it reaches FID 17.65, R@3 62.57%, MPJPE 33.40, and a 0% fall rate. The drop in retrieval advantage on source-disjoint data is informative: the model generalizes, but the gap from the strongest baselines becomes much smaller.
Video conditioning reaches 49.1 mm W-MPJPE100 on Movens-Intent, 16.2% below the strongest baseline. Music reaches BeatAlign 0.61, FID-k 58.43, and a 0% fall rate, but joint-limit violation remains 14.72%. Speech reaches R@3 74.30%, MPJPE 32.63, and a 0% fall rate, with 4.93% joint-limit violation. The strongest system-level evidence is that four very different inputs drive the same G1 action expert; the physical metrics also show that semantic alignment does not automatically solve all hardware constraints.
| Intent track | Semantic or alignment result | Motion or execution result | Main weakness |
|---|---|---|---|
| Text | FID 17.65; R@3 62.57% | MPJPE 33.40; 0% falls | Smaller cross-source margin |
| Video | W-MPJPE100 49.1 mm | 16.2% below strongest baseline | Camera motion remains difficult |
| Music | BeatAlign 0.61; FID-k 58.43 | 0% falls; 14.72% joint limits | Hardware-limit compliance |
| Speech | R@3 74.30% | MPJPE 32.63; 0% falls | Contact sliding and joint limits |
Scaling Data, Capacity, and Curation
The central scaling experiment fixes the XL action expert and trains on nested fractions of Movens-5M. From 1% to 100%, text R@3 rises from 12.72% to 62.57%, speech R@3 from 13.98% to 74.30%, video MPJPE falls from 132.82 to 40.90 mm, BeatAlign rises from 0.23 to 0.61, and tracking score rises from 0.21 to 0.72. All five metrics improve monotonically, including a large gain between 30% and 100%, which is stronger evidence than a comparison between two endpoints alone.
| Data fraction | Unique hours | Text R@3 | Video MPJPE | BeatAlign | Speech R@3 | Tracking |
|---|---|---|---|---|---|---|
| 1% | 122 | 12.72 | 132.82 | 0.23 | 13.98 | 0.21 |
| 10% | 1,223 | 24.92 | 110.55 | 0.38 | 21.76 | 0.36 |
| 30% | 3,669 | 41.80 | 87.90 | 0.45 | 54.34 | 0.50 |
| 100% | 12,231 | 62.57 | 40.90 | 0.61 | 74.30 | 0.72 |
With the full corpus fixed, increasing the action expert from 64.2M to 910.5M parameters reduces validation flow loss from 0.487 to 0.159. Text R@3 rises from 10.30% to 62.57%, speech R@3 from 25.37% to 74.30%, video MPJPE falls from 108.50 to 40.90 mm, and tracking score rises from 0.18 to 0.72. The paper also finds that larger models benefit more from the same data expansion: increasing from 10% to 100% produces a 151.1% relative text gain for XL and 62.5% for S.
| Action expert | Parameters | Flow loss | Text R@3 | Video MPJPE | Speech R@3 | Tracking |
|---|---|---|---|---|---|---|
| S | 64.2M | 0.487 | 10.30 | 108.50 | 25.37 | 0.18 |
| M | 187.9M | 0.392 | 22.96 | 92.80 | 41.25 | 0.24 |
| L | 500.1M | 0.186 | 47.80 | 73.84 | 63.98 | 0.40 |
| XL | 910.5M | 0.159 | 62.57 | 40.90 | 74.30 | 0.72 |
The L-to-XL jump is especially large on video reconstruction and tracking. That supports the paper's joint-scaling argument, but it does not establish an infinite scaling law. Training budget, optimization stability, and task mixture are not held constant across every possible definition of scale. The defensible claim is that larger capacity uses this particular full dataset more effectively, not that parameter count alone determines physical competence.
The curation ablation addresses a different question: with sample count and optimization-token budget fixed, does data quality matter? Moving from raw recovered motion through visual filtering, physical repair, tracking verification, and distribution balancing raises text R@3 from 48.72% to 62.57%, cuts video MPJPE from 126.82 to 40.90 mm, and raises tracking score from 0.41 to 0.72. Tracking verification alone moves the score from 0.58 to 0.67, making executable filtering a direct contributor rather than a preprocessing footnote.
This is a cumulative pipeline, so each row includes all earlier stages. The reported deltas should not be read as fully isolated causal effects of one module. Physical repair can change which samples survive later tracking, and balancing changes category and scene coverage. The broad conclusion is robust: the complete curation path strongly beats raw recovered motion. Exact credit assignment would require factorized module combinations and randomized ordering.
What the Released Code Confirms
The official repository goes beyond a visual demo. It contains the 40D representation, flow-matching training objective, overlapping horizon configuration, simulation bridge, and real-robot server. The code exposes details that matter for reproduction: the exact joint ordering, the yaw-only anchor, the first-two-column rotation representation, the clean-prefix mask, flow-time sampling, and the 180 millisecond deployment deadline.
| Paper mechanism | Code location | Implementation detail |
|---|---|---|
| 40D representation | movens_zero/root40.py:10-45 | 29 joints, 2 displacement, 6 rotation, 1 height, 2 contacts |
| Anchor and state encoding | movens_zero/root40.py:99-185 | Stable forward projection, yaw anchor, frame displacement |
| Flow and prefix objective | training/objective.py:5-54 | Flow time, delay up to 11, clean-prefix mask, channel weights |
| Chunk configuration | configs/inference_xl.json:8-28 | 30-frame horizon, 20-frame execution, 50 Hz |
| Real-time deployment | sim2real/server.py:179-207 | 180 ms preflight and per-chunk deadline |
A sensible reproduction order is offline text inference, then Movens-Intent scoring, then MuJoCo simulation, and only then a real G1. The first step verifies checkpoint loading, the 36D initial pose conversion, 40D normalization, and motion export. The real-robot bridge introduces additional state synchronization, joint-limit, network, and controller risks that should not be mixed into the first debugging pass.
Limitations and Deployment Boundaries
The tracking filter biases the dataset toward motions that current controllers can execute. This is useful for trainability, but it makes airborne, extreme, or object-dependent actions less common. Appendix F contains 30 G1 rollouts that demonstrate capabilities, not a statistically powered hardware success-rate study. Simulation percentages such as 100% joint feasibility and a 0% fall rate do not automatically transfer to different floors, payloads, robot batches, or safety conditions.
Object and scene interaction is another unresolved boundary. Video labels can describe lifting or manipulating an object, and the action trajectory can resemble the visible body motion, but the model is not given object geometry, grasp force, friction, or collision constraints. The paper lists richer interaction, broader embodiments, and deployment studies as future work. Movens-Zero learns a strong body-motion prior; it is not yet a complete task-and-manipulation policy.
Strong camera motion remains difficult. The pipeline separates background and person flow, but occlusion, scale change, and shot transitions can still contaminate SMPL-X recovery. Results on EMDB, RICH, and Movens-Intent show that the method works on evaluated data, not that any handheld video can be converted without review. This caveat matters for teams hoping to use arbitrary social media as an unsupervised corpus.
Music performance has the clearest physical limitation. BeatAlign reaches 0.61 and the fall rate is zero, yet 14.72% of motions violate joint limits. A model can be well aligned to rhythm and still produce references outside the robot's mechanical range. Speech has a lower 4.93% violation rate, showing that the error profile depends on the condition and data distribution rather than monotonically improving with semantic quality.
Real-time inference is constrained by the 180 millisecond per-chunk deadline. The system must share that window across multimodal encoding, action generation, state transmission, scheduling, and controller readiness. The repository checks latency before accepting a live session and enforces a deadline during rollout, but the paper does not provide a broad cross-hardware latency distribution. Deployment teams still need overload tests, network-jitter handling, safe fallback, and watchdog behavior around this node.
Finally, the released repository does not yet state project-specific code and checkpoint licenses, and Qwen2.5-Omni-3B is subject to the Qwen Research License. Public availability is not the same as commercial permission. A product team must resolve licensing, hardware safety, failure recovery, and video-data compliance before treating Movens-Zero as deployable infrastructure.
Independent Assessment
The five-million figure is memorable, but the reusable contribution is the acceptance criterion. A large video-derived corpus is only a motion dataset if it includes source-level split integrity, person-aware temporal filtering, physically plausible human motion, morphology-aware retargeting, and closed-loop execution evidence. Without those stages, scale can amplify visually convincing but untrackable behavior. The fixed-budget curation ablation is therefore more important than the raw dataset headline.
The paper's second strong idea is representation-first system design. The 40D motion frame is shared by learning, offline inference, simulation, and the real-robot bridge. Flow matching and MM-RTC operate on the same chunk language. A new model size or controller changes capacity and control quality without changing what a motion means. This is the kind of interface that makes a foundation-model paper useful to a robotics team, not just impressive as a benchmark entry.
Two questions remain open. First, stronger future trackers may make motions that are currently filtered out learnable, shifting the dataset-quality frontier. Second, a frozen semantic encoder plus an action expert is an efficient condition-to-motion architecture, but Movens-Intent measures conditional motion generation rather than long-horizon planning or object manipulation. The system lacks evidence that it can reason about a cup, a door, a tool, or a multi-stage task where contact changes the environment.
The fairest framing is that Movens-Zero is a data-and-interface paper for executable humanoid motion. It demonstrates that Internet video can be converted into a large robot-relevant distribution, that several input modalities can share one motion prior, and that real-time deployment imposes measurable engineering constraints. It does not prove that a humanoid has acquired general physical intelligence.
Conclusion
Movens-Zero links Internet human video, physical repair, humanoid retargeting, and controller verification into Movens-5M, then trains an understanding-generation-execution stack around a shared 40D G1 representation. Movens-Intent shows that text, video, music, and speech can condition the same motion expert, while the data, model, and curation scaling studies provide a relatively coherent account of where the gains come from.
The work's strength is that semantic alignment and execution feasibility appear in the same experiments. Its unresolved issues are equally concrete: object interaction, hardware success rates, extreme-motion coverage, real-time load, camera-motion robustness, and licensing. The right lesson is not simply to collect more video or train a larger action head.
A humanoid motion foundation model scales only when human video is converted into experience that a robot can execute, track, and reproduce; raw sequence count is the beginning of that pipeline, not the result.
