Skip to content
←Back to Papers

PAPER DEEP DIVE

Vision-language-action (VLA)Robot ManipulationFlow Matching

ScaleVLA: Scaling Vision-Language-Action Models to 552 Billion Parameters

Frontier Vision-Language Models (VLMs) such as GPT-6 Astra have demonstrated remarkable capabilities in robot control, suggesting that broad pretraining provides knowledge valuable for physical action. Yet most Vision-Language-Action (VLA) policies still rely on much smaller VLM backbones, leaving this potential largely untapped. We ask whether frontier-scale representations can provide a stronger foundation for robot learning. We introduce ScaleVLA, which connects a 552B DeepSeek-V4.1-Flash backbone to a trainable action DiT, and systematically compare VLMs from 2B to 552B under a shared training setup. We evaluate three fundamental capabilities of robot foundation models. In sample efficiency, just 10 demonstrations outperform a 2B backbone trained with 50 (87.8% vs. 85.6%). In generalization, LIBERO-PRO success reaches 45.1%, compared with at most 31.3% for smaller backbones. In precise manipulation, real-world charger insertion improves from 4/10 to 9/10 successes, with gains concentrated in contact-sensitive alignment. Despite its scale, ScaleVLA achieves 216 ms median latency per action chunk. Our findings suggest that frontier VLMs can serve not only as reasoning agents, but also as powerful representational foundations for efficient and generalizable robot policies.

Paper identity and source map

ScaleVLA: Scaling Vision-Language-Action Models to 552 Billion Parameters is authored by Ruixi Yu, Dairu Liu, Xuchuan Chen, Zekun Qi, Zekai Li, Zhuo Cao, Luyang Niu, Yingsheng Zhang, Sikai Liang, Ke Fan, He Wang, and Li Yi. The arXiv record points to the complete HTML paper. The public implementation is GalaxyGeneralRobotics/ScaleVLA under Apache-2.0; this analysis was checked against commit c8d24902ecc84a7e2df00db03285095013646066. The authors also provide a project page.

One-sentence assessment

ScaleVLA turns a frozen 552-billion-parameter mixture-of-experts vision-language model into a representation source for continuous robot control, and its strongest evidence appears where demonstrations are scarce, observations shift, or contact alignment is difficult rather than in a clean monotonic scaling law.

Why this experiment needed to exist

Most vision-language-action policies use backbones in the low single-digit billions of parameters. That choice is practical: the backbone must fit the training system, receive camera frames at repeated control intervals, and expose features that a relatively small action network can consume. Frontier vision-language models have meanwhile acquired broader visual recognition, instruction grounding, and spatial reasoning. The paper asks whether those gains survive when the model is no longer allowed to write a long reasoning trace and is used only as a frozen feature source.

The distinction matters. A system that calls a large model as a planner can inspect a scene, produce text, and invoke another controller. ScaleVLA does not perform autoregressive generation during an action chunk. It runs a vision encoder and a causal language encoder once, reads all token features at a fixed layer boundary, and maps them directly into a continuous velocity field. The action head learns from robot demonstrations while the backbone remains frozen.

Prior VLA comparisons often vary architecture, action representation, pretraining data, and robot supervision at the same time. ScaleVLA instead holds the action-side design, initialization recipe, demonstrations, and benchmark budget fixed across several backbone choices. Only the input width of the feature adapter follows the backbone. This does not make the study a pure parameter-count experiment, but it creates a useful comparison of complete pretrained representation systems.

The engineering question is equally important. A dense 552-billion-parameter model would be impractical for repeated robot inference. DeepSeek-V4.1-Flash is a sparse mixture of experts, so only a fraction of experts is activated per token, and ScaleVLA executes only the shallow causal encoder rather than the full language decoder. The paper tests whether this restricted execution path still exposes enough semantic information to justify the hardware cost.

ScaleVLA parameter scale, benchmark gains, and inference latency

Figure 1: The paper's overview of backbone scale, benchmark performance, sample efficiency, and inference latency. The 552B policy leads several averages but requires four H200 GPUs and takes 216 ms per action chunk.

Architecture: from visual tokens to a velocity field

The backbone is DeepSeek-V4.1-Flash-1017. Its vision encoder converts camera images into visual tokens, while the language side processes those tokens together with the instruction. The language stack contains a twenty-layer causal encoder and a twenty-layer decoder. ScaleVLA does not run the decoder. It stops at the input to the attention operation of the next layer and reads the sequence supplied there, with $D_{\mathrm{vlm}}=5120$ channels per token.

The sequence is not pooled into one vector. Every visual and text token remains ordered and carries a modality label. A token-type value of zero denotes text, a positive value identifies a camera, and negative values mark padding. This metadata allows the action transformer to alternate attention over image and non-image tokens and to exclude padding without discarding spatial evidence.

ScaleVLA backbone and action-head architecture

Figure 2: The frozen frontier VLM exports features from its causal encoder. The feature adapter, state and noisy-action encoders, 32-layer action DiT, and action decoder are trained from scratch.

The backbone representation is projected to the action head's common width. The paper describes layer normalization followed by two linear layers and a four-layer self-attention adapter. In compact form, the per-token projection is

$$F=W_{2}\left(W_{1}\operatorname{LN}(X)\right),$$

where $X$ is the backbone token matrix, $W_{1}$ maps from 5120 dimensions into a low-rank space, and $W_{2}$ produces the 2048-dimensional action-head input. The code in scalevla/model/mapper.py:14-23 matches the paper exactly and also returns attention and image masks from the token-type vector:

class FeatureMapper(nn.Module):
    def __init__(self, d_in, d_out=2048, rank=2048):
        super().__init__()
        self.norm = nn.LayerNorm(d_in)
        self.down = nn.Linear(d_in, rank, bias=False)
        self.up = nn.Linear(rank, d_out)

    def forward(self, features, token_type):
        return {
            "backbone_features": self.up(self.down(self.norm(features))),
            "backbone_attention_mask": token_type >= 0,
            "image_mask": token_type > 0,
        }

An embodiment-specific MLP encodes the robot state, and another encodes the noisy action sequence. Future actions are padded to a shared maximum layout and represented as tokens with learned position embeddings. The Sinkhorn-free design is deliberately simple: state and action dimensions stay specific to the robot profile, while masks remove padded dimensions from attention and loss.

The action DiT contains 32 layers that alternate self-attention and cross-attention. Self-attention operates jointly on state tokens and noisy action tokens. Cross-attention reads the adapted backbone features; successive layers alternate between image and non-image token groups. Adaptive layer normalization injects the flow time, and an action decoder emits a velocity for every valid action dimension and future timestep.

The model is therefore split into two responsibilities. Pretrained knowledge lives in a frozen feature source that is expensive to train and relatively cheap to run once per chunk. Control precision lives in action-side modules trained from random initialization on robot data. The paper does not load a pretrained robot expert into the comparison, which makes the action head's learning problem visible across every backbone.

flowchart TB
    O["Cameras + language instruction"] --> V["Frozen DeepSeek V4.1 Flash
vision encoder + 20 causal layers"] V --> F["Layer-boundary tokens
5120D with modality labels"] F --> M["Mapper: LayerNorm rank 2048"] M --> D["32-layer action DiT"] S["robot state"] --> D N["Gaussian noise"] --> D U["flow time"] --> D D --> E["Four Euler updates"] E --> R["Inverse normalization"] R --> C{"deployment profile"} C -->|LIBERO| L["16-step chunk
execute 8 and replan"] C -->|DexJoCo| X["30-step chunk
timestamped async buffer"]

Flow matching and the execution contract

Given an expert action chunk $A$, training samples Gaussian noise $\epsilon$ and a flow time $\tau$. ScaleVLA constructs a straight interpolation from noise to data:

$$A^{\tau}=(1-\tau)\epsilon+\tau A.$$

The network predicts the derivative of that path, $A-\epsilon$, conditioned on adapted features $Z$, robot state $s$, and embodiment identity $e$. The loss is evaluated only over valid action dimensions and valid time positions:

$$\mathcal{L}_{\mathrm{FM}}=\mathbb{E}\left[\left\|v_{\theta}(A^{\tau},\tau\mid Z,s,e)-(A-\epsilon)\right\|_{\mathrm{valid}}^{2}\right].$$

This objective teaches a velocity field over the action manifold rather than asking a single forward pass to regress the clean chunk. At deployment, the policy starts from a noise sample and integrates the predicted field. Four Euler steps are sufficient in the released configuration:

$$A^{(k+1)}=A^{(k)}+\frac{1}{K}v_{\theta}\left(A^{(k)},\tau_{k}\mid Z,s,e\right),\qquad K=4.$$

The backbone is not rerun for each flow step. One backbone pass supplies $Z$, and all four action-head evaluations reuse it. This is the central latency argument: ScaleVLA pays for a frontier encoder once per action chunk, not once per numerical action or denoising step.

State and action dimensions are normalized with empirical percentiles. For a dimension with training percentiles $q_{0.01}$ and $q_{0.99}$, the normalized value can be written as

$$\tilde{x}=\operatorname{clip}\left(2\frac{x-q_{0.01}}{q_{0.99}-q_{0.01}}-1,-1,1\right).$$

The released processor applies the inverse after action prediction. This is not cosmetic: the controller receives physical joint or end-effector commands, so failing to undo normalization would turn a well-trained model into a malformed action stream. The checkpoint stores the statistics, embodiment profile, action layout, and mapper rank so that serving reconstructs the training path.

LIBERO uses 16-step chunks and executes eight commands before requesting another decision. DexJoCo uses 30-step chunks and assigns commands to timestamps while the controller consumes a buffer. The server can generate a replacement chunk during execution. That reduces idle time, but it also means commands may be based on older observations. The appendix correctly separates action-chunk latency from feedback freshness and asks for observation age and buffer-empty intervals alongside controller rate.

The serving implementation in scalevla/serving/policy.py:115-147 follows the same sequence as training: project cached features, obtain an action prediction, trim to the embodiment horizon, and undo the stored normalization.

with torch.autocast("cuda", dtype=torch.bfloat16):
    bo = self.net.mapper(F, TT)
    pred = self.net.head.get_action(
        BatchFeature(data=bo), ai
    )["action_pred"].float()

na = pred[:, :self.horizon].cpu().numpy()
un = self.proc.unapply_action(
    {k: na[..., a:b] for k, (a, b) in self.ranges["action"].items()},
    self.tag,
)

Training uses 64 H200 GPUs, eight nodes with eight GPUs each, a global batch of 640, and policy seed 42. LIBERO trains for 5000 AdamW updates at learning rate $10^{-4}$ with cosine decay and a 5% warmup. DexJoCo trains for 20000 updates. The real-robot policy trains for 40000 steps on 303 episodes and roughly 0.52 million frames, with 26 episodes reserved for validation.

How the evaluation isolates representation quality

The benchmark set spans four different demands. LIBERO measures tabletop language-conditioned control across spatial, object, goal, and long-horizon suites. LIBERO-PRO reuses trained policies without adaptation and perturbs scenes or objects. RoboCasa evaluates kitchen tasks in out-of-distribution layouts with either 30 or 100 generated demonstrations per task. DexJoCo contains five bimanual dexterous tasks and separates object randomization from full randomization.

For each benchmark, the action DiT, common random initialization, data, and optimization budget stay fixed. The feature adapter input width changes with the backbone. The smaller Cosmos-Reason2 and Qwen3-VL models use intermediate token readouts selected by a relative-depth rule, while ScaleVLA uses the twentieth causal-encoder boundary. No policy in the controlled comparison loads pretrained robot-action weights.

BackboneSpatialObjectGoalLongAverage
Qwen3-VL-2B98.498.290.477.891.2
Cosmos-Reason2-2B98.897.683.283.290.7
Cosmos-Reason2-8B99.899.088.687.293.7
Qwen3-VL-32B98.899.493.283.093.6
Cosmos-Reason2-32B98.498.495.283.894.0
ScaleVLA 552B98.698.696.890.296.1

Table 1: Full-data LIBERO success rates in percent. Spatial and object suites are near saturation; the meaningful separation occurs on instruction-grounded and long-horizon tasks.

ScaleVLA reaches a four-suite average of 96.1%. Cosmos-Reason2 rises from 90.7% at 2B to 94.0% at 32B, and Qwen3-VL rises from 91.2% to 93.6%. The two dense families improve with scale, but the 32B Qwen model does not overtake the 32B Cosmos model. The experiment therefore supports a family-level trend plus a particularly strong frontier result; it does not establish that every additional billion parameters causes a fixed gain.

The gap is much larger under perturbation. ScaleVLA averages 45.1% on LIBERO-PRO, compared with at most 31.3% for smaller backbones. It reaches 47.8% on goal-environment perturbation and 56.4% on goal-object perturbation. Object-environment perturbation remains difficult for everyone: ScaleVLA's 31.2% is essentially tied with the best dense result of 32.4%. The pattern is selective. Frontier features help preserve task and object semantics under shift but do not solve every visual distribution change.

Demonstrations per taskGoal 2BGoal 32BGoal 552BLong 2BLong 32BLong 552B
539.461.277.424.037.662.2
1055.083.087.856.066.475.0
2585.290.497.669.683.684.6
5085.695.295.482.483.886.8

Table 2: LIBERO success rates in percent under a fixed budget of 5000 optimizer updates. The 2B and 32B columns are Cosmos-Reason2 models.

The sample-efficiency result is the paper's cleanest argument. With five demonstrations, ScaleVLA reaches 77.4% on Goal and 62.2% on Long. The 2B policy reaches 39.4% and 24.0%. With ten demonstrations, ScaleVLA's Goal score of 87.8% exceeds the 2B policy trained with fifty demonstrations at 85.6%. The frontier backbone has effectively reduced the number of robot trajectories required to reach a given score under this update budget.

The advantage is not uniform at larger data counts. With fifty demonstrations, ScaleVLA scores 95.4% on Goal while the 32B policy scores 95.2%; the Long gap also narrows to 86.8% versus 83.8%. More task-specific data partly compensates for a weaker representation. A fair interpretation is that frontier pretraining shifts the data-performance curve toward fewer demonstrations, not that it makes additional data irrelevant.

RoboCasa shows the opposite data trend. ScaleVLA reaches 30.7% with thirty demonstrations per task and 54.9% with one hundred, versus 25.6% and 45.6% for the best smaller backbones. The absolute advantage grows from about 5 points to about 9 points as generated data increases. This is evidence that strong representation and supervised robot data can complement each other, especially in out-of-distribution kitchen scenes.

SettingRoboCasa Gen-30RoboCasa Gen-100DexJoCo object randomizationDexJoCo full randomization
Best smaller backbone25.645.638.122.1
ScaleVLA 552B30.754.942.124.4

Table 3: Key average success rates in percent. Each DexJoCo setting trains one policy jointly on five bimanual tasks.

DexJoCo narrows the story again. Joint training over Unlock iPad, Hanoi, Assembly, Microwave, and Photograph reaches 42.1% under object randomization and 24.4% under full randomization. ScaleVLA leads the five-task average, with its clearest margins on the three hardest coordination tasks, but full randomization leaves every policy below 25%. The benchmark remains a stress test rather than a solved dexterous-manipulation result.

Simulation tasks from LIBERO, DexJoCo, RoboCasa, and LIBERO-PRO

Figure 3: Simulation environments used in the study. The panels cover tabletop LIBERO tasks, bimanual DexJoCo tasks, RoboCasa kitchens, and the goal, object, and environment perturbations of LIBERO-PRO.

Latency and physical deployment

ScaleVLA has 276 times the total parameters of Cosmos-Reason2-2B, yet its RoboCasa action-chunk latency is 215.8 ms versus 114.7 ms, a factor of about 1.9. Sparse expert activation explains part of the difference, but the architecture matters even more: only the causal encoder runs, and it runs once per chunk. The action head takes 72.1 ms for ScaleVLA versus 67.1 ms for the 2B policy, so the extra backbone work is added to a nearly fixed action-side cost.

SystemGPUsBackbone timeAction-head timeMedian totalReal-robot success
Cosmos-Reason2-2B + Action Head144.7 ms67.1 ms114.7 ms4/10 in-domain; 5/15 OOD
PI0.5 + Action Head1not separatednot separated262 ms8/10 in-domain; 7/15 OOD
ScaleVLA 552B4140.7 ms72.1 ms215.8 ms9/10 in-domain; 8/15 OOD

Table 4: RoboCasa latency and Plug In Charger outcomes. The timing includes preprocessing, backbone inference, feature transfer, four action-head steps, postprocessing, and client-side overhead. The real robot's median round trip is 384 ms per chunk.

The real task asks a bimanual ARX X5 robot to pick up a phone charger, hand it between arms, align the plug, insert it into a power strip, and release it. Training includes 171 full demonstrations and 132 insertion-focused segments, totaling 303 episodes and roughly 0.52 million frames. In-domain trials start near the dominant training position. Out-of-distribution trials translate the charger by up to about 16 cm, rotate it by about 30 degrees, or move the power strip by about 10 cm.

ScaleVLA succeeds in 9 of 10 in-domain trials and 8 of 15 OOD trials. Cosmos-Reason2-2B succeeds in 4 of 10 and 5 of 15. PI0.5 plus the same action head, which has seen large-scale robot pretraining, succeeds in 8 of 10 and 7 of 15. The fact that robot-pretrained PI0.5 narrows but does not close the gap is important: the result cannot be explained only by whether a backbone has encountered robot data before.

The difference concentrates in contact-sensitive insertion. In successful trials, all policies reach the insertion stage at comparable times. ScaleVLA then needs a median 58.1 seconds to align and insert the plug, compared with 81.2 seconds for PI0.5 and 85.9 seconds for the 2B policy. Median completion times are 114.4, 132.0, and 147.1 seconds, respectively. ScaleVLA has the highest chunk latency yet the shortest successful task time.

That inversion is the practical lesson. A policy that spends more computation per decision can still finish sooner if it emits better trajectories and needs fewer corrective motions. For contact-rich manipulation, chunk latency and task completion time measure different parts of the system. Reporting only the former can rank a policy as slower even when it reduces the total time a robot occupies the workspace.

Real bimanual charger insertion task and policy success rates

Figure 4: The real ARX X5 Plug In Charger experiment. ScaleVLA leads both in-domain and OOD success and spends less time in the insertion stage, despite requiring a slower backbone than the two smaller policies.

What the result does and does not establish

The strongest positive claim is representational. A frozen frontier VLM can provide a useful interface for continuous control when its token sequence is preserved and an action head is trained with sufficient care. The gain is largest when the policy must infer task semantics from a few trajectories, when visual conditions move away from the training distribution, or when contact errors demand precise spatial alignment.

The paper also shows why the word "scaling" needs qualification. The total parameter count changes together with architecture, pretraining corpus, active expert routing, vision encoder, token count, and readout layer. Within dense families, larger backbones generally improve LIBERO averages, but rankings are not perfectly monotonic across families. The 552B result is better described as evidence for frontier pretrained features than as evidence for a universal scaling exponent.

There is a useful distinction between total and active parameters. Calling this a 552B policy is accurate for checkpoint size, but it can mislead readers into assuming that every token traverses all 552B parameters. The sparse mixture-of-experts design activates a subset of experts, and the decoder is not executed. A deployment analysis should therefore report total parameters, active parameters or expert routing, input token count, GPU topology, and wall-clock time together.

The study does not claim that a larger backbone is always worth the hardware. Four H200 GPUs are a different deployment class from a single-GPU small model. Model loading, interconnect traffic, failover complexity, batching behavior, and memory headroom are not captured by one median latency. For a short task where a small policy is already reliable, the frontier representation may not repay this operational burden.

Limitations and unresolved risks

The appendix states that each policy is trained with one seed. Evaluation uses many episodes and initial states, but the training-side variance is not repeated across seeds. Small differences in RoboCasa Gen-30 and DexJoCo full randomization could shift under a different initialization or data order. The results are meaningful for research direction, yet they are not a production selection guarantee.

Model-family confounding is explicit. The comparison controls the action head, data, and update budget, but the backbones differ in architecture and pretraining. The smaller language models use a relative-depth feature readout, while ScaleVLA reads a boundary special to DeepSeek. The same training recipe may also be more favorable to one feature geometry than another. Parameter count alone cannot explain the complete result.

The sample-efficiency study fixes optimizer updates at 5000. Its normalization statistics are still computed from the full training set, so the reported demonstration count refers to trajectories used for policy updates rather than all information used during preprocessing. The appendix additionally shows that the 2B policy keeps improving with more updates. Fixed-budget efficiency should therefore not be rewritten as performance after both models converge.

The real-robot evidence covers one task and one robot platform. That task is substantial because it combines grasping, handover, alignment, insertion, and release, but it does not represent mobile manipulation, multiple object categories, long-horizon autonomy, or human-robot interaction. OOD charger insertion still fails in 7 of 15 trials, and all in-domain failures occur during insertion. Contact-rich generalization remains incomplete.

The action-side architecture is another boundary. ScaleVLA adopts a GR00T N1.7-style action DiT and flow matching. A tokenized action model, diffusion policy with a different denoising schedule, or a larger pretrained action expert could change the ranking. The paper establishes a strong result for this particular interface, not a proof that every VLA should use a frozen frontier feature source.

Finally, the task-level success results and the latency measurements are not a full economics study. The 216 ms RoboCasa median comes from a carefully specified batch-one timing setup, while the real-robot median is 384 ms. Neither number states how many robots one serving cluster can support, how the model behaves under bursty concurrent requests, or what uptime is required in a commercial deployment. Those are necessary follow-up measurements for anyone considering the architecture in production.

Conclusion

ScaleVLA demonstrates that a 552B sparse vision-language backbone can support continuous robot control without autoregressive reasoning. Its layer-boundary features retain image and language tokens, a low-rank mapper aligns them with the action head, and four flow-matching integration steps produce the next command chunk. The empirical gains are concentrated where pretrained semantics matter most: LIBERO reaches 96.1% average success, LIBERO-PRO reaches 45.1%, RoboCasa reaches 54.9% at Gen-100, and the real robot improves from 4/10 to 9/10 in-domain.

The result is persuasive precisely because it is bounded. The 552B figure is a total sparse parameter count, not a promise of monotonic scaling. Four H200 GPUs and imperfect OOD insertion remain real costs. What the paper does establish is a concrete bridge from frontier vision-language representations to fast enough action generation, along with a controlled methodology for asking where that bridge is worth using.

The value of frontier scale is not a larger number in a title; it is the ability to preserve useful spatial meaning from scarce demonstrations to shifted scenes, provided the interface, latency, hardware, and failure modes are measured as carefully as the headline score.

Related Papers

Training Reasoning Models in Simulation to Natively Use Real Robots

Training Reasoning Models in Simulation to Natively Use Real Robots

The authors train an open-weight 27B reasoning model to control real robots through simulation-only reinforcement learning and no demonstrations. PEAR uses robot-use reinforcement learning with next end-effector pose prediction; LIBERO success rises from 1.6% to 74.4%, and the trained model reaches 60% success on a real dual-arm robot.

Jiaming Tang, Zhuoyang Zhang, Junxian Guo · 216 views · Nov 13, 2026
Robot-Use AgentsReinforcement Learning
Nov 13, 2026
From Egocentric Human Video to Robot Manipulation in the Era of Embodied Intelligence

From Egocentric Human Video to Robot Manipulation in the Era of Embodied Intelligence

Egocentric human video records close-range hand-object interaction and task temporal structure at a scale that could supplement costly robot demonstrations. This survey defines Ego2Robot learning as acquiring or adapting manipulation policies from such video with optional robot grounding, and organizes it into explicit embodiment transfer, explicit action and task supervision, egocentric representation pretraining, and latent action or dynamics modeling. It compares human-video, robot-demonstration, and synthetic or simulated resources, then examines physical feasibility, agent coordination, in-context adaptation, evaluation, and closed-loop predictive control.

Yi Wang, Jingqiu Wu, Lei Yao · 234 views · Nov 10, 2026
Egocentric VideoRobot Manipulation
Nov 10, 2026
AIRoA-0: Learning Long-Horizon, Precise Cross-Embodiment Manipulation from Handheld Data

AIRoA-0: Learning Long-Horizon, Precise Cross-Embodiment Manipulation from Handheld Data

AIRoA-0 turns 16,151 hours of bimanual handheld-gripper demonstrations into a vision-language-action policy that uses anchored tool motion, selective wrist perception, stage prediction, proprioceptive memory, and streaming decoding to carry one set of weights across four robot embodiments.

Yueh-Hua Wu, Kei Ota, Hanlong Li · 310 views · Nov 2, 2026
Bimanual ManipulationHandheld Demonstrations
Nov 2, 2026
UniWAM: Unified World-Action Model

UniWAM: Unified World-Action Model

Vision-language-action models benefit from the understanding and reasoning capabilities of pretrained vision-language models, but action-only supervision provides limited grounding in world dynamics. Conversely, world-action models inherit spatiotemporal priors from video generation models, yet remain limited in semantic understanding and reasoning under distribution shifts. We introduce UniWAM, a unified architecture that integrates a physical reasoner, a world generator, and an action predictor to jointly learn semantic understanding of the physical world, visual generation, and action prediction. To ensure the quality of the training data, we developed a rigorous data cleaning and annotation pipeline for both human egocentric data and robot data. To adapt the vision-language component to embodied tasks while preserving its inherited language capabilities, we represent low-level actions in natural language and introduce a pre-training recipe that assigns complementary supervision from visual question answering (VQA) data, human egocentric data, and robot demonstrations to the appropriate model components. During post-training, future visual noise augmentation reduces reliance on precise future predictions, while history-conditioned flow matching uses encoded action history to initialize action generation. Together, these designs significantly reduce denoising steps while maintaining performance. UniWAM achieves state-of-the-art (SOTA) performance across multiple evaluations, including in-distribution performance, robustness, generalization, instruction following, and long-horizon task execution. Furthermore, we uncover a log-linear scaling law of unified human-robot co-training, demonstrating the effectiveness of large-scale pre-training on a mixture of human and robot data.

Wenxuan Song, Jiayi Chen, Jingbo Wang · 253 views · Oct 1, 2026
World-action modelsVision-language-action (VLA)
Oct 1, 2026

As an Amazon Associate, we earn from qualifying purchases.