AIRoA-0: Learning Long-Horizon, Precise Cross-Embodiment Manipulation from Handheld Data

1. Why Robot Data Remains the Bottleneck

Vision-language-action models have made the interfaces between perception, language, and control look increasingly uniform. A camera supplies an observation, a sentence states the task, and an action expert emits a trajectory. The difficult part is no longer only the architecture. It is the supply of demonstrations that can cover many objects, scenes, contact conditions, and task durations without making every hour of data depend on the availability of a robot and an expert teleoperator.

Robot-collected datasets grow with the number of robots, their uptime, the number of trained operators, and the reliability of the control stack. That coupling is especially costly for bimanual manipulation. Two arms must be commissioned, cameras must be mounted, control latency must be measured, and each collection site must preserve a usable setup. Scaling the number of demonstrations by ten may therefore require a comparable expansion in physical infrastructure rather than only more annotators.

Handheld grippers attack this constraint by moving the recording interface from the robot arm to the human hand. Each hand holds a small instrumented gripper with a wrist camera and tracked pose. The person performs the task naturally, while the system records what each gripper sees, where each gripper moves, and how widely it opens. The same end effector can later be mounted on a robot. Data collection no longer requires a robot to execute the task, and one human can move between tables, rooms, and objects without reconfiguring an entire arm.

This idea is not new in itself. UMI established that handheld demonstrations with wrist cameras can transfer to robot manipulation, and YUBI developed an open hardware and software stack for bimanual collection. The open question addressed by AIRoA-0 is whether this data source remains useful at foundation-model scale. A short pick-and-place demonstration can tolerate large mismatches between human and robot execution. A minute-long maintenance sequence, a narrow insertion, or a handover between two grippers exposes every weak link: coordinate mismatch, visual distractors, ambiguity about task progress, discontinuity between action chunks, and missing corrective behavior.

AIRoA-0 treats those issues as consequences of the data contract rather than as unrelated engineering defects. Because the camera and gripper move together, visual and action coordinates can remain tool-centric across embodiments. Because each wrist view is local, the policy needs a way to discard irrelevant scene content. Because a local frame often cannot identify which step of a long task is underway, the policy needs explicit progress estimation. Because demonstrations are collected without a robot, there are no robot failures, no teleoperation corrections, no reinforcement learning, and no simulated rollouts. The system must therefore gain robustness at deployment from the observation stream, not from exposure to robot-collected recovery data.

The paper reports two forms of evidence that matter independently. First, on ten bimanual Franka tasks at a site absent from training, the policy reaches a mean success rate of 78 percent, compared with 27 percent for the strongest of four post-trained baselines. Second, the same task weights run on a bimanual Franka, the TX-G2 humanoid, OpenArm, and a pair of UR industrial arms without embodiment-specific retraining. The first result tests task competence and generalization. The second tests whether defining state and action at the gripper actually creates a portable control interface.

2. Data: A Gripper-Centered Collection System

YUBI is the physical anchor of the study. It is finger-driven: closing the demonstrator's hand closes the gripper jaws. A camera sits at the wrist, and an external tracking system measures the gripper pose in six degrees of freedom. Each demonstrator holds two units, so every episode contains the two wrist images, the trajectories of both tools, their relative spatial arrangement, and both opening commands. The recording is therefore not a human hand video retargeted after the fact. It is a record of a tool that can later become the robot's end effector.

The released corpus contains 16,151 hours of bimanual demonstrations, divided into 10.65 million subtask segments with an average duration of about 5.5 seconds. At 30 Hz, the data contain 1.74 billion time steps. It spans more than one hundred tasks, including pick-and-place, sorting, assembly, insertion, cable routing, opening and closing, tool use, and handovers. The scale is important, but the language structure is equally consequential: every segment carries both a task instruction and a subtask instruction.

The task instruction names the whole job. The subtask instruction names the current short step. For robot-vacuum maintenance, for example, the task instruction might ask the robot to replace the vacuum cleaner parts, while the subtasks separately describe removing the first used mop pad, attaching a new pad, or taking out the brush. At deployment, AIRoA-0 receives only the task sentence. During pre-training, it also learns to predict subtask text from the observation. During post-training, consecutive subtasks are grouped into stages that explicitly condition the action expert.

Corpus property	Reported value	Role in AIRoA-0
Bimanual demonstrations	16,151 hours	Large-scale source that does not require robot execution
Subtask segments	10.65 million, 5.5 seconds on average	Fine-grained language supervision and stage construction
Time steps	1.74 billion at 30 Hz	Preserves contact timing and short motion transitions
Visual observations	Two wrist cameras	Consistent tool-mounted views, but no global scene camera
Proprioception	Pose and opening of both grippers	Eleven-dimensional relative state
Action stream	Translation, 6D rotation, and opening	Twenty-dimensional bimanual action at each step

Figure 1, wrist-view fragment: one crop from the paper's overview figure. It shows the limited field of view available during both handheld collection and robot deployment.

The wrist-only setup is the source of both portability and difficulty. It is portable because the camera remains fixed relative to the gripper when the gripper moves from a person to a robot. It is difficult because the image may omit the full table, the destination container, or an obstacle approaching from outside the local view. Two wrist images can be spatially consistent while still being semantically ambiguous. The design of the model must therefore recover task progress from motion history and language rather than assume that one frame contains a complete scene description.

Another limitation follows directly from how the data are produced. Human demonstrations show successful intent, not the robot's motor limits, controller tracking errors, gripper slips, or collision recovery. The absence of robot data is a deliberate scaling choice, but it also means the policy cannot imitate a corrective response it has never seen. Closed-loop behavior has to emerge from action prediction conditioned on current observations, while the robot's low-level controller remains responsible for execution safety.

3. Pre-Training: A Shared Coordinate System for People and Robots

AIRoA-0 starts from the π0.5 architecture: a SigLIP So400m/14 image encoder, a Gemma 2B vision-language backbone, and a 300M-parameter action expert trained with flow matching. The public π0.5 base checkpoint initializes the model. AIRoA-0 then changes the robot interface, adds a visual budget gate, and trains the backbone and gate on the handheld corpus for 300,000 steps with batch size 2,048. The input consists of two 224 by 224 wrist images, one instruction, and an 11-dimensional proprioceptive state. Pre-training predicts a 32-action chunk on a 20 Hz grid.

3.1 Anchored actions remove references to a robot body

The central representational choice is to define every action relative to the gripper pose at the moment of observation. Let $T_i\in SE(3)$ be the tool-center-point pose at time step $i$. The $k$-th action for a chunk of length $H$ is the future pose expressed in the frame of the current pose:

$$a_k=T_i^{-1}T_{i+k},\qquad k=1,\dots,H.$$

Each action is represented by 3D translation, 6D rotation, and one gripper-opening command. A bimanual action therefore contains 20 numbers. The state is not a robot-base pose or a set of joint angles. It is the pose of the right gripper in the left gripper's frame, followed by both openings, for a total of 11 numbers. Nothing in this representation asks which arm carries the tool, where the robot base stands, or how many joints the arm has.

The choice borrows the relative trajectory representation used in UMI, but its role in AIRoA-0 is broader than data conversion. It defines a single semantic interface for human-held and robot-held tools. A translation that moves one gripper toward the other means the same thing on a Franka, a humanoid, and an industrial arm once the tool is mounted. The policy predicts a target, not an increment relative to the previous predicted action. This reduces accumulation of prediction error within a chunk and lets each robot controller follow the same target sequence at its own control rate.

The portability claim still has a precise boundary. It applies to robots that can accept the same YUBI grippers and wrist cameras. AIRoA-0 does not solve general embodiment transfer between arbitrary end effectors, camera placements, or hand morphologies. The common frame is the instrumented tool, not an abstract robot-agnostic body. This distinction is essential when interpreting the four-robot experiments.

3.2 A visual budget selects evidence instead of cropping objects

A wrist camera records a large amount of information irrelevant to the current manipulation step. Table texture, room lighting, background objects, and nearby clutter can differ dramatically between collection sites. If the action expert treats these details as stable context, cross-site deployment becomes a test of memorized backgrounds rather than manipulation skill. AIRoA-0 applies the See Less, Specify More idea through a learned visual evidence budget before image tokens enter the backbone.

For an image token $z_p$ and a pooled prompt summary $q$, a small network $h$ produces a keep probability $m_p$. The token is softly scaled rather than hard-deleted:

$$m_p=\sigma\!\left(h(z_p,q,z_p\odot q)/\tau\right).$$ $$\hat z_p=\left(\gamma+(1-\gamma)m_p\right)z_p.$$

Here $\tau$ controls the sharpness of the gate, and $\gamma$ is a floor that preserves a trace of every token. The floor matters because a zeroed token would remove the possibility of recovering evidence later. To prevent the gate from assigning high weight everywhere, a regularization term constrains the average keep probability over the $P$ tokens of a view:

$$\mathcal{L}_{\text{budget}}=\left(\frac{1}{P}\sum_{p=1}^{P}m_p-\rho\right)^2.$$

The gate is trained by the control objective, not by object masks or human saliency labels. In visualizations across 25 subtasks, high weights concentrate around the grippers and manipulated objects, while table and background regions recede. This does not prove that every selected token is causally necessary. It does show that the budget learns a task-conditioned evidence selection that is consistent with the action objective. The mechanism is also not a promise of compute reduction: tokens remain in the sequence, and the budget acts as a soft weighting signal.

3.3 Two levels of language shape the same representation

During pre-training, each sample is conditioned on either the full task instruction or the current subtask instruction. This gives the model experience operating at both temporal levels. More importantly, when the prompt is only the task instruction, the language backbone is asked to predict the subtask text as an auxiliary target. The action expert never receives that auxiliary prediction during pre-training. The prediction exists to force the shared representation to encode where the task stands.

Recovering a subtask from two wrist views and a high-level instruction is an information bottleneck. The model cannot answer by copying the prompt, because the prompt omits the current step. It must combine object appearance, gripper configuration, and recent spatial relation into a progress-sensitive representation. The action expert reads the same underlying features. This is why subtask supervision can improve action control even though it is not a separate high-level controller at that stage.

The split also anticipates the later deployment interface. AIRoA-0 is supposed to receive one sentence at the beginning and continue through many stages. Pre-training teaches it to understand step-level language. Post-training converts that capability into a compact stage classifier whose output can be smoothed over time. The same semantic channel is reused instead of introducing an unrelated module that the action expert must learn from scratch.

4. Post-Training: Estimating Progress and Remembering Motion

4.1 Stage classification resolves local-view ambiguity

A frame containing a gripper and a green part may be consistent with picking, transferring, approaching a container, or placing. The local wrist view often cannot resolve these alternatives without knowing what happened earlier. AIRoA-0 creates task-specific stages by grouping consecutive subtask instructions. A stage head reads the backbone representation and predicts the current stage. The stage prediction is then supplied to the action expert in the same language-conditioning form used during pre-training.

The stage head is trained with a categorical cross-entropy loss. For a batch of observations $o_b$ and stage labels $y_b$, the objective is:

$$\mathcal{L}_{\text{stage}}=-\frac{1}{B}\sum_{b=1}^{B}\log p_\theta(y_b\mid o_b).$$

This is a useful reduction of an otherwise open-ended language problem. A subtask instruction can be long and syntactically variable, but post-training needs only distinguish the current stage within a known task. The demonstration labels already contain the necessary supervision, so no extra annotation is required. The classifier also gives the inference system a probability vector that can be smoothed instead of switching discretely on every frame.

The tradeoff is that stage boundaries now become part of the control interface. If two semantically different steps are grouped into one stage, the stage head cannot communicate the distinction to the action expert. If a long task has too many fine stages, classification becomes more sensitive to local visual ambiguity. The paper reports the grouping used for its tasks, but it does not establish an automatic or universally optimal boundary-selection rule.

4.2 Proprioceptive memory restores short temporal context

A single wrist observation does not indicate whether a gripper is approaching an object or has just left it. AIRoA-0 stores the last ten measured proprioceptive states, each with its age in seconds. Four learned queries conditioned on the instruction and current state attend over this history and produce four memory tokens. These tokens join the prompt and travel with the rest of the model context.

The memory is designed for robots with different query rates. Training samples the history at 5, 8, 10, 12, and 15 Hz and encodes the true age of each state instead of treating one index as a fixed amount of time. The policy can therefore interpret the same number of stored states at a slower or faster rate. History is dropped entirely with probability 0.25, which prevents the model from making memory indispensable and trains a visual-action path that remains viable when the history is unavailable.

At deployment, the history contains measured robot states rather than the policy's own predicted states. This is a conservative design choice. Feeding predictions back into memory could create a self-confirming sequence during occlusion or model error. The memory also does not contain visual frames, force, tactile readings, or a latent scene map. Its role is limited to short motion context, not full task state estimation.

4.3 The visual gate becomes both an embedding and an attention prior

Post-training keeps the pre-trained visual budget gate and applies it at two interfaces. First, the differentiable gate scales image embeddings before the action forward pass, as in pre-training. Second, a detached and budget-projected copy of the keep probabilities is converted into a log prior on attention from action tokens to image tokens. If $\tilde m_p$ denotes the projected probability, the prior is:

$$b_p=\log\!\left(e^{-\kappa}+(1-e^{-\kappa})\tilde m_p\right).$$ $$A=\operatorname{softmax}\!\left(\frac{QK^\top}{\sqrt d}+\beta b\right).$$

With $\kappa=4$, the prior lies between roughly -4 and 0. The attention strength $\beta$ rises from 0 to 1 over the first 500 updates. The embedding path remains differentiable and lets action gradients shape the gate. The attention path stops gradients, projects the probabilities to satisfy the budget, and supplies a stable selection bias where the action expert reads visual evidence.

This dual use is more specific than simply adding an attention mask. The gate has to be useful in the representation that the language backbone sees and in the attention matrix that the action expert uses. The paper reports both as active at run time. The public code exposes the two paths separately, which is also a useful safeguard against accidentally doubling gradients through the same logits.

4.4 One recipe, four loss terms, and less than four hours

The post-training objective combines the flow-matching action loss with the visual-budget, stage-classification, and subtask-language terms:

$$ \begin{aligned} \mathcal{L}_{\text{post}}={}& \mathcal{L}_{\text{act}}+\lambda_b\mathcal{L}_{\text{budget}} \\ &+\lambda_c\mathcal{L}_{\text{stage}} +\lambda_s\mathcal{L}_{\text{sub}}. \end{aligned} $$

Every reported task uses the same recipe: 20 anchored actions at 20 Hz, visual budget 0.9, temperature 0.805, a gate floor annealed from 0.1 to 0.02, budget weight 0.1, stage weight 0.1, and subtask weight 0.01. Training runs for 20,000 updates with batch size 64, AdamW, gradient clipping at 1, a 500-step warmup followed by a constant learning rate of $5\times10^{-6}$, and an EMA with decay 0.99. A task finishes in under four hours on one node of eight H200 GPUs.

Post-training element	Configuration	Purpose
Action chunk	20 anchored actions at 20 Hz	Short execution horizon with tool-relative targets
Visual budget	0.9 mean, temperature 0.805	Select task-relevant wrist evidence
Auxiliary losses	Stage 0.1, subtask 0.01	Represent task progress and step semantics
Optimization	20,000 updates, batch 64, AdamW	One fixed recipe across evaluation tasks
Learning rate	$5\times10^{-6}$ after 500 warmup steps	Stable adaptation from pretrained weights
Deployment weights	EMA decay 0.99	Reduce checkpoint noise
Hardware	Eight H200 GPUs, under four hours	Practical task adaptation cost

The common recipe is a stronger experimental control than it may first appear. The four baselines are searched per task over action horizons, frequencies, and 60,000 to 100,000 updates. AIRoA-0 receives none of that search. The comparison therefore asks whether a pre-trained handheld-data model can be adapted with a stable default, not whether a team can spend more compute to tune the new model per task.

5. Inference: From Probabilities to Continuous Motion

Long-horizon behavior cannot be delivered by post-training alone. The policy is queried repeatedly while the scene changes. A stage classifier can be wrong on one frame, and independently generated action chunks can produce visible seams. AIRoA-0 adds an inference harness with two parts: exponential smoothing of stage probabilities and non-uniform streaming decoding of action chunks.

The stage filter maintains an exponential moving average of the predicted class probabilities and conditions the action expert on the most probable averaged stage. An isolated error has little effect, while persistent evidence of a real transition gradually moves the average. Because the filter operates on probabilities rather than a fixed number of queries, it does not need to be retuned when the controller changes its query frequency. This is a lightweight solution, but it cannot recover a stage that remains visually ambiguous for a long interval.

Standard flow-matching inference starts every action chunk from noise and performs ten action-expert evaluations before execution. Consecutive chunks are generated independently, so their boundary may not be smooth. Streaming decoding instead maintains a buffer of chunks at staggered noise levels. The first chunk is nearly clean and ready to execute, while later chunks remain close to noise. At each new observation, the whole buffer is denoised one level, the first chunk is executed, and fresh noise enters at the end. Causal attention lets later chunks condition on earlier ones.

AIRoA-0 changes the schedule of that buffer. With two slots and levels $(0.1,1)$, a newly entering chunk takes a large step from pure noise to 0.1 while the chunk ahead is already close to execution. The next query gives it only a small correction from 0.1 to 0 using the newest observation. This concentrates most of the denoising early and reserves the final update for scene-responsive refinement.

For training, the noise time of slot $k$ is sampled within its own interval:

$$t_k=\ell_{k-1}+(\ell_k-\ell_{k-1})u.$$ $$u\sim\mathrm{Beta}(1.5,1).$$

The Beta distribution favors larger values of $u$, exposing training to times closer to the noisy end of each interval and matching the distribution encountered by a streaming buffer. The same recipe covers steady-state buffers and cold starts. The decoder adds no parameters; the sampled noise level enters the action expert through its existing flow-time conditioning.

flowchart LR
    A[Bimanual YUBI demonstrations] --> B[Wrist views  gripper pose  task and subtask text]
    B --> C[pi 0.5 vision-language backbone]
    C --> D[Visual evidence gate]
    C --> E[Subtask prediction]
    C --> F[Post-training]
    F --> G[Stage classifier]
    F --> H[Proprioceptive memory]
    G --> I[Stage probability smoothing]
    H --> J[Anchored action expert]
    I --> J
    J --> K[Non-uniform streaming decoder]
    K --> L[Franka  TX-G2  OpenArm  UR]

Method-specific data flow. The paper's four system layers meet at the action expert: language defines the task and stage, visual gates select wrist evidence, memory supplies recent motion, and streaming decoding turns repeated predictions into one continuous trajectory.

In steady state, each query evaluates the action expert twice rather than ten times. The paper reports about 60 ms per query on an H200. This number describes model inference, not complete robot cycle time, including camera capture, transport, controller interpolation, and safety checks. It is nevertheless evidence that the streaming design reduces the cost of repeated denoising while preserving responsiveness to visual change.

6. What the Public Code Verifies

The public repository is a post-training release, not a complete reproduction of the 16,151-hour pre-training pipeline. Its README explicitly asks the user to supply pre-trained weights and a LeRobot dataset. This boundary is important. The repository can verify how the objective is assembled, how the visual gate is projected, how stage supervision is computed, and how non-uniform flow times are sampled. It does not by itself prove the filtering, outlier detection, retiming, distributed pre-training, or full data-processing story in the paper.

The objective assembler in src/post_training_losses/objective.py combines action, visual-budget, anchor, stage, and subtask terms. The paper's main equation groups the conceptual terms more compactly; the code exposes an additional anchor regularizer with default weight 0.0175. The implementation also makes clear that stage and subtask losses arrive as already weighted scalars, while budget terms are weighted inside the assembler. This is a code-level detail that should not be confused with a contradiction in the high-level objective.

total = action + budget + anchor
total = total + stage
total = total + pa

The visual-budget implementation in src/post_training_losses/visual_budget.py has two deliberately separate interfaces. Differentiable sigmoid gates scale image embeddings. A detached FP32 projection uses 25 bisection steps to find a non-negative logit shift that keeps the mean keep probability under budget. The resulting probabilities become the attention prior. This matches the paper's claim that the gate is used for representation and again for action-to-image attention, but it also shows that the attention path is not simply another gradient path through the same logits.

gates = jax.nn.sigmoid(logits / tau)
effective = floor + (1.0 - floor) * gates
prior = log(exp(-4.0) + (1.0 - exp(-4.0)) * probability)

The non-uniform flow sampler in src/post_training_losses/nuf.py constructs cold-start buffers, samples time with Beta(1.5, 1), and builds packed causal attention. The attention permits chunks to condition on earlier chunks while keeping independent training buffers separate. Stage supervision is implemented as ordinary cross-entropy in vendor/openpi/src/openpi/models/discrete_stage.py. The repository therefore supports the paper's most concrete algorithmic claims even though it does not expose full pre-training reproducibility.

7. Main Evaluation: Ten Tasks and Four Robots

The principal benchmark contains ten bimanual Franka tasks at a site where none of the training data were collected. Every trial begins with one task instruction. Each success rate averages 15 trials, with partial credit for partially completed work. The tasks range from towel folding and letter-block spelling to color sorting, parts kitting, robot-vacuum maintenance, a fuel bracket, a pencil case, cup and orange pyramids, and an earbud case. This spread matters because a single success metric can otherwise hide whether the method only works on pick-and-place.

The baselines are π0.5, Xiaomi-Robotics-1 (XR-1), GR00T N1.7, and MolmoAct2. Each starts from a public pretrained checkpoint and is post-trained on the same task demonstrations. Each baseline receives a per-task search over action horizons 4 through 16, several action frequencies, and update budgets from 60,000 to 100,000, with the best result reported. AIRoA-0 uses one recipe for every task.

Task	Demo hours	AIRoA-0	π0.5	XR-1	GR00T N1.7	MolmoAct2
Towel folding	3.3	73	22	21	0	8
Letter blocks	3.4	84	13	0	24	0
Orange pyramid	3.4	90	2	52	13	28
Parts kitting	4.0	83	30	13	7	12
Color sorting	4.1	97	45	39	20	19
Robot-vacuum maintenance	5.3	73	1	14	1	3
Fuel bracket	6.3	93	34	33	47	11
Pencil case	8.8	73	36	31	15	23
Cup pyramid	17.9	80	71	11	23	36
Earbud case	25.8	30	13	18	10	3
Average		78	27	23	16	14

The aggregate result is a 51-point gap over the strongest baseline, but the per-task pattern is more informative. AIRoA-0 is highest on all ten tasks, so the mean is not carried by one exceptional task. Baselines have different strengths: π0.5 reaches 71 on the cup pyramid, XR-1 reaches 52 on the orange pyramid, and GR00T N1.7 reaches 47 on the fuel bracket. The gap remains substantial across folding, sorting, assembly, maintenance, and insertion.

Figure 1, wrist-view fragment: a green part between the two grippers. The caption identifies only this local view, not the full composed overview figure.

The task-level outliers expose the limits of a single number. Earbud-case handling uses 25.8 demonstration hours yet reaches only 30 percent, the lowest AIRoA-0 result. More data did not overcome the task's precision, occlusion, or object-state demands. The fuel bracket uses 6.3 hours and reaches 93 percent. Task structure and whether wrist views expose the needed evidence can dominate data quantity, so a scaling law for average loss should not be read as a guarantee of uniform success improvement.

The dynamically changing-scene experiment tests closed-loop response without introducing disturbance demonstrations. During color sorting, a person pushes the bins after the robot has begun carrying a green part. The red bin moves under the gripper. AIRoA-0 redirects to the new green-bin location and places the part correctly. Later, the green and yellow bins are swapped, and it again follows the current visual location. This behavior is consistent with visual action conditioning and the streaming decoder's final observation update, but the paper presents it as a qualitative demonstration rather than a controlled distribution-shift benchmark.

Cross-embodiment evaluation provides a second kind of evidence. AIRoA-0 performs four tasks on TX-G2, three on OpenArm, and two on paired UR arms, using the same task weights with no robot-specific retraining or adjustment. None of the training data were collected on those robots. The result is meaningful because the control interface is tool-centered, not because the model has inferred an arbitrary robot morphology from pixels.

Figure 1, wrist-view fragment: a maintenance scene in which precise contact and task progress must be inferred from a local tool view.

The scope of this transfer should remain explicit. All tested robots mount the same YUBI gripper and cameras. The paper does not establish transfer to a different parallel jaw, a three-finger hand, a moving camera mount, an eye-in-hand configuration with altered geometry, or an arm whose controller cannot follow the requested tool trajectory. “Embodiment-agnostic” here means compatible arms behind a shared instrumented end effector, not embodiment independence in the broadest robotics sense.

8. Scaling: More Handheld Hours Still Matter After Post-Training

The authors test whether the corpus behaves like a useful scaling axis rather than a fixed collection stunt. They train the same architecture, batch size, and schedule on 50, 25, 12.5, and 6.25 percent of the pre-training data. Held-out action loss is measured on 3,072 action chunks. The full corpus produces the lowest loss and continues improving longer than the smaller runs.

The more demanding question is whether this advantage survives task adaptation. The authors take the 100,000-step checkpoint from each run and post-train it with identical settings on the same 16 task sets. These post-training demonstrations were not sampled during pre-training. After the first thousand updates, the gap between data shares no longer narrows, and the smallest pre-training share has the highest loss on every one of the 16 sets. More pre-training data is therefore not merely delaying convergence to the same post-trained solution.

Averaging the last 1,000 updates produces a regular power-law fit with $D$ measured in hours:

$$L=0.063D^{-0.035},\qquad R^2=0.98.$$

Within the tested range of roughly 1,000 to 16,000 hours, doubling the pre-training corpus reduces post-trained action loss by about 2.4 percent. The exponent is small, so the curve shows diminishing returns. The practical interpretation is still important: across many tasks, each additional doubling produces a measurable average improvement rather than saturating immediately.

Scaling question	Observation	Interpretation
Pre-training trajectory	Full data gives the lowest held-out loss	More handheld hours improve the learned action model
Post-training persistence	Advantage remains after identical task adaptation	Smaller pre-training cannot simply catch up with more updates
Across task sets	Smallest share is worst on all 16 sets	The trend is not caused by one benchmark outlier
Fitted relation	$L=0.063D^{-0.035}$, $R^2=0.98$	About 2.4 percent lower loss per data doubling

This experiment measures action loss, not success rate, safety, recovery, or robustness to altered camera geometry. A lower average loss can coexist with severe failures on a high-precision task, as the 30 percent earbud result demonstrates. The scaling fit is therefore evidence for continuing data investment, not proof that every new hour has equal value or that success rates follow the same exponent.

9. What AIRoA-0 Actually Contributes

The strongest contribution is the alignment between data assumptions and model mechanisms. Anchored actions follow from the fact that the gripper is shared between human and robot. A visual evidence budget follows from the fact that wrist views contain site-specific clutter. Two-level language and stage prediction follow from the fact that a local frame cannot determine progress through a long task. Proprioceptive history follows from the fact that one measured state cannot reveal motion. Streaming decoding follows from the fact that the policy must revise predictions as the scene changes.

This is more convincing than treating each component as an independent architectural upgrade. The paper explains why the data source creates each failure mode and where that failure is addressed. The public code then verifies several of the claimed interfaces. The overall system is therefore more than a large dataset attached to a stock VLA model, even though it intentionally starts from π0.5 rather than designing a new backbone.

The experimental controls strengthen the empirical story. A single post-training recipe avoids per-task tuning of the proposed model, while each baseline receives a search. The evaluation uses a held-out site, one task instruction at deployment, wrist cameras only, and 15 trials per task. Cross-robot tests use unchanged task weights. These choices target real deployment mismatches rather than only in-distribution benchmark performance.

The results also leave several questions open. Ten tasks and 15 trials can reveal large differences but cannot estimate rare failures with tight confidence intervals. The dynamic-bin experiment is qualitative; it does not report perturbation timing, motion distance, response latency, recovery success, or variation across scenes. The four embodiments share the same end effector and camera geometry. There is no force or tactile input, and no evaluation of contact-rich failure recovery under slip, jamming, or controller saturation.

Stage prediction is another source of coupling that deserves more study. The stage labels are built from subtask annotations, so their quality and granularity affect both the classifier and the action interface. A mislabeled boundary or an overly coarse grouping can make the policy appear to fail at action prediction when the real issue is progress definition. The stage EMA reduces one-frame noise but cannot repair systematic ambiguity or an incorrect stage taxonomy.

10. Limitations and Deployment Boundaries

The authors explicitly state that AIRoA-0 uses no teleoperation, reinforcement learning, simulation, or on-robot corrections. All training data come from human operation of handheld grippers. This is the source of scalability, but it also means there are no demonstrations of motor faults, gripper slips, collisions, controller lag, or recovery after a robot misses a grasp. Visual feedback can support reactive correction, yet the model has not learned a distribution of robot-specific recovery actions.

The deployment envelope is explicitly tied to the same gripper and camera setup. A robot must be able to carry the YUBI end effector and its wrist cameras. The model has not been shown to transfer across arbitrary end effectors or camera transforms. This restriction reduces the apparent generality of “embodiment-agnostic” and should be part of any comparison with methods that learn a broader cross-embodiment action space.

Two wrist cameras also impose a spatial information limit. They are excellent for maintaining a consistent local view of the tool and the manipulated object, but they do not provide a reliable map of the full table, distant obstacles, or objects outside the current view. The short proprioceptive memory addresses temporal context, not global spatial context. Tasks that require navigation, large-scale obstacle avoidance, or reasoning over distant containers may need additional sensing or a high-level spatial representation.

Reproducibility has a similar boundary. The released repository supports post-training from the released weights with LeRobot data and documents the objective, stage loss, visual gate, and sampling code. The README does not provide an end-to-end reproduction of the full pre-training corpus processing or the 300,000-step distributed pre-training run. A reader can reproduce the adaptation workflow more directly than the construction of the foundation model itself.

Finally, a 78 percent average is a research result, not a production guarantee. Several tasks remain at 73 percent, and earbud handling reaches 30 percent. Production deployment would also need recovery success, human takeover cost, cycle time, force safety, and behavior under repeated failures. The paper demonstrates a credible path from handheld data to cross-embodiment manipulation; it does not claim that one average success number is sufficient for deployment.

11. Conclusion

AIRoA-0 converts a large handheld-gripper corpus into a complete policy pipeline. YUBI records bimanual tool motion with two wrist cameras and two levels of language. The model anchors actions in the gripper frame, learns task-relevant visual evidence, predicts subtasks during pre-training, and adds stage and proprioceptive memory during post-training. A non-uniform streaming decoder turns repeated model queries into smoother and cheaper motion generation.

The reported evidence is substantial: 16,151 hours of data, ten held-out-site tasks, a 78 percent average, one recipe for every task, and unchanged task weights on four robot embodiments that share the same gripper and camera interface. The boundaries are equally clear: no robot failure data, a constrained end-effector geometry, only local wrist views, no force sensing, and a public codebase centered on post-training rather than full pre-training reproduction.

The broader lesson is that handheld manipulation becomes a foundation-model data source only when the entire learning and deployment path respects its weaknesses. The tool-centered frame provides portability. The visual budget removes irrelevant background evidence. Language and stage prediction recover progress. Memory supplies short history. Streaming decoding keeps the latest observation actionable. The system works because these pieces correspond to the same data contract, not because one module alone solves cross-embodiment manipulation.