PAPER DEEP DIVE
Systematically Exploring the Capabilities of GPT-6 Astra as Embodied Policies
Evaluates GPT-6 Astra as an embodied policy across six domains: hybrid control reaches 48% on RoboDojo and 92% on RxR navigation, while direct in-hand control and dense locomotion references stay unreliable; token use and inference latency constrain practical control.
Paper: Systematically Exploring the Capabilities of GPT-6 Astra as Embodied Policies
Authors: Galbot Team (Xuchuan Chen, Xiaoqian Cheng, Yu Deng, Lihe Ding, Shaocong Dong, Xiangjun Gao, Haozhe Jia, Zekai Li, Zhoujian Li, Yunrui Lian, Sikai Liang, Chenghuai Lin, Dairu Liu, Jiahang Liu, Qingtao Liu, Yuxuan Ma, Zekun Qi, Jiayi Su, He Wang, et al.)
Date / category: September 2026 · arXiv:2609.38537v1 [cs.RO]
Links: arXiv abstract · project page · code repository (GPT-as-Policy) · interactive web report
Code status: public. The anonymous GPT-as-Policy repository ships together with the project page and the web report, and the web report carries every video case plus per-task trajectories.
Figure 1: Controlled comparison on matched physical start states in RoboCasa365. The four rows are four task/seed combinations; the columns are the initial scene, standalone pi0.5, Astra Direct, and the Astra + pi0.5 end state. In the coffee-mug case only the standalone policy succeeds; in the bottle-recycling case only Direct succeeds.
One-sentence summary
This report treats GPT-6 Astra as a policy that emits numerical actions directly, and measures across six domains — gripper manipulation, dexterous manipulation, mobile manipulation, navigation, locomotion, and humanoid loco-manipulation — how wide the gap is between "making useful task decisions" and "reliably producing physical effects".
Background and motivation
Community experiments around Astra have already demonstrated a range of possibilities, from 3D scene understanding and real-to-sim workflows to robot manipulation. Those demos force a question that has to be answered head on: when a reasoning model can generate robot actions directly, which tasks can it actually complete, where does it stay unreliable, and what does each of its decisions cost in resources? Reporting success cases alone cannot answer that, so the evaluation has to cover failures as systematically as successes.
Prior work falls into roughly three families. The first is spatial understanding and reasoning aimed at robot action. The second is vision-language-action models such as pi0.5, which map observations and instructions to actions learned from robot experience. The third wires language or programs into control: SayCan selects executable skills, Code as Policies composes perception and control into programs, and Prompt a Robot to Walk together with Natural Language as Policies explore numerical feedback control. Recent manipulation evaluations of Astra also report uneven performance across tasks. Each of these answers a local question; none provides a unified measurement across domains and across control architectures.
The entry point of this report is to make the control path a first-class object. The authors separate two architectures. Direct means Astra itself generates the robot commands, executed by an analytic controller (inverse kinematics plus proportional-derivative control). Hybrid means Astra cooperates with a learned task policy or a whole-body controller: it either reviews and edits the action proposals of that policy, or supplies commands and motion references to a frozen controller. How the same model behaves under these two architectures exposes exactly which capabilities come from model decisions and which come from the interface and the controller.
The choice of six domains is also deliberate. Gripper and dexterous manipulation probe grasping, local correction, and contact change. Mobile manipulation adds base-arm coordination and multi-stage goals. Navigation tests spatial decisions and stop judgement. Locomotion demands coordinated motion of the body itself. Humanoid loco-manipulation hands task execution to a pretrained whole-body controller. Every study keeps its native metric and protocol, so the scope of each conclusion stays visible instead of being flattened into a single aggregate score.
Preliminaries: actions, interfaces, and control paths
In the vocabulary of this report, an "action" is one command submitted to the robot control interface: a target pose, a joint delta, or a navigation move all qualify. Astra receives the observations a study permits, picks an action, and then picks the next action using the feedback after execution. The interface checks command bounds and the environment decides whether the task succeeded; tools support computation, image inspection, and logging. This closed-loop definition decouples the word "policy" from any particular network architecture and binds it to the decision-feedback loop instead.
Direct and Hybrid differ in who produces the action. In Direct, Astra's commands pass through no learned policy and no whole-body controller — inverse kinematics and PD control are enough. In Hybrid a learned component is present. In manipulation tasks, Hybrid means Astra accepts, edits, or replaces the action proposals of a task policy, and RoboCasa additionally permits rewriting the sub-goal instruction. In humanoid control, Hybrid means Astra feeds dense motion references, velocity commands, or sparse body targets to a frozen whole-body controller. The locomotion study contains both: direct joint PD targets as well as references handed to ScaleTrack for execution.
Figure 2: The execution loop and the development loop. The upper half shows the three execution paths inside a single episode (Direct end-effector control, Hybrid policy review, Hybrid whole-body control); the lower half shows the exploration loop in which researchers and development agents modify instructions, tools, and interfaces outside the evaluation. Model weights stay frozen throughout the evaluation.
One distinction is easy to miss: controller frequency and inference speed are different things. A controller can execute actions quickly while Astra takes far longer to decide the next command. When physics is paused during inference, the robot motion clock excludes decision latency. Resource accounting therefore has to record control steps, action segments, model requests, and cached-input / uncached-input / output tokens separately, rather than looking only at motion duration.
Evaluation design in detail
Gripper manipulation. Two closed-loop architectures are compared on RoboDojo and RoboLab. Direct has Astra generate dual-arm end-effector targets and gripper commands from images, proprioception, the instruction, and execution history, then re-observe after executing 1 to 5 control steps. Hybrid has a task-finetuned pi0.5 propose a 50-step joint-space sequence; Astra reviews it and either executes its first 1 to 15 steps or replaces them with a 1-to-5-step end-effector correction — the two branches are mutually exclusive. Both architectures share the task description, success criterion, inference configuration, and end-effector interface. RoboDojo selects 10 tasks stratified by the published pi0.5 success rate (6 from the lowest band, 2 from the next, and 1 each from the two higher bands), with 5 trials per task per architecture. RoboLab uses 10 zero-shot tasks on a single-arm Franka, with baselines running DROID-trained weights.
Dexterous manipulation. The first benchmark covers grasping, picking, placing, insertion, stacking, and rearrangement on a simulated Sharpa hand, with a multi-task pi0.5 finetuned on 100 demonstrations per task (1000 in total). Direct, standalone pi0.5, and Hybrid are tested on the same 5 unseen cases per task, with simulation horizons of 20 to 60 seconds. The policy action consists of two wrist poses (3 position plus a 6-D rotation representation each) and 22 joint angles per hand:
$$2\times(3+6)+2\times 22=62$$
Scoring uses a 0-to-100 partial-completion score rather than binary success. For method m, task t, and case i with internal score s, task and total scores are aggregated with equal weights:
$$S_{t,m}=\frac{100}{5}\sum_{i=1}^{5}s_{t,i,m},\qquad S_{m}=\frac{1}{10}\sum_{t=1}^{10}S_{t,m}.$$
A full score requires task-specific verification: a maintained grasp or a stable release. The second study is two-disk Tower-of-Hanoi stacking in DexJoCo, where dual Franka arms with Allegro hands complete two ordered transfers. Astra reviews a fixed pi0.5 trained for this task, accepting action prefixes or correcting wrist and finger targets, with at most 1500 control steps per episode at 50 Hz. The third study isolates "after a grasp has been established": starting from saved hand/object states, it compares direct Astra joint commands against four frozen task-specific RL policies, with matched physical starts but different observation and decision rates.
The in-hand study measures orientation tracking with the quaternion geodesic error, and reaching the goal requires the error to stay below 0.1 radians:
$$e_{t}=2\arccos\!\left(\left|\langle q_{t},q_{t}^{\star}\rangle\right|\right),\qquad e_{q}(t)<0.1\,\mathrm{rad}$$
The RL policies used for comparison are trained with a continuous rotation reward containing an inverse-error term, an at-goal indicator, an action-smoothness penalty, and a contact reward:
$$r_{t}=\Delta t\left[\frac{0.02}{e_{t}+0.1}+\mathbb{I}(e_{t}\leq 0.1)-10^{-6}\lVert\mathbf{a}_{t}-\mathbf{a}_{t-1}\rVert_{2}^{2}+0.02r_{t}^{\mathrm{contact}}\right]$$
Translation and "translation plus rotation" tasks have their own reward designs. The translation reward penalizes squared position error, pose drift, torque norm, and work done, and heavily penalizes insufficient height and tilt:
$$\tilde{r}_{t}^{\mathrm{trans}}=-100e_{p,t}^{2}+10I_{p,t}-0.3D_{\mathrm{pose},t}-0.1D_{\tau,t}-2D_{\mathrm{work},t}-1000D_{z,t}-170D_{\mathrm{tilt},t}$$
where e is position error and I is the at-goal indicator. Adding rotation replaces the indicator term in the translation reward and adds orientation-error and approach-velocity terms:
$$\tilde{r}_{t}^{\mathrm{trans+rot}}=\tilde{r}_{t}^{\mathrm{trans}}-10I_{p,t}+10I_{p,t}I_{\theta,t}-2e_{\theta,t}^{2}+15v_{\mathrm{toward},t}+10I_{\theta,t}$$
Execution on the Allegro side accumulates normalized joint-target deltas: after the model emits a 16-D vector a, the native controller applies the target update and clips to joint limits:
$$q_{\mathrm{target}}\leftarrow q_{\mathrm{target}}+0.04167\,a$$
Mobile manipulation. RoboCasa365 fixes 15 tasks (5 atomic seen, 5 composite seen, 5 composite unseen) with 5 seeds each, giving 75 episodes per condition. The three conditions share initial-state fingerprints, the PandaOmron robot, the controller, the task horizon, and the native success check. Astra receives only three RGB streams plus end-effector / base / gripper state — no object poses, no contact ground truth, no reference trajectories, no intermediate rewards. Hybrid additionally connects a frozen multi-task pi0.5 checkpoint, and Astra may accept a 20-step prefix, construct actions itself, or rewrite the policy instruction into a sub-goal.
Navigation. Four fixed subsets of 50 episodes each: val-unseen of R2R and English RxR, plus the ObjectNav validation sets of MP3D and HM3D. Astra sees a single 512x384 forward RGB image, the instruction or object category, a step counter, and action feedback; GPS, compass, depth, semantic labels, goal coordinates, and evaluation distances are all withheld. An adapter converts model output into discrete primitives of 0.25 m forward motion and 30-degree turns, with a budget of 500 primitives. Success requires VLN-CE to STOP within 3 m of the goal, or ObjectNav to land strictly within 1 m of an annotated viewpoint.
Locomotion and humanoid. The locomotion study keeps the scene-aligned 25x65 representation of PASSAGE and the frozen G1 ScaleTrack controller, replacing only its learned motion generator with Astra. Each proposal covers 0.5 s at 50 Hz of root height, projected gravity, planar velocity, yaw rate, and position and velocity for 29 joints — 1625 mutually consistent numbers. Astra makes five sequential attempts on the same open clutter curriculum; later attempts retain textual summaries of progress, falls, and collisions, and before the fifth attempt it also receives a retargeted walking clip. For the humanoid part, Astra commands a frozen Humanoid-GPT controller on the 30 executable HumanoidBench tasks, and on SIMPLE's 6 L2 tasks (10 scenes each) it uses a frozen ScaleBFM with written instructions revised from experience.
flowchart TD O[Observation
RGB images + proprioception + execution history] --> M{Control mode} M -->|Direct| D[Astra emits end-effector targets
bounded damped least-squares IK + PD for 1-5 steps] M -->|Hybrid manipulation| P[pi0.5 proposes a 50-step joint sequence] P --> R[Astra reviews the proposal] R -->|accept prefix of 1-15 steps| E1[execute policy prefix] R -->|replace with 1-5 step correction| E2[Astra end-effector correction] M -->|Hybrid humanoid| H[Astra gives velocities or sparse 5-point targets] H --> W[frozen whole-body controller at 50 Hz] D --> PHY[physical response
contact change and task judgement] E1 --> PHY E2 --> PHY W --> PHY PHY --> O
Figure 3: The three control paths evaluated here. In Direct, Astra produces commands itself; in Hybrid manipulation, Astra reviews or replaces pi0.5 proposals; in Hybrid humanoid, Astra supplies commands to a frozen whole-body controller. All paths return to observation, closing the loop.
Experimental results
Gripper manipulation: the benefit of cooperation depends on task structure. On RoboDojo, Hybrid succeeds in 24/50 (48%) versus 13/50 (26%) for Direct, with mean scores of 62.60 against 37.81; after reweighting published results to the same task and scene mixture, pi0.5 reaches only a 15.67% success rate and a score of 24.43. The gains concentrate on long-horizon and contact-sensitive tasks: sequential imitation 53 versus 0, tower stacking 64 versus 12, cloth folding 100 versus 40, bottle tossing 100 versus 36. On language classification and object classification Direct is instead higher (60 versus 38, and 100 versus 71). Across the 42,750 executed control steps of 50 Hybrid trajectories, 36,576 steps (85.6%) follow pi0.5 while 6,174 steps (14.4%) are generated or corrected by Astra — the model intervenes selectively rather than taking over.
RoboLab reverses the ordering completely: Direct 49/50 (98%), Hybrid 46/50 (92%), while pi0.5, Cosmos3-Nano-Policy, and DreamZero reach only 36%, 36%, and 34%. The authors' explanation is blunt: RoboLab is dominated by single-arm semantic pick-and-place, which Direct can already plan from the current observation; and when a zero-shot policy itself succeeds only 36% of the time, its proposals are not necessarily a useful prior, so Hybrid spends decisions correcting candidate actions that do not fit the task. The contrast between the two benchmarks shows that the value of policy assistance depends on how well the learned motion prior matches the task.
Figure 4: Gripper manipulation results. (a)(b) native RoboDojo score and success rate, where Astra + pi0.5 leads; (c) recorded RoboLab success rate, where Direct is highest at 98%; (d) of the Hybrid executed steps, 85.6% come from the policy and 14.4% from Astra.
| Benchmark and condition | Success rate | Mean score (0-100) | Notes |
|---|---|---|---|
| RoboDojo · Hybrid | 24/50 = 48% | 62.60 | 10 tasks x 5 trials, seed-paired with Direct |
| RoboDojo · Direct | 13/50 = 26% | 37.81 | Score uses 48 valid entries; success rate uses all 50 |
| RoboDojo · published pi0.5, reweighted | 15.67% | 24.43 | Reweighted to the same task and scene mixture |
| RoboLab · Direct | 49/50 = 98% | — | Zero-shot tasks, 9 tasks at full score |
| RoboLab · Hybrid | 46/50 = 92% | — | Three fewer successful trajectories than Direct |
| RoboLab · pi0.5 / Cosmos3 / DreamZero | 36% / 36% / 34% | — | Zero-shot transfer of DROID weights |
Dexterous manipulation: a small share of corrections buys a large gain, but grasp acquisition remains the bottleneck. On the ten-task benchmark Hybrid averages 61.6, pi0.5 44.2, and Direct only 16.6; Hybrid beats Direct on all ten tasks and pi0.5 on eight. Mahjong tidying, upright-egg placing, and bottle-can sorting gain 40, 32, and 28 points. Astra's interventions account for just 11.98% of Hybrid executed steps. In DexJoCo Tower-of-Hanoi stacking, 5 of 10 experience-guided episodes are completed and 73.6% of executed steps use unmodified policy actions; those ten episodes plus five reviews record 63.6 million tokens, of which only 136,000 go to review — even when the policy supplies most physical actions, token consumption still happens mostly during online control.
Figure 5: Two-disk stacking success rates. Astra + pi0.5 reaches 50% in experience-guided episodes, above DP-T 24.7%, pi0.5 23.3%, DP-C 12.7%, ACT 6%, and GROOT N1.5 0.7%.
The in-hand comparison delivers the sharpest negative result. On cylinder rotation, RL holds the target orientation during 76.90% of steps while Astra manages 0.51%; on the cuboid it is 63.50% versus 4.40%. The cuboid case is especially telling: both controllers keep the object in hand across all five horizons, but Astra rotates too slowly — the gap lies between "holding" and "advancing", not in whether the object is dropped. On translation Astra succeeds in 1/5 and RL in 4/5 (terminal error 59.05 mm versus 17.29 mm); on translation plus rotation Astra gets 0/5 while RL gets 4/5 at matched endpoints and 5/5 after 15 s. Astra's budget of 120 decisions is exhausted in 6.00 to 7.65 s, and RL is already better within the same duration, so a longer action horizon does not explain the gap.
Figure 6: Continuous rotation on five paired initial states. Top row cylinder over 20 s, bottom row cuboid over 10 s; columns are orientation error, axial velocity, cumulative at-goal rate, and cumulative reward. Astra's cylinder velocity is near zero, and on the cuboid it slowly falls behind the target velocity.
| Task | Controller | At-goal step share | Error (rad) | Velocity MAE |
|---|---|---|---|---|
| Cylinder 20 s | Astra Direct | 0.51% | 1.626 | 0.960 |
| Cylinder 20 s | Task-specific RL | 76.90% | 0.173 | 0.276 |
| Cuboid 10 s | Astra Direct | 4.40% | 0.742 | 0.143 |
| Cuboid 10 s | Task-specific RL | 63.50% | 0.096 | 0.063 |
Mobile manipulation: assistance reshapes the capability profile rather than lifting it uniformly. On RoboCasa365, Hybrid completes 29/75 (38.7%), Direct 25/75 (33.3%), and standalone pi0.5 17/75 (22.7%). By group, on atomic seen tasks Hybrid matches the standalone policy at 52% while Direct reaches only 28%; on composite seen tasks Hybrid leads at 28%; on composite unseen tasks Direct overtakes Hybrid, 56% versus 36%. On paired start states Hybrid alone succeeds in 13, Direct alone in 9, and both in 16, with an exact McNemar test giving p = 0.5235 — the two are not in a simple containment relation. Instruction rewriting is used in 66/75 episodes, and rewritten instructions condition 72.6% of proposals; of executed steps, 55.2% come from accepted policy actions and 44.8% from Astra. The reverse cases deserve more caution: on CoffeeSetupMug and WashLettuce both Astra conditions fail on all five seeds, while the standalone policy completes 3 and 2 respectively — connecting a policy does not guarantee preserving the successes that policy already had.
Navigation: success rate leads, search efficiency exposes the weakness. Across the four subsets Astra achieves R2R 39/50 (SR 78, SPL 65.27), RxR 46/50 (SR 92, SPL 77.25), MP3D 28/50 (SR 56, SPL 22.43), and HM3D 41/50 (SR 82, SPL 43.69). Under identical episode lists, scoring rules, and primitive budgets, the locally evaluated LightNav-0, Uni-NaVid 7B, OmniNav Flow, NavFoM, and SPAN-Nav all score below Astra on SR and SPL for every subset they report, with SR gains over LightNav-0 of 20, 20, 36, and 16 points. But ObjectNav SPL is only 22.43% and 43.69%: even when the endpoint succeeds, detours are penalized by SPL, which is precisely the inefficient search that success rate hides; 10 MP3D episodes and 5 HM3D episodes exhaust the 500-step budget.
Figure 7: Navigation results. (a) SR and SPL across the four subsets; (b) termination composition — successful STOP, failed STOP, and budget exhaustion. All VLN-CE episodes end with STOP, while ObjectNav contains budget-exhausted episodes.
| Task / dataset / split | Successes | SR | SPL | nDTW | sDTW |
|---|---|---|---|---|---|
| VLN-CE / R2R / Val-Unseen | 39/50 | 78 | 65.27 | 72.20 | 59.35 |
| VLN-CE / RxR English / Val-Unseen | 46/50 | 92 | 77.25 | 84.73 | 80.42 |
| ObjectNav / MP3D / Val | 28/50 | 56 | 22.43 | — | — |
| ObjectNav / HM3D / Val | 41/50 | 82 | 43.69 | — | — |
Locomotion: five sequential attempts still fail to cross the dense-reference interface. On the same matched curriculum, PASSAGE reaches the goal in 13.18 s (8.622 m of progress, 0.495 m terminal error). Astra's first attempt falls after 1.561 m of forward motion; attempts two through four stay upright but make almost no progress; the fifth stays upright for 30 s, walks 3.833 m, shortens the distance to goal by 2.598 m, and still stops 6.519 m short of the main obstacle region. The authors are explicit that these five are consecutive stages of one adaptation trajectory, not independent trials. The time cost is more intuitive still: the final 30-second episode used 250 synchronous calls with a mean recorded latency of 39.86 s each, while PASSAGE plans in about 0.08 s — with physics paused during inference. The analytic interface without Astra is no more optimistic: Five-point and WholeBody-14 both score zero strict successes across 12 rollouts (6 curricula x 2 physics backends), reach "arrive and hold" in 4/12 and 3/12, and on the low-ceiling curriculum only 1/6 and 0/6 reach the endpoint.
| Planner / attempt | Outcome | Time (s) | Progress (m) | Terminal error (m) | Fall |
|---|---|---|---|---|---|
| PASSAGE | Goal reached | 13.18 | 8.622 | 0.495 | No |
| Astra attempt 1 | Fall | 9.80 | 1.561 | 7.556 | Yes |
| Astra attempt 2 | No progress | 20.28 | -0.024 | 9.141 | No |
| Astra attempt 3 | No progress | 8.00 | 0.012 | 9.104 | No |
| Astra attempt 4 | No progress | 8.48 | 0.159 | 8.958 | No |
| Astra attempt 5 | Time limit | 30.00 | 2.598 | 6.519 | No |
Humanoid loco-manipulation: strong command selection, weak inter-phase contact. On the 30 HumanoidBench tasks Astra's mean native return is 581.9, above TD-MPC2 at 338.2, SAC at 42.7, and DreamerV3 at -41.9; it exceeds the three on 16, 19, and 25 tasks respectively, beats the best of the three on 13 tasks, ties on Kitchen, and reaches the native threshold on six tasks (Stand, Walk, Maze, Crawl, Push, Sit Simple). Maze at 1358.8 clears the 1200 threshold, while Reach at 11430.2 stays just under 12000. The effect of calibration is also quantified: raising the Walk gait-clock frequency from 1.2 to 1.8 Hz lifts the mean return from 705.72 to 746.60 across eight paired seeds, with weights, commanded speed, and feedback rules unchanged; a commanded 1.25 m/s actually walks at about 1.48 m/s. Under SIMPLE's L2 protocol Astra completes 50/60 (83.3%), but the native Handover success criterion does not require the carton to remain stable after full release, and in failure cases closed fingers or a tilted carton are misjudged as a firm grasp.
Figure 8: Mean return and standard deviation over the 30 HumanoidBench tasks. Green is Astra; the rest are published DreamerV3, TD-MPC2, and SAC results. Dashed lines are the native per-task thresholds.
Figure 9: DexJoCo wrist-camera sequences. (a) In a successful episode Astra corrects first, then hands control back to the policy to complete the release, and the carton stays supported after release; (b) in a failed episode, after repeated seating corrections the small disk remains tilted above the target peg until the step limit.
The resource ledger: delegating motion is not delegating review. Over 50 task instances per condition on RoboDojo, Hybrid and Direct record total tokens (including cached input) of 624,762,828 and 1,132,343,772, of which cached input is 607,555,840 and 1,107,349,760, uncached input 15,901,963 and 22,907,448, and output 1,305,025 and 2,086,564; executed control steps are 42,750 versus 38,221 and action segments 3,776 versus 7,729. Hybrid cuts total tokens by 44.8%, yet uncached input and output remain substantial. On RoboCasa, Direct and Hybrid record 7,910 and 8,941 model requests, with per-episode medians of 85 and 108 — Hybrid reduces the action segments Astra writes itself, but records more requests. The first-principles conclusion is that inference efficiency depends on review frequency, not only on who generates the actions.
| RoboDojo metric | Hybrid | Direct |
|---|---|---|
| Executed control steps | 42,750 | 38,221 |
| Executed action segments | 3,776 | 7,729 |
| Total tokens (incl. cached input) | 624,762,828 | 1,132,343,772 |
| Cached input tokens | 607,555,840 | 1,107,349,760 |
| Uncached input tokens | 15,901,963 | 22,907,448 |
| Output tokens | 1,305,025 | 2,086,564 |
Limitations
Stated by the authors, first: dense motion references are unreliable and expensive. None of the five sequential attempts reaches the goal, and the authors attribute this to the interface itself: joint limits do not guarantee a self-consistent reference, since root velocity, limb pose, and their evolution must describe compatible motion; a tracker can follow a reference and still produce marching in place, spurious contacts, or a fall. At the same time, a mean call latency of 39.86 s means "a decision takes far longer than the motion it executes", and pausing physics is only an experimental convenience — a real scene will not wait.
Stated by the authors, second: direct in-hand control trails task-specific RL, and assistance can lose successes the policy already had. In the in-hand experiments Astra's at-goal step share is two orders of magnitude below RL; on RoboCasa both Astra conditions fail entirely on CoffeeSetupMug and WashLettuce while the standalone policy completes them. The authors conclude from this that maintaining the current grasp blocks the release and repositioning that continued motion requires, and that "detecting an error, proposing a correction, and verifying its effect" are three distinct capabilities.
Stated by the authors, third: improvement is not reproducible success. The Push development log records a failure immediately after a success; the fixed Handover procedure passes the native success plus an extra hold check on only two of three previously successful scenes; and instructions revised from experience do not necessarily preserve success in later attempts. A diagnosis can correctly identify the problem without guaranteeing that the next action is good enough.
My assessment: sample size and the simulation boundary. Most studies use only 5 instances or 5 seeds per task, so percentages such as 48% and 98% carry wide confidence intervals — the report itself only gives descriptive Wilson intervals and makes no significance claims (the single exception being McNemar at p = 0.5235). All results come from simulated environments such as MuJoCo, Habitat, and RoboCasa, with no real-robot validation; the navigation comparison also modifies the camera setup of some baselines, so it is not fully comparable with their published configurations. In addition, the locomotion study is a single adaptation trajectory and cannot be extrapolated into "average capability over five independent trials".
Conclusion and outlook
The value of this report is not a total score for Astra but the separation of "decision" from "control" as measured quantities. In navigation, Astra leads the locally compared systems using RGB-only observation. In manipulation, the Hybrid architecture is clearly better than Direct on long-horizon and contact-sensitive tasks while using only about a tenth of the executed steps. In the humanoid setting, a frozen whole-body controller plus command selection beats several reinforcement-learning baselines. At the same time, in-hand contact coordination, dense motion references, and completion verification fail persistently, and each fails differently: one is the tension between holding and advancing, one is reference self-consistency, and one is mistaking "looks right" for "is right".
The directions the report gives for follow-up work are equally concrete: a cooperating policy has to learn both when to intervene and when to hand control back; completion checks need a verification mechanism independent of action generation; the resource ledger makes review frequency a first-class optimization target; and any move toward real robots must assess inference latency together with the rate at which physical feedback changes. The egocentric-video-to-interactive-scene pipeline (Chamfer-L1 dropping from 35.93 mm to 24.73 mm, then to 12.14 mm after adding CAD dimensions) points to another route — letting the model build and correct its own perception and retargeting toolchain — but only one of three apple-transfer cases completes the full interaction, showing that geometric plausibility is not physical interaction success.
Golden quotes
These findings reveal a gap between useful task decisions and reliable physical control.
A stable grasp is only an intermediate condition: continued manipulation requires some fingers to release and reposition while others preserve support.
Delegating motion therefore does not necessarily delegate the work of reviewing it.
Pausing physics makes that delay tolerable in the experiment; a physical scene cannot be assumed to wait.
A plausible rationale for moving does not ensure correct goal judgment or spatial progress.
