Skip to content
←Back to Papers

PAPER DEEP DIVE

Agentic RoboticsAIEmbodied AI

Envisioning Agentic Robotics: A New Era of Robotic Systems

The release of GPT-6 Astra marks the emergence of a new paradigm in robotics: agentic robotics. Within weeks, frontier models with agentic harnesses matched or exceeded task-specific policies on many embodied benchmarks. Instead of training a dedicated policy for each family of tasks, an agentic robotic system applies a foundation model to open-world embodied tasks. It operates in a closed loop: observing the environment, interpreting user intent, reasoning about the task, invoking skills and tools, verifying outcomes, and adapting its strategy. Despite these advances, the community has yet to converge on a shared definition, a reliable evaluation framework, or a research roadmap. We argue that Embodied System 2, realized by a foundation model together with an embodied harness, forms the decision-making core of agentic robotics. Its central challenge is to translate general-purpose reasoning into effective and verifiable actions under physical constraints such as limited reaction time, mechanical limits, unpredictable environment changes, and irreversible consequences. Motivated by this perspective, we propose an evaluation framework that measures both model capabilities and closed-loop system performance. Furthermore, we identify the principles that carry over from digital to physical agents and the challenges that must be addressed to achieve reliable physical autonomy. On top of the Embodied System 2 and the evaluation framework, we envision a research roadmap toward sustained autonomy: the ability to perform work safely and reliably with progressively broader responsibility, from executing individual primitives to fulfilling standing roles.

Authors: Grid Robotics, Qingfeng Lan, Duo Li, Siyao Li, Yiheng Lyu, Shijian Lu, Zhenghao Qi, Shizun Wang, Xinchao Wang, Ming Yang, Zhiqin Yang, Zuhao Yang, Tao Yu
Paper: arXiv:2611.18359
Project page: Agentic Robotics Roadmap
Artifact status: Commit 03da248 of the official project page contains content.js and evaluation-data.js, which present the roadmap and evaluation material. It does not contain a runnable robot model, training pipeline, or complete reference system. The paper should therefore be read as a framework and research agenda, not as an implementation report.

One-sentence summary

The paper defines Embodied S2 as a foundation model plus an embodied harness, separates model-level from system-level evaluation, and measures progress by how much verified physical responsibility a robot can safely carry.

1. The problem is broader than policy generalization

Classical robot autonomy decomposed perception, planning, and control into engineered modules. Learned policies then absorbed many of those functions, first for locomotion and manipulation, and later through vision-language-action models that map observations and instructions to actions. The best of these policies can be highly capable inside a task distribution. Their weakness appears when the task wording, object layout, body, or required sequence moves outside the data and assumptions used during training.

One response is to collect more robot trajectories and train a larger policy. The paper accepts that path but argues that it cannot be the only response. Physical interaction is slow to collect, sparse in contact information, expensive to reset, and unevenly distributed across bodies. A second path is to let a general foundation model organize behavior at run time: interpret an underspecified request, choose existing skills, invoke a tool, inspect evidence, and repair a failed plan. This is the transition the authors call agentic robotics.

The proposal is not simply a language model connected to a robot. A chat model can produce a plausible plan and still fail because it sees stale state, sends a pose in the wrong frame, cannot cancel a running skill, or mistakes a successful function return for a completed physical outcome. The paper therefore shifts attention from the policy alone to the complete decision and execution loop. General knowledge proposes a course of action; a runtime turns that proposal into bounded calls, observations, evidence, and revisions.

The timing of the article matters. After frontier models were paired with agentic harnesses, reported success rates changed dramatically within a short period. On the 42 tasks of RoboDojo, Astra results are discussed across a range from 22% to 85%, depending on tool libraries, task knowledge, action budgets, and protocol. Such dispersion is not a minor reproducibility footnote. It shows that the field is often comparing complete systems while describing them as model evaluations.

The authors make three contributions in response. They define functional roles for System 2, System 1, and System 0; they propose an evaluation framework that treats foundation-model ability and closed-loop behavior separately; and they organize a research roadmap around progressively broader responsibility. The contribution is conceptual and methodological. The paper does not introduce a new network or demonstrate one unified system that resolves its agenda.

2. A functional architecture, not a fixed product diagram

S2 interprets intent, reasons under goals and constraints, selects the next action or information request, and checks whether the previous step achieved its purpose. S1 contains callable capabilities: action policies, perception modules, motion planners, and programs used as temporary skills. S0 is the body-specific real-time layer that maintains balance, tracks trajectories, solves inverse kinematics, and executes control targets within hardware limits.

Reference architecture for agentic robotics

Figure 3: Embodied S2 combines a foundation model with a harness for context, tools, planning, memory, and verification. A real-time event monitor can interrupt S2 or trigger a local protective response in S0.

The decomposition is functional rather than physical. A single model may cover more than one role, and one role may span several models or modules. S2 is not required to be a large language model. A classical planner or behavior tree that decides how to proceed and verifies outcomes can satisfy the functional definition. Conversely, labeling a module "System 2" does not make it S2 if the module never decides what follows from evidence.

The architecture also includes a real-time event monitor outside S2, S1, and S0. It watches for developments that cannot wait for deliberation: a user says "stop," a collision produces unexpected motion, or a person enters the workspace. If the event changes the task, the monitor interrupts S2. If immediate protection is required, predefined rules can command S0 to slow or stop without waiting for a language model. This external monitor is an explicit admission that deliberation is too slow and fallible for every safety function.

The paper describes three execution paths. In layered orchestration, S2 calls an S1 skill, and S1 produces targets for S0. In direct control, S2 sends a permitted target to S0 through an action interface, while S0 continues its own real-time loop. In code as policy, S2 writes a program that the harness executes in the S1 role. These paths may coexist in one task: a perception tool can locate an object, a VLA can grasp it, and a generated program can repeat a stable sub-process.

Layered, direct, and code-as-policy execution paths

Figure 4: S2 can change abstraction level without entering the millisecond control loop. Layered orchestration, direct action targets, and generated code are paths through the same system.

What distinguishes S2 from a planner or a success classifier is the closure of two decisions. First, it decides how to proceed: choose a subgoal, a skill, a tool, arguments, or a choice to continue, retry, replan, ask, or stop. Second, it decides what the outcome means: use evidence to judge whether the last step or whole task reached its goal. Both decisions must be visible through the interface or demonstrated by a controlled replacement experiment.

This criterion excludes several components that are sometimes casually called the embodied brain. A planner that sequences skills but never checks effects implements only part of S2. A success detector scores an observation but does not decide what to do with the score. State estimators, perception modules, world models, and event monitors supply evidence, but they do not decide its meaning for the task. A dual-system VLA whose upper module emits only an inaccessible latent is not classified as S2 unless replacement tests establish that the latent carries the required decisions.

3. The agentic loop ends in evidence, not a command return

An episode begins when the harness assembles context: the request, relevant world state, task progress, body state, available skills, and interaction history. The foundation model proposes a decision, and S2 dispatches a capability under a contract. After execution, the system observes and verifies the result. The next decision depends on that evidence, not merely on whether a function call raised an exception.

Agentic loop from intent through execution to verified outcome

Figure 5: The loop interprets intent, reasons, invokes a capability, observes, verifies, and adapts. An event monitor can interrupt any stage and initiate cancellation or protective action.

The verification section separates four questions that are often collapsed into one success flag. Was the request accepted? Did execution terminate? Does the intended physical effect hold? Did the result satisfy the user's intent? A gripper command can terminate normally while the wrist camera shows that the cup remains on the shelf. The grasp then passed the termination check but failed the physical-effect check. Repeating the same command is irrational; the system should change the viewpoint or grasp point.

Cancellation has the same staged structure. A user can revise the request while a grasp is running: not the red cup, the blue one. The monitor interrupts S2, and S2 requests a pause or cancel. The cancellation request, acceptance of that request, and actual cessation of motion are different states. A conflicting action must not be dispatched until the harness confirms that the old motion has stopped and the body is in a safe holding state.

Timeouts are also ambiguous. An operation may complete after the caller stops waiting. Blind retry can then create a second grasp, move an object twice, or collide with the first action's continuing effect. Every physical call should disclose units, reference frames, preconditions, expected effects, resource limits, timeout semantics, cancellation behavior, and terminal states. A call record and evidence bundle should make failure attributable after the fact.

The paper's lifecycle is proposal, admission, dispatch, execution, confirmation of termination, effect verification, and progress commit, with explicit branches for refusal, failure, cancellation, and unknown outcome. This is an execution contract rather than a prompt convention. It gives the harness a principled place to reject invalid requests, track asynchronous work, and prevent the model from declaring completion without evidence.

State must persist beyond one observation, one episode, and one session. An episode is a single task; a session is a continuous period containing multiple episodes. Within an episode, the harness records object identity, progress, and prior attempts. Across episodes and sessions, it maintains environment, user, and body information. Physical records can become stale without warning, so each memory item needs provenance, observation time, and a revision or invalidation rule.

4. A compact formal account of the system boundary

The paper numbers Equation (1), which defines observation age. To make the surrounding definitions easier to inspect, this article states five additional expressions derived directly from the paper's prose. The five restatements are not numbered equations in the original.

The first expression separates the foundation model from the harness. Let F be the model and H the harness, with context, tools, planning, memory, and verification represented by C, T, P, M, and V:

$$ \mathrm{EmbodiedS2}=\langle F,H\rangle,\qquad H=\langle C,T,P,M,V\rangle $$

This says that S2 is not a model alone. The model decides from context, but the harness determines what enters the context, which calls are legal, how execution is monitored, what evidence returns, and what information survives the current episode. A safety gate, budget controller, and self-improvement loop are augmentations built around the five core parts, not implicit properties of the model.

The second expression divides current context into world, user, and body state:

$$ c_t=\langle W_t,U_t,B_t\rangle $$

World state includes objects, places, relations, and observation time. User state includes preferences, habits, and commitments. Body state includes geometry, kinematics, calibration, current condition, available S1 skills, and S0 interfaces. These states change on different timescales and have different privacy requirements. Explicit body state is also what allows one high-level system to reason about a robot it has not previously operated.

The third expression is the sole numbered equation in the paper. It defines observation age as the interval between capture and physical effect:

$$ \Delta t_{\mathrm{obs}}=t_{\mathrm{effect}}-t_{\mathrm{capture}} $$

The value includes sensing, communication, queueing, inference, tool execution, and control response whenever those stages lie on the critical path. A synchronized timebase is necessary when sensors, compute, and actuators are distributed. Model inference time alone is not a sufficient latency measure for a physical outcome.

The fourth expression converts between period and rate for the timing bands compared in Figure 8:

$$ \Delta t=\frac{1}{f},\qquad f=\frac{1}{\Delta t} $$

This conversion exposes the scale gap. S0 runs around 50 Hz to 1 kHz, or once every 20 ms to 1 ms. S1 control skills run at roughly 5 Hz to 200 Hz depending on platform. S2 is event-driven, but the paper places dynamic manipulation decisions on an approximately 0.1 s to 1 s scale. A model that takes tens of seconds per call cannot substitute for lower layers even when its semantic answer is better.

The fifth expression represents the four separate verification outcomes. Let acceptance, termination, physical effect, and intent satisfaction be a, r, e, and i:

$$ v_t=\langle a_t,r_t,e_t,i_t\rangle,\qquad a_t,r_t,e_t,i_t\in\{0,1\},\qquad V_t=a_t\land r_t\land e_t\land i_t $$

The value of the expression is diagnostic, not merely logical. It forces the system to collect separable evidence. A tool return can establish acceptance or termination. Controller state can establish that a motion ended. Independent sensing can establish contact or placement. Intent satisfaction may require a user-defined acceptance criterion or a task checker. Without separate evidence, an agent can mistake a successful call for a successful outcome.

The sixth expression captures capped scaling. S2 tries to maximize task return while respecting decision-latency and computation limits:

$$ \max_{\pi\in\Pi}J(\pi)\quad\mathrm{s.t.}\quad L(\pi)\leq L_{\max},\; C(\pi)\leq C_{\max} $$

This is a restatement of the paper's prose objective, not another numbered equation. It explains why a correct answer can be operationally useless. The object may have fallen, the person may have entered the route, or the action window may have closed. Capability must be evaluated inside the deadline that the physical situation imposes.

5. The harness carries much of the observable capability

Embodied S2 has eight abilities: knowledge, understanding, reasoning, tool use, memory, cross-embodiment, safety, and efficiency. Knowledge, understanding, and reasoning are mainly supplied by the foundation model. Tool use, memory, cross-embodiment, safety, and efficiency emerge from interaction between the model and harness. This split explains why changing the runtime around a fixed model can move system success substantially.

Eight core abilities of Embodied S2

Figure 6: The eight abilities combine model knowledge with runtime mechanisms. The harness is part of the capability, not a thin adapter around it.

The context window is not a substitute for memory. The paper cites an estimate that 128K tokens hold only about three minutes of dense multimodal experience. A frozen Astra that kept an entire episode in context reached 92.1% over 800 RoboMME test episodes in one run, but model calls averaged 18.3 seconds and the simulator paused during calls. That result supports in-context episodic recall while showing its limits for responsive work. Cross-session memory still needs storage, retrieval, update, and invalidation outside the prompt.

Tools are physical contracts, not ordinary API names. A grasp may succeed while occluding the next insertion; a perception result may use a different frame; an inverse-kinematics solution may be collision-free only under assumptions the caller did not check. The tool interface therefore needs explicit preconditions, argument semantics, failure modes, resource costs, and evidence. Retries and handoffs are safe only when the caller knows what previous call actually did.

The paper separates three feedback loops. The lowest loop couples perception to motion inside S1 and S0. The planning loop lets S2 choose a new subgoal after verification. The evolution loop changes skills, prompts, context rules, tools, harness code, or weights between runs. Treating these loops as one "learning" mechanism hides different safety and validation requirements. An edit to harness code is a software change even if no model weight moves.

Safety cannot reside inside the model's judgment. A safety gate can reject a call before dispatch; the tool layer can enforce permissions; S1 and S0 can impose force, velocity, workspace, and stability limits; the event monitor can invoke a protective stop. The model still needs to recognize risk and choose cautious behavior, but the system must remain bounded when the model is wrong, late, or unavailable.

The harness also controls efficiency. It should call the model only when a decision is needed, compile stable routines so they can run without it, and delegate fast feedback to S1 and S0. The paper calls this "capped scaling": improve decision quality within a fixed time and compute budget rather than assuming that more reasoning tokens are always better. The objective is not maximum thought per decision; it is useful thought before the physical window closes.

6. Evaluation must name both the object and the clock

Model-level evaluation asks what a foundation model can do without tools, usually in a single turn. It can isolate knowledge, situation understanding, and model-side reasoning or safety. System-level evaluation places the model inside a harness and lets it call S1, S0, and external tools. It measures closed-loop reasoning, tool use, memory, cross-body transfer, safe execution, efficiency, and cost.

The two levels answer different failure questions. A model-level error points toward the model. A failure that appears only at system level may come from the model, harness, interface, S1 skill, S0 controller, or their coordination. Replaying the observations and call results available at failure can narrow the cause. Comparing systems without knowing whether the harness was fixed, developer-supplied, or under study makes attribution impossible.

Evaluation questionModel levelSystem levelWhy the distinction matters
Spatial understandingRelations, frames, and affordances in a promptLocalization and action after movementA correct answer does not prove a reachable action
PlanningChoose feasible subgoals for a supplied capability setDiagnose and recover after execution failsFeedback changes the plan
Tool useChoose an interface and argumentsHandle real latency, state, and side effectsText tool descriptions omit physical noise
SafetyRecognize risk or produce a refusalStop harm before, during, and after actionText refusal is not robot refusal
EfficiencyTokens and model latencyObservation age, response delay, and task costInference time is only one segment

The framework contains nine dimensions and 53 sub-dimensions. Eight dimensions correspond to the abilities, and cost records resources consumed rather than ability. Of the 53 sub-dimensions, 29 are specific to embodiment or depend strongly on it. Results should be reported as a profile, not collapsed into one number. Safety constrains every ability; intent satisfaction is the end-to-end result those dimensions serve.

The protocol requirements are unusually concrete. Fix and disclose the action abstraction, observation contract, tool inventory, reasoning budget, call and step budgets, retries, and verification method. Use a checker that has ground truth unavailable to the agent. Report confidence intervals, missed deadlines, observation age, false completion claims, human takeover, and per-success cost. Evaluate in a world that continues to run whenever the claim concerns responsiveness.

Benchmark names are not enough. A written task recipe, privileged object state, in-context demonstration, hidden success checker, or different step limit can change the result more than a model upgrade. The paper calls for a harness card that records these choices, and for controlled comparisons that fix the model when studying the harness and fix the harness when studying the model.

7. The reported evidence is a profile, not a leaderboard

The paper's cross-benchmark table is explicitly observational. Most entries are preprints published within two months of Astra's release, and the authors did not reproduce them. Settings differ, several simulators pause during inference, and some systems receive task-specific information that baselines do not. The table is useful because the pattern is stronger than any single headline score, not because its rows form a controlled ranking.

Benchmark or experimentReported resultAppropriate interpretation
RoboMMEAstra 92.1%; 800 episodes; one run; 18.3 s per call; simulator pausedLong-context episodic memory works, but real-time response is not established
RoboDojo-SimDirect interface 22.48%; official tool-based evaluation 46.9%The harness and protocol materially change the result
RoboLab-120Direct control 66.3%; agentic session 74.8%Higher performance, but not a controlled cross-model comparison
LIBERO-PRO94.0% by composing motion primitives, segmentation, pi0.5, and codeA strong result on a narrower perturbed suite
LADDERAstra 62.0%; learned-policy systems at most 29.6%Orchestration helps as tasks lengthen, under this setup
RobotWorld19.0%; 63 tasks unsolved by all five modelsBroad embodied work remains far from solved
RoboQuest23.2%Active exploration and evidence gathering are bottlenecks
HumanCLAWTarget recognition 64.9%; full completion 16.8%Recognition does not imply whole-body task completion
AbstainEQABest frontier model 42.79%; humans 91.17%Models struggle to recognize insufficient evidence
EmbodiedSkillBenchNative control 92%; skill-only 61%; on-demand skills 85%Tool abstraction can cap a stronger model when mismatched
RoboHarmText refusal 72%; real-robot refusal 18%Safety intent does not transfer reliably to action

Three trends emerge. First, on narrow simulated suites that reward semantic understanding, memory, or composition, a general model with a harness can match or exceed trained policies. Second, broad or exploratory benchmarks remain difficult: RobotWorld is at 19.0%, RoboQuest at 23.2%, and HumanCLAW completion at 16.8%. Third, interface and execution conditions are first-order variables. The same benchmark can produce very different agentic scores without any model change.

The latency evidence explains why architecture matters. Direct action inference took about 18.56 seconds per model call. Generating locomotion references averaged 39.86 seconds per call, and 30 seconds of walking required 250 calls with a paused simulator. On the hardest RT-Safe routes, Astra took 7.7 seconds per decision. Safe success was 19.8% with a paused world and 0.7% when the world continued, while collisions rose 12.3 times.

Cost also scales with the harness. The paper reports an agentic Astra benchmark session using nearly 19 times the API fees of direct prompting. On RoboQuest, an average episode represented 10.4 minutes of simulated time but 53.8 minutes of wall-clock time and cost 11.29 dollars. These are system measurements. They cannot be reconstructed from token length or benchmark success alone.

The harness can improve execution without changing the planner. With the same Astra planner, primitives, and call budget, organizing calls as Python cells raised success from 63.1% under ordinary tool calling to 71.7%. In EmbodiedSkillBench, selective skill use did not improve Astra's success but saved 14.1% of tokens and 11.9% of time. More tools are not automatically better; an interface can lower a strong model's performance when it hides actions the model would otherwise choose.

The evidence also exposes perception and verification gaps. HumanCLAW shows that target recognition can be far above whole-task completion. AbstainEQA shows that models often answer when evidence is insufficient. RT-Safe shows that longer deliberation becomes a collision risk when the environment does not pause. The common failure is not only a lack of semantic knowledge. It is the difficulty of maintaining timely, body-aware, verifiable control.

Finally, the heterogeneity itself rules out cross-paper rankings. A single benchmark may include written recipes, task-specific tools, in-context demonstrations, different reasoning effort, different step budgets, or a hidden verifier. The paper argues for success-latency-cost curves under disclosed contracts. Without them, a high score is evidence for a particular system configuration, not a transferable statement about the foundation model.

8. Digital-agent lessons survive only after physical assumptions are rebuilt

Software agents benefit from four unusually favorable conditions. State and action interfaces are close to model-native formats. Outcomes can often be checked automatically. Training data is abundant. Environments can be copied, reset, and inspected. Coding tasks add compilers and tests that turn each attempt into fast, structured feedback.

Robotics weakens all four. Observations are local, noisy, occluded, and delayed. Physical effects require sensing the scene rather than reading a return code. Dexterous manipulation has no internet-scale action corpus. Real environments are expensive to reset, and a failed action can permanently change the state.

Five principles still transfer: close the loop around verified outcomes, expose explicit tool contracts, assemble context deliberately, separate execution from judgment, and reuse skills. What must be rebuilt is the physical interface, calibrated verifier, cross-task memory, cross-body adaptation, and runtime governance. A compiler exit code becomes an evidence bundle with observation time, terminal state, physical effect, and provenance. A sandbox snapshot becomes a safety gate, capability permissions, protective stop, and auditable call record.

The physical world adds irreversibility and body variation. A digital agent can wait a minute and resume from the same state; a robot may face a scene that has moved. A code failure can be rolled back; a collision cannot. Motor heating, current saturation, voltage sag, calibration drift, and wear can change the effect of the same command. The paper cites a hardware-disturbance experiment in which the aggregate success of eight VLAs fell from 93.2% to 50.9% under the most severe setting. Keeping model weights unchanged does not keep system capability unchanged.

9. The roadmap measures delegated responsibility

L1 through L4 describe progressively broader responsibility rather than a taxonomy of tasks. At L1, the robot executes a local primitive and checks its result, such as picking up a cup. At L2, it coordinates skills to complete a task, such as making coffee and bringing it to a table. At L3, it coordinates interdependent tasks into a workflow, such as preparing breakfast for four people. At L4, it sustains a standing role across tasks, workflows, and sessions, such as managing breakfast according to household schedules and preferences.

L1 to L4 ladder of delegable work

Figure 1: The ladder of delegable work. Higher levels include lower-level capability, but duration alone does not prove that a level has been reached.

Action policy and Embodied S2 paradigms

Figure 2: The paper contrasts a direct action policy with Embodied S2, which organizes skills, tools, verification, and adaptation around a foundation model and harness.

Responsibility has independent spatial coordinates. A workcell lies within reach and view. A room requires locomotion and re-evaluating reachability. A building exceeds one viewpoint and requires maps, place memory, and search. A site may contain multiple buildings, outdoor routes, other agents, and handoffs between regions. A level claim should therefore state responsibility level, spatial scope, duration, and task complexity.

Responsibility levelWork independently carriedNew burden on S2Evidence assessment
L1 primitiveExecute and verify one local skillArguments, contact, and immediate resultReliability is rising in narrow tasks
L2 taskCoordinate skills through a complete taskDecomposition, recovery, and acceptanceCredible demonstrations exist in unstructured settings
L3 workflowCoordinate interdependent tasksTiming, priority, shared resources, and stateLimited to company demonstrations or simulated chains
L4 roleSustain an agreed responsibility across daysLong-term memory, preference, initiative, and boundariesNo robust unaided evidence in open settings

The current evidence is strongest for narrow L1 and L2 work. Building-scale benchmarks are mainly navigation and memory tasks. Site-scale work appears in driving, aerial control, and urban navigation. The reviewed evidence does not include a benchmark for a workflow with shared resources and deadlines, nor a multi-day open-ended role. L3 and L4 are therefore research targets rather than established system capabilities.

Four research directions follow. S2 must reason under physical deadlines; maintain verifiable state across tasks and space; adapt efficiently to new environments and bodies; and improve from physical experience without fossilizing mistakes. These directions are coupled. Unreliable state corrupts long-term learning, expensive adaptation prevents workflow deployment, and late reasoning makes a sound plan unsafe.

The paper distinguishes adaptation, continual improvement, and recursive self-improvement. Adaptation changes behavior in the current task. Continual improvement preserves validated changes to tools, skills, memory, harness, or weights and demonstrates gains on later tasks. Recursive self-improvement improves the process that proposes, selects, and validates changes. Current evidence does not support recursive physical self-improvement. A change can pass existing tests and still fail the intended task, so evaluation must report held-out benefit, regression, and the contribution of model, harness, and S1 separately.

10. Limits and an independent assessment

The authors acknowledge coverage limits. Most benchmarks test one task or one episode in a workcell or room. Building-scale tests concentrate on navigation and memory. No reviewed benchmark establishes multi-task workflows with shared resources and deadlines or roles sustained across days in open environments. The evaluation framework is a proposal that future benchmarks can implement; it is not already backed by one comprehensive test suite.

They also acknowledge simulator validity, continuous-time comparison, complex acceptance, and real-robot cost. Contact, deformation, wear, and fluid behavior remain difficult to simulate. A continuously changing world does not repeat the same trajectory, so paired comparisons are hard. Vision-language judges favor success on ambiguous evidence. Real trials are expensive and often use few samples. These limits are why the reported table is a snapshot rather than a ranking.

The strongest contribution is conceptual: the paper gives model capability, harness behavior, and delegated responsibility separate names and measurement units. It also explains why a foundation-model score cannot stand in for a physical system result. The distinction is useful for both researchers and engineers because it turns vague disputes about whether "agentic robots work" into concrete questions about interface, evidence, timing, body, and task scope.

The main weakness is that the framework has no unified reference implementation. The project page presents roadmap and evaluation data, not a reproducible Embodied S2, tool protocol, or benchmark adapter. Most numerical evidence comes from other papers, many posted shortly after Astra's release, and the authors do not rerun them. Tool libraries, prompts, action budgets, and world-clock settings vary too much to make the aggregate into a controlled proof.

Causal attribution also remains unresolved. The proposal to fix the model while varying the harness, or vice versa, is sound, but real harness components are coupled. A context change can alter tool selection, retry policy, observation frequency, and safety behavior at once. Component-level attribution requires interface logs, a shared evidence format, repeated trials, and held-out scenes. The paper proposes the direction but does not supply the cross-system standard.

Safety requires the strongest caution. The 72% text refusal and 18% real-robot refusal in RoboHarm show that linguistic caution is not action-level control. A deployable system should place permissions, force and speed limits, protective stops, and human takeover outside the model's judgment. It should report safe success, unnecessary stop rate, response latency, and intervention count together. A model that can explain why an action is unsafe may still dispatch it incorrectly when arguments, timing, and physical consequences change.

Finally, L1 through L4 is a useful responsibility vocabulary, not a complete capability metric. An L4 claim also depends on task distribution, environment openness, recovery, user preference, privacy, regulation, and human collaboration. A long run in a structured laboratory does not establish an open-ended role. The paper is candid about this gap. Future work needs multidimensional acceptance that combines responsibility, space, time, and complexity rather than allowing one level label to substitute for evidence.

11. Conclusion

The paper reframes agentic robotics from "a model that emits actions" to "a system that converts intent into verified physical outcomes under constraints." The foundation model contributes general knowledge and reasoning; the harness contributes bounded context, callable capabilities, planning state, memory, execution, and verification. Performance belongs to the combination. The difficult engineering is in asynchronous execution, stale observations, irreversible effects, embodiment differences, and long-term responsibility.

Its evaluation proposal follows from that view. Test the model and the complete system separately, disclose the harness, verify completion independently, report latency and cost, and keep the world running when responsiveness is part of the claim. The existing results are promising and uneven: narrow suites can be solved well, broad embodied tasks remain difficult, and safety changes when text becomes physical action.

The paper does not prove that L3 workflows or L4 roles are solved, and it does not provide one implementation that validates the full agenda. It does provide a disciplined way to ask what has actually been demonstrated. For robot developers, the practical message is not to add a longer prompt. It is to build execution contracts, independent verification, revisable memory, real-time safety boundaries, and evaluations that count the physical clock.

A robot's maturity is not measured by how many steps it can describe, but by whether it can produce a verified outcome after the world has changed.

flowchart TD
    U[User intent] --> C[Context manager]
    U --> M[Real-time event monitor]
    C --> F[S2 foundation model]
    F --> P[Planner]
    P --> T[Tool interface]
    T --> L[Layered S2 to S1 to S0]
    T --> D[Direct permitted S0 action]
    T --> K[Generated code as policy]
    L --> X[Physical execution]
    D --> X
    K --> X
    X --> O[Observation and evidence]
    O --> V{Accepted, terminated, effect holds, intent satisfied}
    V -->|continue or recover| C
    V -->|clarify or stop| U
    M -->|interrupt and cancel| F
    M -->|predefined safety rule| S[Protective S0 response]
    S --> O
  

Related Papers

ASPIRE: Agentic /Skills Discovery for Robotics

ASPIRE: Agentic /Skills Discovery for Robotics

ASPIRE diagnoses robot-program failures from multimodal traces, repairs executable policies through evolutionary search, and compounds verified fixes into a reusable skill library that transfers across tasks, embodiments, and real robots.

Runyu Lu, Yubo Wu, Ethan Kou · 534 views · Jun 30, 2026
ASPIREAgentic Robotics
Jun 30, 2026
AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics

AgenticRoboRL: Bringing Coding-Agent Post-Training to Agentic Robotics

Agentic robot systems pair a language-model planner with a robot harness, but recent gains have largely come from stronger closed-source planners, better harnesses, and external memory rather than from improving the planner itself. AgenticRoboRL investigates whether coding-agent post-training can improve robot planners across both VLA skill invocation and control-code generation. Its recipe combines supervised fine-tuning on successful teacher trajectories with multi-turn reinforcement learning using task success as the terminal reward. A recording proxy preserves each sampled response and its actual context, while harness-specific adapters associate model calls with task outcomes and exclude infrastructure-invalid episodes. Only the planner is trained; perception and execution components remain fixed. With a 9B planner, success rises from 18.1% to 71.0% on LIBERO-PRO and from 19.2% to 72.2% on LIBERO-Plus. With the CaP-X control-code harness, the same post-training pipeline raises success from 2.7% to 28.5%.

Xinxin Liu, Bingxin Xu, Wenrui Bao · 289 views · Oct 20, 2026
Agentic RoboticsRobot Manipulation
Oct 20, 2026
VBot Exo2Ego Policy: Lightweight Data Collection and Exocentric-to-Egocentric Video Generation

VBot Exo2Ego Policy: Lightweight Data Collection and Exocentric-to-Egocentric Video Generation

We present VBot Exo2Ego Policy, a video generation model that converts third-person videos into first-person observations of a selected actor, expanding first-person operation data from real-world footage. The system includes a lightweight collection pipeline and mobile app supporting first-person, third-person, and synchronized dual-view capture, combining accessible single-view data with paired RGB and camera geometry. Built on a pretrained video diffusion transformer, it uses geometric camera guidance plus semantic-mask and hand-keypoint supervision to improve viewpoint transfer and hand-object interaction fidelity. Camera geometry is stochastically injected and gradually removed during training so inference does not require target-camera calibration. Source-video perception and reasoning, optional actor references, and multi-shot adaptation further identify the intended actor and follow operations across camera cuts.

Tong Rao, Zhihao Zhang, Yuntian Wang · 39 views · Nov 12, 2026
Embodied AIFirst-Person Video
Nov 12, 2026
XBot-Omni: An Omni-Modal Model for Embodied Understanding and Interaction

XBot-Omni: An Omni-Modal Model for Embodied Understanding and Interaction

Embodied interaction requires an agent to integrate multimodal perception with verbal and nonverbal behaviors to effectively communicate its intentions, including speech, body motion, and facial expressions. We present XBot-Omni, a Thinker–Talker–Actor architecture that extends the Omni paradigm to robotic modalities, including body motion and facial expressions, with ``X'' representing the extensibility to robotic-specific modalities. The Thinker processes text, images, audio, and video to formulate contextually appropriate responses, which are then realized by the Talker and Actor through speech and coordinated bodily expressions, respectively. In this report, we innovatively introduce the concept of ``Actor'' as a dedicated component for embodied agents: a coordinated, streaming module that leverages the native representations of the Thinker and Talker within an end-to-end pipeline, introducing only modest latency overhead. This unified architecture enables multimodal generation conditioned on omni-modal inputs directly to prevent information loss in staged designs. We comprehensively evaluate XBot-Omni across seven benchmarks, demonstrating strong multimodal understanding capabilities on both general-domain and robot-centric videos, as well as high-quality, natural, and well-coordinated motion generation. Furthermore, our results validate its capability to generate embodied multimodal responses directly from omni inputs, meeting the diverse requirements of embodied interaction. XBot-Omni enables robots not only to perceive the omni world but also to express themselves through speech and bodily behaviors, extending Omni models from conversational interaction to embodied communication in physical environments.

Kerui Gu, Xing Fu, Shuoyang Sun · 90 views · Nov 12, 2026
Embodied AIOmni-Modal Model
Nov 12, 2026

As an Amazon Associate, we earn from qualifying purchases.