PAPER DEEP DIVE
Training Reasoning Models in Simulation to Natively Use Real Robots
The authors train an open-weight 27B reasoning model to control real robots through simulation-only reinforcement learning and no demonstrations. PEAR uses robot-use reinforcement learning with next end-effector pose prediction; LIBERO success rises from 1.6% to 74.4%, and the trained model reaches 60% success on a real dual-arm robot.
Paper Information
Title: Training Reasoning Models in Simulation to Natively Use Real Robots
Authors: Jiaming Tang, Zhuoyang Zhang, Junxian Guo, Fangzhou Zhao, Yilong Zhao, Zekai Li, Xingyang Li, Yuyang Zhao, Song Bian, Zhijian Liu, and Song Han.
Resources: arXiv 2611.18562 and mit-han-lab/pear. This analysis uses code commit d604856a1b16fdccd47e22a27d4dd935e3a62014.
System: A 27B vision-language reasoning model learns robot use through simulation reinforcement learning alone, without demonstrations, supervised fine-tuning, or real-robot training data. The model emits strict JSON containing named robot parts and quantities. Its task-level action is a relative next end-effector pose; inverse kinematics and low-level control remain outside the model.
Executive Summary
PEAR turns manipulation into a verifiable tool-use problem. A long-context reasoning model observes images, calls robot and observation tools, parses execution feedback, and continues planning until it declares completion. Simulation success is optimized with GRPO. The resulting model raises LIBERO success from 1.6% to 74.4% and completes 18 of 30 trials on a real dual-arm YAM platform that it never trained on.
The Research Question: Is the Missing Capability Action or Robot Use?
Conventional vision-language-action models usually compress a task into a mapping from observations to action chunks or continuous controls. They depend on demonstrations, perform well near their training distribution, and rarely expose a separate decision process for inspecting the scene, choosing an interface, validating a parameter, and deciding whether the last action worked.
PEAR starts from a different premise. The model should explicitly maintain those decisions. Instead of outputting joint torques or an entire trajectory, it reads the current image, instruction, and history; selects a tool; supplies strict arguments; and observes the result. The loop then repeats. The paper calls this native robot use: the model knows which capabilities the robot exposes, how to invoke them, and how to act on their feedback.
This separation has two attractions. First, high-level semantic reasoning is decoupled from inverse kinematics, motion planning, and control. The language model does not need to rediscover motor dynamics. Second, the training signal can be simple: a simulator can judge whether the task succeeded, and reinforcement learning can propagate that outcome through a long sequence of tool calls. No human must label every intermediate thought.
The difficulty is equally clear. A successful episode may contain dozens of observations and actions, while the reward arrives once at the end. The model must simultaneously learn JSON syntax, spatial relations, motion scale, stopping behavior, and recovery. A malformed object, an out-of-range pose, or one bad grasp can send a long episode permanently off course. The central empirical question is whether simulation RL can acquire all of these behaviors and whether the resulting interface habits survive the move from simulator to physical hardware.
Task Formulation: An Episode as Tool Use
An episode alternates environmental observations with model responses:
$$\tau=(o_1,y_1,\ldots,o_T,y_T).\tag{1}$$Here $o_t$ is the observation returned at round $t$, and $y_t$ contains the reasoning text and tool invocation produced by the policy. The model conditions on the task prompt and the full interaction prefix:
$$h_t=(p,o_1,y_1,\ldots,o_t),\qquad y_t\sim\pi_\theta(\cdot\mid h_t).\tag{2}$$The prompt is $p$ and the model parameters are $\theta$. At termination, the environment supplies a binary outcome:
$$R(\tau)\in\{0,1\}.\tag{3}$$This differs from a conventional observation-to-action MDP because $y_t$ is not only a motor command. It contains language, a tool name, and structured arguments. The action space is therefore shaped by both natural-language semantics and a JSON schema. Reasoning and execution share one autoregressive stream, so syntax failures, bad arguments, and spatial mistakes all become part of the optimization problem.
Long context is algorithmic rather than decorative. The paper reports a 256k-token context on LIBERO and 160k on RoboDojo. An average episode contains roughly 94k tokens, 19.7 interaction rounds, and 39 to 59 images. The model must preserve the goal, moved objects, earlier failures, and current pose across this history instead of treating each image as a fresh problem.
System Architecture
PEAR consists of five tightly coupled components. The agent loop maintains the multimodal conversation, calls the model, parses tool requests, and appends observations. The protocol separates observations, actions, and final answers. The simulator tool layer executes named requests and produces new images. An asynchronous replay buffer organizes variable-length episodes into comparable groups. A distributed trainer applies group-relative advantages and a customized REINFORCE objective.
flowchart TD
P["Task prompt and robot capability contract"] --> L["PEAR agent loop: Qwen3.8-27B"]
O["Multimodal observation"] --> L
H["Full interaction history"] --> L
L --> J{"Strict JSON response"}
J -->|Observe| S["Simulator returns image and state"]
J -->|Act| K["Named robot tool with quantities"]
K --> I["External IK and controller"]
I --> S
S --> H
H --> L
J -->|Finish| R["Episode result"]
R --> B["Async replay buffer: groups of eight"]
B --> A["GRPO advantage, DAPO filtering, score centering"]
A --> T["Token-mean REINFORCE update"]
T --> L
Figure 1: A paper-specific diagram of PEAR's execution and training loop, reconstructed from the method and public code.
The model deliberately does not control joints. The tool layer knows the robot body, camera calibration, and available objects. PEAR chooses semantic quantities such as a named arm, a direction, and a relative transform. Inverse kinematics and the controller enforce joint limits and execute the motion. This preserves the symbolic composition strength of a reasoning model while avoiding the need to infer low-level dynamics from sparse task outcomes.
Protocol Design: Turning Thought into an Executable Request
The most important contribution may be the protocol rather than the model. A sentence such as "pick up the red block" cannot be sent directly to a controller. pear/agent_loop/protocol.py defines typed observation, tool-call, and final-answer messages. Each message has an allowed type, tool name, and parameter structure. The model remains free to reason, but only schema-valid output reaches the robot.
The parser makes the boundary even tighter. pear/agent_loop/tool_parser.py extracts only the final fenced JSON block:
JSON_BLOCK_PATTERN = re.compile(
r"```json\s*(\{.*?\})\s*```",
re.DOTALL,
)
Lines 19 through 47 reject missing JSON, malformed structure, and unsupported multi-tool requests. Keeping only the last block lets the model analyze first and commit to one explicit action afterward. The runtime does not have to guess intent from prose. For a long-context agent, this is a useful compromise: thought remains continuous inside the model response, while execution has an unambiguous boundary.
pear/envs/system_prompt.md lines 3 through 33 document the observation, action, and response contract. They tell the model which named parts exist, how relative motions are represented, when to inspect, and how to finish. Sharing this contract across every task creates a reusable robot-use manual. It also enables the real transfer: if the YAM controller implements the same semantic tools, the model does not need to know whether the backend is MuJoCo, RoboDojo, or physical hardware.
The cost of strictness is brittleness. A response can be semantically correct yet lose the entire action because a key, nesting level, or unit is wrong. Reward measures task completion, not JSON validity, so early training contains many zero-reward episodes. Group comparison and long-context learning gradually select behaviors that both solve tasks and satisfy the interface, but the process requires enormous rollout volume.
Episode Execution: Read the World Again at Every Step
pear/agent_loop/agent_loop.py lines 98 through 172 implement the interaction loop. It constructs a prompt from the task, current image, and history; calls the model; parses a tool request; executes it; appends the returned observation; and repeats. A final response ends the episode. Each step re-observes the scene rather than executing a predicted trajectory open loop.
pear/envs/tool.py lines 63 through 125 perform simulator execution. The layer validates the tool and arguments, maps them to a robot control interface, advances the simulator, and returns a new camera image with readable feedback. The model does not receive privileged simulator state that would be unavailable on hardware. It learns from a deployable observation contract.
Lines 198 through 229 handle continuation and score-centering placeholders. Long episodes may exceed a single generation window, requiring history continuation or compression. Rewards must also be centered at the group level so episodes do not receive disproportionate weight merely because they contain more generated tokens. Keeping these operations at the boundary between agent loop and trainer makes PEAR more than an infinite chat transcript; it is an engineered long-horizon training system.
From a robotics perspective, an average of 19.7 rounds means the policy repeatedly observes and corrects. This is more appropriate than an open-loop action chunk when objects move after contact or the camera view changes. It also exposes errors. If the model forgets that an object is already held, it may grasp again. If it underestimates a displacement, the next image reveals the mistake and the model must recover. The reported physical failures are exactly these closed-loop problems.
Learning from Binary Outcomes
PEAR computes GRPO-style advantages across rollouts of the same task. For rollout $i$,
$$\hat A_i=\frac{R(\tau_i)-\operatorname{mean}_j R(\tau_j)} {\operatorname{std}_j R(\tau_j)+\epsilon}.\tag{4}$$If all eight rollouts in a group succeed or all fail, there is no comparative signal. The implementation applies DAPO-style group filtering and keeps only groups with reward variation. This is not a cosmetic optimization. Long robot tasks have rare successes and many syntax errors; zero-variance groups would otherwise dominate compute without telling the model which behavior is better.
For token-level optimization, the context of token $k$ in round $t$ of rollout $i$ is
$$c_{i,t}^k=(h_{i,t},y_{i,t}^{<k}).\tag{5}$$The objective averages over generated behavior tokens rather than treating every token in the prompt as equally important. With retained groups $G$ and $N$ rollouts, the base REINFORCE gradient can be written as
$$g=\frac1N\sum_i\hat A_i\sum_{t,k} \nabla_\theta\log\pi_\theta(y_{i,t}^k\mid c_{i,t}^k).\tag{6}$$The configuration enables REINFORCE bypass instead of PPO clipping. It uses no KL penalty and no entropy bonus. The resulting objective is close to policy gradient with a group baseline, with fewer auxiliary knobs and a clearer attribution of change to within-task reward differences. This is not an absence of stabilization: group filtering, advantage normalization, score centering, and a fixed learning rate carry that burden.
Score centering addresses how a long multi-turn response is weighted. A trajectory better than its group receives positive weight across its generated tokens, while a worse trajectory receives negative weight. The model therefore needs neither an external value network nor manually labeled intermediate rewards. The tradeoff is high variance: two rollouts may differ in many decisions, yet the objective sees only one success bit.
PEAR uses AdamW with a learning rate of $1\times10^{-6}$ and a token-mean loss. Each step contains 128 groups of 8 rollouts, or 1,024 long episodes. This batch is large because a single robot trajectory is highly stochastic and terminal reward is sparse. Group comparison only becomes informative when the same task is attempted several times under comparable conditions.
filter_groups:
enable: true
metric: reward
rollout_correction:
bypass_mode: true
loss_type: reinforce
score_centering: true
These fields appear in pear/configs/training.yaml lines 13 through 79. They distinguish PEAR from a standard PPO recipe. Tool-use episodes are extremely long, and terminal success is binary. Per-token clipping can overfit a few successful trajectories before the model has learned reusable behavior. pear/configs/base.yaml lines 18 through 38 also define asynchronous training and a maximum staleness of five policy versions, pipelining rollout generation with optimization.
Asynchronous Rollouts and Prefix Caching
pear/trainer/replay_buffer.py lines 39 through 112 collect trajectories, form task groups, and compute group pass rates. Episode lengths vary sharply: a simple pick may finish in a few rounds, while a block-assembly task can approach the context limit. Waiting synchronously for every worker wastes accelerator time, so PEAR permits rollout workers to continue with slightly older parameters and sends completed samples to the replay buffer.
Asynchrony introduces stale-policy bias. If the model has updated too far since a trajectory was generated, the advantage and gradient refer to behavior from a different policy. The maximum staleness of five limits this mismatch, while importance weights and clipping manage the remainder. Throughput improves, but the effective batch becomes harder to interpret and reproduction must control worker count, scheduling delay, and cache state.
pear/rollout/lmcache.py lines 21 through 263 maintain prefix key-value states. Every episode reuses a long system prompt and task preamble; re-encoding all text and images from scratch would consume prohibitive compute and memory. LMCache preserves reusable prefixes and clears stale entries when the task or policy version changes. Because images alternate with actions, the cache must distinguish visual states after the same textual prefix rather than keying only on text.
The replay buffer also determines which samples belong together. Eight rollouts are useful because they share a task, not because their token lengths happen to match. A long-context robot trainer therefore has to treat data organization as part of the algorithm. A successful episode's meaning depends on the failures sampled alongside it.
Curriculum: Short Tasks First, Broader Robots Later
Training follows three stages. Steps 1 through 148 use LIBERO Goal, Spatial, and Object tasks, establishing basic end-effector movement, spatial grounding, and object interaction. Steps 149 through 225 include all four LIBERO suites, adding Long. Steps 226 through 340 introduce RoboDojo general_pickup and align_blocks, exposing the model to a second simulator, different cameras, and bimanual actions.
The ordering follows plausible dependencies. Goal and Spatial provide clearer targets; Object introduces contact; Long chains subgoals; RoboDojo changes the embodiment and scene distribution. Starting with the hardest bimanual tasks could leave too few successes for within-group comparison, allowing the model to learn protocol compliance but not task structure.
The curriculum also leaves an attribution gap. The paper does not provide a complete stage-by-stage ablation. It is therefore difficult to tell how much the final physical transfer comes from LIBERO, Long, RoboDojo, or their interaction. The second simulator is clearly important for a bimanual YAM, but the result does not establish that any additional simulator would yield the same transfer.
| Stage | Steps | Task distribution | Purpose |
|---|---|---|---|
| One | 1–148 | LIBERO Goal, Spatial, Object | Learn observation, spatial grounding, and basic manipulation |
| Two | 149–225 | All LIBERO suites, including Long | Maintain state over multiple subgoals |
| Three | 226–340 | RoboDojo general_pickup and align_blocks | Transfer the protocol to a second simulator and bimanual tasks |
Table 1: The curriculum schedule reported in the paper's appendix.
Simulation Results
The Qwen3.8-27B base model succeeds on only 1.6% of LIBERO tasks. It may understand the instruction, but it does not reliably use tools, sequence actions, or recover across rounds. PEAR reaches 74.4%, showing that simulation RL changes task competence rather than merely fixing output format. Suite results are 85.8% on Spatial, 97.0% on Object, 61.4% on Goal, and 53.2% on Long.
The difficulty gradient is informative. Object is near saturation, suggesting that pick-and-place behavior can be learned from simulation feedback. Spatial also transfers well, meaning the model can turn visual relations into tool arguments. Goal and Long remain substantially lower because they require longer state chains and reliable completion checks. The 74.4% headline hides the fact that nearly half of Long tasks still fail.
Reasoning effort is another major variable. Low effort produces 21.9%, while xhigh reaches 74.4%. This is not a small "think longer" improvement in a text benchmark. The model must retain more history, inspect more often, and avoid premature termination. Deployment has to consider latency and cost together with success, because the best reported result uses a large inference budget.
On LIBERO-Pro, PEAR scores 62.1% versus 53.3% reported for π0.5. In a new-task, familiar-scene condition it reaches 57.4%, while π0.5 is at 0.75%. After swapping object positions, PEAR reaches 57.5% versus 20.8%. These are large differences, consistent with a controller that rereads the scene and replans. The comparison is not perfectly controlled, however: the systems use different bases, training sources, and action interfaces, so the full gap cannot be assigned to PEAR's objective alone.
On RoboLab-120, PEAR reaches 37.3% and the base model 5.4%. Cosmos3-Nano is close at 36.8%; π0.5 scores 28.0%, DreamZero 25.7%, and GR00T N1.6 7.2%. The value of this result is cross-environment performance, not a large lead over every method. It also sets a sober boundary: almost two-thirds of tasks still fail, so open-world competence should not be inferred from the benchmark.
| Evaluation | PEAR | Comparison | Base or reference | Interpretation |
|---|---|---|---|---|
| LIBERO overall | 74.4% | — | 1.6% | Simulation RL adds 72.8 points |
| LIBERO Spatial | 85.8% | — | — | Reliable spatial parameter use |
| LIBERO Object | 97.0% | — | — | Object manipulation is near saturation |
| LIBERO Goal | 61.4% | — | — | Goal completion still accumulates error |
| LIBERO Long | 53.2% | — | — | Multi-stage execution remains difficult |
| LIBERO-Pro | 62.1% | π0.5 53.3% | — | Advantage on a shifted benchmark |
| New task, familiar scene | 57.4% | π0.5 0.75% | — | Strong difference under scene reuse |
| Swapped object positions | 57.5% | π0.5 20.8% | — | More robust to layout changes |
| RoboLab-120 | 37.3% | Cosmos3-Nano 36.8% | Base 5.4% | Cross-environment gain, low absolute success |
Table 2: Main simulation results reported by the paper.
Real Robots: Eighteen Successes Without Real Training Data
The physical study uses a dual-arm YAM robot and block alignment and placement tasks. The final model succeeds in 18 of 30 trials, or 60%. The LIBERO-only version succeeds 7/30, or 23.3%, and the base model 0/30. The ordering supports the intended story: the base lacks usable robot behavior, LIBERO supplies part of the manipulation skill, and RoboDojo broadens that skill enough for the bimanual target.
Crucially, no real-robot data participates in training or fine-tuning. PEAR receives real camera images and sends the same relative-pose JSON through the physical IK and controller. This demonstrates that the semantic contract can span simulation and hardware. It does not erase the reality gap. A 60% success rate leaves four out of ten trials failing, but the failures do not prevent basic task execution.
Figure 2: Paper Figure 7. The real dual-arm YAM alignment and placement experiment.
The twelve failures of the final model fall into four categories: six reach the round limit without success, three leave blocks unaligned, two topple an object, and one lets a ball roll away. This distribution is more useful than the aggregate. The round-limit failures indicate inefficient loops or unsuccessful completion verification. Alignment and toppling indicate pose or contact error. The rolling ball exposes weak prediction of movable-object dynamics.
These failures follow from the architecture. A relative-pose interface does not directly observe contact force, gripper closure, or object slip. The external controller can enforce kinematic feasibility and safety, but it cannot decide whether a semantic action is appropriate. PEAR's physical transfer is real, yet stable deployment would need richer feedback and better uncertainty handling.
The experiment is also small. Thirty trials establish a large change from zero, but they cannot produce a tight confidence interval or cover many robots, lighting conditions, materials, and camera placements. Results are not broken down by object, initial pose, or operator. The 60% figure should be treated as feasibility evidence, not a reliability certification.
| Real YAM condition | Successes | Success rate | Interpretation |
|---|---|---|---|
| Final PEAR | 18/30 | 60.0% | RoboDojo enables a transferable closed loop |
| LIBERO-only training | 7/30 | 23.3% | Short single-arm tasks do not cover the physical target |
| Base model | 0/30 | 0% | No learned robot-use behavior |
| Final model failure: round limit | 6 | — | Inefficient repetition or missing completion signal |
| Final model failure: misalignment | 3 | — | Pose accuracy or visual judgment is insufficient |
| Final model failure: toppled object | 2 | — | Contact and collision handling is insufficient |
| Final model failure: rolling ball | 1 | — | Movable-object dynamics are difficult to predict |
Table 3: Real-robot success and the failure categories reported in the paper.
Full RoboDojo: Context Exhaustion Becomes the Dominant Boundary
On the full 540-task RoboDojo evaluation, the final model succeeds on only 21 tasks. More revealingly, 370 episodes exhaust the 256k context, or 68.5%. This changes the naive interpretation. In many episodes the model is not merely refusing to act; it is still observing and calling tools while its history grows beyond the available or trained window.
Long-context capacity is expensive. Every image consumes visual tokens, and every tool return adds execution feedback. After twenty rounds, the transcript may be longer than many language models' entire training context. Without retrieval, summarization, or structured state compression, a model can know what to do and still lose the ability to finish because its own record fills the workspace.
This also explains part of the gap between 37.3% on RoboLab-120 and 3.9% on the full RoboDojo set. Task composition, success criteria, and round limits differ, but context exhaustion is visibly active. Simply increasing the window raises memory, latency, and cache pressure. A better system must learn which images to discard, turn completed subgoals into state summaries, and retrieve only the feedback relevant to the current decision.
The result exposes the weakness of a single success metric. One model may make steady progress for 100 rounds but time out; another may fail immediately with malformed JSON. Both receive zero. Reporting termination reason, rounds, context length, and progress would distinguish inability, inefficiency, syntax failure, and context overflow. PEAR's deployment case would be much stronger with that diagnostic breakdown.
Does Robot RL Damage General Reasoning?
The paper evaluates nine general benchmarks. The base model averages 81.64 and the final model 81.49, a difference of 0.15. Every paired 95% confidence interval includes zero. The authors conclude that simulation robot training causes no detectable loss of general reasoning.
This is reassuring but not proof of complete invariance. An average can hide gains in some benchmarks and losses in others. A confidence interval containing zero means the current evidence cannot reject no difference; it does not establish equivalence. The safest conclusion is that no significant degradation was observed on the selected benchmarks at the reported sample size.
The engineering implication is still important. PEAR uses one 27B model for language understanding, spatial judgment, and tool control instead of attaching an untrainable external planner. If robot RL destroyed general semantic ability, long-horizon rule following and recovery would also suffer. The result supports the possibility of adding embodied behavior without giving up the pretrained reasoning asset.
Why Relative End-Effector Pose Is a Sensible Interface
PEAR outputs the relative next end-effector pose, not joint angles or a future action chunk. This keeps the capabilities most suitable for a vision-language model inside the policy: recognizing the target, choosing a direction and arm, and replanning after feedback. It moves inverse kinematics, joint limits, velocity control, and real-time execution outside.
A relative pose also reduces sim-to-real interface mismatch. Simulators and hardware can have different joint counts, camera positions, and control rates as long as they interpret "move a named end effector along this relative transform" consistently. The model does not memorize a robot-specific absolute coordinate system or simulator body identifier. This semantic stability connects LIBERO, RoboDojo, and YAM through one protocol.
The abstraction has hard limits. Relative pose cannot directly represent force control, compliant assembly, in-hand manipulation, or fast dynamic contact. A model cannot infer from a pose alone whether the gripper closed on the object. PEAR's misalignment, toppling, and rolling-ball failures sit near this boundary. Richer tools could expose gripper width, torque, touch, and collision events, but that expands the action space and makes credit assignment harder.
Compute Cost and Reproducibility
The paper reports roughly 50,000 H100-equivalent GPU hours for training. This number matters. "Simulation-only RL" does not mean low cost. It replaces human demonstrations with enormous rollouts, long-context inference, asynchronous scheduling, repeated image encoding, and many failed attempts. At typical cloud prices, one training run is a six-figure expense, far outside routine academic reproduction.
Context is the multiplier. An average episode consumes about 94k tokens, and each update compares multiple rollouts per task. LMCache, asynchronous replay, and group filtering are therefore core infrastructure, not minor optimizations. They make the experiment executable within a budget that is still very large.
The compute barrier affects the research ecosystem. Few groups can reproduce a 27B model, 340 update steps, and 1,024 episodes per step. Public code improves transparency, but without released weights, training caches, and smaller recipes, the community cannot independently audit every design choice. A valuable next result would be a smaller model or curriculum that preserves the qualitative transfer while reducing rollout cost by an order of magnitude.
Relationship to Imitation Learning and End-to-End VLA
PEAR does not make imitation learning obsolete. Demonstrations remain highly efficient for contact-rich tasks whose success is difficult to specify as a simulator reward. PEAR addresses a different regime: tasks can be judged automatically in simulation, the robot exposes reliable low-level tools, and many environments can be rolled out in parallel.
Compared with an end-to-end VLA, the system has clearer action semantics, reusable IK, round-by-round correction, and task rules expressed in the prompt. It also has higher inference latency, dependence on long visual history, and a larger parsing and controller surface. It is better understood as a high-level robot-use agent than a universal replacement for action policies.
The comparison with π0.5 must also remain careful. PEAR's rereading and explicit planning may help with new tasks and moved objects. An action-chunk policy may still be superior for fast continuous contact. The paper does not compare both paradigms on the same real platform under equal pretraining and inference budgets, so the right conclusion is task-dependent rather than one architecture defeating the other.
Author-Stated Limitations
The reality gap remains. The physical study is small, and zero-shot transfer is not as stable as simulation. The 60% success rate and the reported failures directly show visual, contact, and dynamics mismatch.
Long context is a concrete bottleneck. In full RoboDojo, 370 of 540 episodes exhaust the 256k context. The model may continue acting but run out of room to finish. The paper does not show how state compression would restore performance across the full set.
The action interface is restricted. Relative end-effector pose depends on external IK and control. It does not directly express fine force control, touch, or in-hand dexterity. The physical errors around alignment, collision, and rolling objects reflect this boundary.
Evaluation coverage is limited. Physical testing uses one dual-arm platform and a limited block task. Thirty trials cannot establish broad generalization. Aggregate results do not fully separate differences among LIBERO, RoboDojo, and YAM.
Compute is expensive. The H100-hour estimate makes full reproduction difficult. Open code supports auditability, but independent reruns are still needed to establish whether the paper's numbers can be reproduced with the released recipe.
Independent Limitations and Risks
Terminal reward obscures credit assignment. A binary outcome does not identify the bad round. GRPO can prefer one trajectory over another, but a success may contain accidental actions and a failure may contain many correct ones. Process verification, keyframe rewards, and failure classifiers could improve sample efficiency, but their effect is not systematically evaluated.
Comparison fairness needs external auditing. PEAR has a strong base model, a long inference budget, and an explicit tool interface. Some baselines may use action chunks or different few-shot agent scaffolding. Matching task names is not enough if information access, compute, and controller reliability differ. The results do not justify the broad claim that RL is universally superior to imitation.
Physical safety is not established. IK and a controller can enforce joint range and speed, but the model can still choose the wrong object, collide with a neighbor, or repeat a harmful action. The experiment appears supervised and controlled. Open deployment requires collision checks, force limits, workspace monitoring, emergency stops, and human takeover.
Protocol errors can reach hardware. JSON validation catches syntax, not semantic danger. If the real tool layer interprets units, frames, or bounds differently from simulation, a legal model request can cause an unsafe motion. The runtime needs independent range, frame, and collision validation rather than trusting an RL-trained policy.
Compression may erase task state. Summarizing history reduces cost, but it may discard the fact that an object moved or the reason a prior attempt failed. Keeping every image preserves information but scales poorly. The paper does not identify the minimum sufficient state for long-horizon robot use; this is a central systems problem for product deployment.
Figure 3: Paper Figure 1. The overview connects simulation RL, strict tool protocols, long-context reasoning, and physical robot use.
Engineering Notes for Reproduction
A reproduction should begin by pinning the paper commit and dependencies, then testing the protocol parser. Invalid JSON must be rejected rather than silently repaired. The next milestone is a minimal closed loop across observation, named action, IK, controller, image return, and termination. Legal model output does not imply correct tool semantics; each dimension, frame, and bound needs an explicit test.
Each rollout should log termination reason, context length, number of tool calls, and group success rate. Logging only terminal reward collapses context overflow, malformed output, poor action, and timeout into the same zero. Full RoboDojo already shows why termination diagnostics matter. Replaying a successful trajectory from saved images and action logs also checks whether the result is independent of hidden state.
A useful ablation should freeze model and data while removing group filtering, score centering, LMCache, the RoboDojo stage, and asynchronous training one at a time. Measure success, valid-sample rate, throughput, and context exhaustion, not only accuracy. The most valuable reproduction may not be a full rerun but a cheaper long-context recipe that preserves transfer.
Figure 4: Paper Figure 2. One robot-use round showing observation, reasoning, tool invocation, and feedback.
Why the Result Matters
PEAR moves robot learning from memorizing actions toward learning how to use a robot's exposed abilities. In simulation, repeated failure teaches when to observe, how to specify a relative pose, and when to stop. On hardware, the same behavior runs because the tool semantics remain stable. That is a different form of embodied knowledge from an end-to-end action policy and a credible alternative abstraction for robot foundation models.
The second contribution is placing long-context reasoning inside the control loop rather than asking it only for an initial plan. Every result changes the next input, so the model must maintain state, detect failure, and select another tool. The jump from 1.6% to 74.4% shows that pretrained language ability alone is insufficient; verifiable reward and an executable interface are what turn a general model into an agent.
The third contribution is transparency through code. Protocol, prompts, tool execution, replay, configuration, and cache logic are inspectable. This lets readers separate the paper's method description from the actual runtime and exposes the systems cost of long-context agency. Full external reproduction is still needed, but the code makes the claims auditable in detail.
Conclusion
PEAR gives a clear but expensive answer: a 27B reasoning model can learn robot use through simulation RL alone and transfer part of that behavior zero-shot to hardware. It raises LIBERO from 1.6% to 74.4% and succeeds on 18 of 30 real dual-arm trials. At the same time, Long tasks remain at 53.2%, most of full RoboDojo exhausts context, and physical failures cluster around alignment, contact, and movable objects. The system is a strong capability demonstration, not a finished robot policy.
The next research step should attack state rather than only scale. A deployable agent needs compressed object-centric memory, explicit subgoal status, and the ability to forget irrelevant imagery. It also needs richer tool feedback such as gripper width, force, touch, and collision events. Finally, physical evaluation should be larger and report failures by task, object, initial state, and termination reason instead of only aggregate success.
In the longer arc, PEAR represents a robot-use layer for reasoning models. It does not replace low-level control and does not force every capability into one network. It learns a verifiable protocol for invoking capabilities and recovering from mistakes. If cost, context, and contact feedback improve, this abstraction may fit tool-rich, decomposable, replanning-heavy robots better than a single end-to-end action policy.
SOURCE LINKS



