OPEN SOURCE DEEP DIVE
ASPIRE: Agentic /Skills Discovery for Robotics
ASPIRE is the continual-learning system NVIDIA open-sourced with the University of Michigan, UIUC, Berkeley and CMU (paper arXiv:2607.00272, Apache-2.0). No gradient descent: a coding agent repeatedly writes, runs and repairs robot control programs (code-as-policy) inside a closed-loop execution engine, promotes only verified fixes into a reusable skill library, and uses evolutionary search to escape single-trajectory patching. Driven by Claude Opus 5.5 it averages 99% success across all 80 LIBERO-Pro tasks and 100% on representative Robosuite and BEHAVIOR-1K tasks; perturbed manipulation improves by up to 77%, and unseen long-horizon tasks reach 31% zero-shot against a 4% baseline. On hardware, skills discovered in simulation transfer across embodiments to a real dual-arm station, and online learning even teaches the robot to solve a Rubik's Cube.
Training, redefined: skill libraries instead of gradient descent
ASPIRE is a continual learning system open-sourced in 2026 by NVIDIA with the University of Michigan, UIUC, UC Berkeley and CMU (paper arXiv:2607.00272, Apache-2.0). It replaces three familiar nouns at once: training is skill refinement rather than gradient descent; the trained model is a repository of sensorimotor skills rather than floating weights; distributed training is a panel of agents each practicing a different skill rather than sharded minibatches. Skills exist as Python programs in a code-as-policy (CaP) paradigm: coding agents such as Codex or Claude Code write them, execute them, diagnose their failures, repair them, and distill validated fixes into a library that later tasks reuse.
The numbers behind that reframing are hard to ignore. Across all 80 LIBERO-Pro Goal / Object / Spatial / Long tasks, ASPIRE driven by Claude Opus 5.5 averages 99% success, with 100% on representative Robosuite and BEHAVIOR-1K tasks; gains reach 77% on manipulation under perturbation, 72% on Robosuite's bimanual handover, and 32% on long-horizon household tasks. The accumulated library also transfers zero-shot: 31% success on representative unseen long-horizon tasks versus 4% for prior methods. On hardware, skills discovered in simulation cut real-robot programming effort across embodiments, and skills acquired online on a physical dual-arm station let the robot solve a Rubik's Cube with progressively improving efficiency.
The closed-loop engine: a restricted program runtime with auditable run artifacts
Generated programs execute inside the cap-harness runtime under a strict boundary: a generated program.py cannot import modules and receives no NumPy, filesystem, network, native simulator, reward, or success-predicate access. It can only call injected top-level tools. They come in two layers: shared_atomic primitives (get_observation, segment_text, mask_to_point_cloud, estimate_geometry, generate_grasps, solve_ik, plan_motion, step, execute_trajectory, set_gripper) and shared_high_level local compositions of them (localize_object, select_grasp, move_to_pose, move_to_joints, open_gripper, go_home). Each benchmark injects its own extension namespace: libero.*, robosuite.*, and behavior.* (which adds holonomic-base navigate_to_pose and torso move_torso).
Perception and planning live outside the runtime as long-lived HTTP provider services: SAM3 for text and point-prompt segmentation, Contact GraspNet for grasp candidates, PyRoki for IK, and cuRobo for collision-aware trajectory planning. The topology is declared in configs/environments.json and bound to hardware by configs/profiles/ (shipping rtx5090 and l40). Simulator environments (LIBERO, Robosuite, BEHAVIOR-1K) install only client dependencies and share one provider stack, which keeps simulator installs light and lets a single provider instance serve every simulator client.
Every cap-harness run writes an immutable episode: run.json (identity and capture settings), outcome.json (program status, task and protocol success, aggregate reward, and a disjoint timing bucket map), manifest.json (SHA-256 per finalized file), episode/ (one action and robot-state record per control tick), trace/ (parent-linked program and provider spans), and media/ (one H.264 MP4 per camera, lossless keyframes around public motion calls, and top-1 SAM mask overlays). Reward and success predicates never appear in anything the generated program can read, so the agent must judge outcomes from visual and state evidence rather than leak an oracle into its policy. Recording costs about 4 ms on a 1000-step episode.
The fix loop: debug by failure mode, promote only reproducible fixes
The learning loop ships as two agent skills, iterative-debugging (the fix loop) and evosearch (evolutionary search), packaged for both Claude Code and Codex. In the fix loop a coordinator agent also acts as curriculum: each round it picks tasks from what the library can currently teach and dispatches one worker per pending task. A worker inspects a single initial observed scene, writes code, then debugs on development seeds (51-65 on LIBERO); debugging is organized by failure mode rather than by seed, and every version is scored on all assigned development seeds. The selected program is measured once on unseen-check seeds (66-80) before the coordinator runs the held-out evaluation on seeds 1-50.
Skill promotion has a hard gate: when a task proposes a new skill, a separate verifier agent must re-solve the task from the skill library and the proposal text alone (without the proposer's code) within two attempts on the first development seeds; only verified proposals are promoted into skills/. The library therefore cannot accumulate fragments that merely worked on one seed. Evosearch targets tasks the fix loop cannot crack: it seeds candidate_A verbatim from the fix loop's fix_code.py and runs a K=8 multi-candidate evolutionary search before the same validation and held-out protocol.
flowchart LR
A["Observe initial scene
RGB-D + joint state"] --> B["Agent writes program.py"]
B --> C["cap-harness records run
video / keyframes / trace"]
C --> D{"Debug on dev seeds 51-65
by failure mode"}
D -- below threshold --> E["Read trace and keyframes
synthesize repair"]
E --> B
D -- passes --> F["unseen-check 66-80
then held-out 1-50"]
F --> G{"Verifier reproduces fix
from library alone, 2 tries?"}
G -- verified --> H["Promote into skills/ library"]
G -- failed --> I["Stay in task directory"]
H --> B
Evidence: three benchmarks, twelve models, one cost frontier
ASPIRE scales with the capability of the underlying coding agent. On 10 LIBERO-Pro tasks, success rises from 15% with GPT-6 Luna to 99% with Claude Opus 5.5, and the strongest models are not the most expensive: Sonnet 5.5 also reaches 99% at $12 per task while Opus 5 spends $60 for 91%. Across LIBERO-Pro, Robosuite and BEHAVIOR-1K, Opus 5.5 lands between 97% and 100% on every task.
| Coding-agent model | Success on 10 LIBERO-Pro tasks | API cost per task |
|---|---|---|
| GPT-6 Luna | 15% | $0.83 |
| Haiku 5.5 | 68% | $4 |
| GPT-6.1 Sol | 88% | $10 |
| Sonnet 5.5 | 99% | $12 |
| Claude Opus 5.5 | 99% | $18 |
| Opus 5 | 91% | $60 |
The long-horizon household activities of BEHAVIOR-1K (R1 Pro bimanual robot with holonomic base in Isaac Sim) show the same model ladder, with Opus 5 and Opus 5.5 saturating four representative tasks and GPT-6 Astra close behind.
| BEHAVIOR-1K task | Opus 4.6 | Opus 5 | Opus 5.5 | GPT-6 Astra |
|---|---|---|---|---|
| Trash Navigation | 92% | 100% | 100% | 96% |
| Trash Task | 88% | 100% | 100% | 96% |
| Radio Navigation | 100% | 100% | 100% | 96% |
| Radio Task | 88% | 92% | 100% | 96% |
Real robots: the System 2 operator runbook and the YAM dual-arm station
The real-robot side is a separate execution stack documented in the docs/system2.md operator runbook. The topology: SAM3 (port 8114) and cuRobo (8118) providers, one yam-servers process per arm (SocketCAN to DaMiao motor drivers), a CAP MCP server on 8222, and a Viser page on 8080 with a live 3D robot model, camera views and a Go home button. --allow-motion is the safety gate: without it the server rejects actuation and disables the Home button; with it, a fresh agent session opens both grippers, homes both arms, and reports home_verified plus joint residuals against a 0.03 rad tolerance.
The repository's AGENTS.md sets the System 2 agent's execution discipline: act only through registered mcp__cap__* tools, never through source edits or shell-wrapped motion; keep an object-centric working state (plan, object identities, verified relations, unresolved predicates); and treat robot pose and gripper closure as execution reports, not as proof of grasp, placement or task completion, which must be established from returned images and measurements. Four text-based task recipes ship with the repo: Rubik's Cube, unscrew-and-pour, pill insertion with bottle closure, and inverted cup nesting. They specify contact strategy, recovery paths and physical completion evidence without episode recordings or fixed-coordinate replays.
Real-world results run along two lines. Cross-embodiment sim-to-real: the v0.1 long-horizon table-bussing skill was discovered in simulation by Claude Opus 4.6 and executed on hardware by GPT-5.5, with the skill text reused across embodiments and robot APIs. Online real-world learning: GPT-6 Astra practices directly on the physical station, solving the Rubik's Cube, nesting cups, opening bottles and closing pills in, with cube-solving efficiency improving across sessions as the skill library compounds on hardware.
Safety boundary and engineering discipline
ASPIRE states its safety boundary with unusual bluntness. Generated programs inside the CaP runtime are restricted (no imports, no filesystem or network, no reward), but the coding agent's own trial processes are not a sandbox: they execute language-model-generated Python with full import access, and watchdogs do not provide isolation. The README therefore requires running generated code on an isolated host without credentials or sensitive mounts, restricting network access, and never granting a simulation agent access to physical hardware; real-robot work additionally requires the runbook's controls and separate operator authorization.
The engineering notes read like a field manual worth copying. The provider supervisor traps INT/TERM/EXIT in the foreground, so closing the terminal takes the whole stack down without restarting it (detach it yourself with setsid nohup). On LIBERO and Robosuite the gripper command is a velocity, not a position: set_gripper accepts only 0.0 and 1.0, and intermediate values burned 60 ticks in measurements while ending fully open. go_home does not converge in one call from a folded pose and needed three consecutive calls in a measured case; calls accumulate, which is why repeating works. MuJoCo's EGL device enumeration ignores CUDA_VISIBLE_DEVICES, so MUJOCO_EGL_DEVICE_ID must be set as well, or a contended GPU 0 surfaces as intermittent depth failures that look like policy bugs. Requesting an unavailable planning backend returns a typed failure; the harness never falls back implicitly.
Getting started
The shortest simulation path is handing the repository to a coding agent: the README quick-start prompt has it read .claude/skills/iterative-debugging/SKILL.md and docs/libero-pro.md, then run the libero_goal_swap suite over ten tasks. Host prerequisites (CUDA, uv, and the Hugging Face token for gated SAM3 weights) are in docs/host-setup.md. Environments bootstrap independently via scripts/bootstrap_libero.sh, bootstrap_robosuite.sh and bootstrap_behavior.sh, and an unchanged bootstrap is a fingerprint-checked no-op. Structural health uses cap-harness doctor and cap-harness validate --plan (one cross-embodiment entry point); single programs run with cap-harness run and seed sweeps with run-batch. For paper-scale reproduction, name the suite and experiment explicitly (fix loop, evolutionary search, zero-shot transfer, library-size scaling, inference-time scaling, or the debugging-token ablation); the agent then reports tasks, seed schedule, trial counts, GPU and credential requirements and waits for confirmation before launching.
ASPIRE's deeper claim may not be any single benchmark number: it turns what a robot has learned into a repository that can be diffed, reviewed and contributed to under a DCO sign-off. Skills are text, progress is a commit, and compounding is the library growing.