Papers

WaterfallPagination

AllShow tags

Humanoid106VLA77Reinforcement Learning76Embodied AI68World Models62Whole-Body Control40Research28Mobile Manipulation253dgs243D Gaussian Splatting22Imitation Learning22vision-language-action22Quadruped robots21Dexterous Manipulation20Gait optimization17Vision-language navigation16Loco-manipulation12SLAM12Simulation12Vision-language models12

OmniBrain-0: A Unified Agentic Model for Embodied Intelligence

General-purpose embodied intelligence requires robots to coordinate diverse skills over long horizons, track task progress, and recover from failures. Specialist policies alone do not ensure task-level reliability, while powerful multimodal agents rely on costly reasoning and external experience that compact models struggle to reproduce. We introduce OmniBrain-0, a lightweight unified embodied agent that coordinates manipulation and navigation through a shared high-level decision model, integrating task understanding, planning, progress tracking, and feedback-driven recovery. Our progressive training framework combines embodied pretraining, domain-specialized agent distillation, and multi-teacher on-policy distillation (MOPD). It distills powerful teachers' interaction experience into manipulation and navigation experts, then consolidates their decision capabilities into OmniBrain-0. Unification therefore occurs within the learned decision model rather than only at the tool interface. We further introduce Rubric-Guided Recursive Self-Improvement (RSI), a data flywheel in which a Meta-Agent turns execution evidence into refined rubrics, step-level reminders, and verified trajectories for domain-expert updates before their reconsolidation into OmniBrain-0 through MOPD. We evaluate OmniBrain-0 on LIBERO-Pro, LIBERO-Plus, RoboCasa365, RoboTwin 2.0, and R2R-CE Val-Unseen, as well as multiple real-world manipulation tasks. OmniBrain-0 combines strong cross-domain task performance with substantially lower inference and execution overhead than its closed-source teacher, demonstrating efficient transfer and consolidation of embodied capabilities into a compact unified agent.

Embodied AIVLA

Xiaomi Embodied Intelligence Team, Yuechen Luo, Chong He · 81 views · Oct 26, 2026

GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulation

Pretrained vision-language-action (VLA) policies provide strong priors for robot manipulation, yet adapting them online to fine-grained biomedical tasks remains challenging. Task success often hinges on subtle, view-dependent visual cues, while task-level rewards provide little guidance about which regions matter, making it difficult to learn task-relevant visual grounding from limited real-robot interaction. Online adaptation is further constrained by the computational cost of VLA inference and replay-based updates. We introduce GRAFT (Grounded Reinforcement Adaptation for Fast Task Learning), a framework for efficient online VLA adaptation through grounded perception. GRAFT uses region-level supervision to learn view-specific visual anchors that focus perception on task-relevant local cues without requiring region proposals at deployment. It further combines single-step action generation with cached visual-language prefix reuse to accelerate online learning. Across four biomedical manipulation tasks, GRAFT improves success rates by 32.5 percentage points under matched adaptation budgets, while reducing the computational overhead of online policy updates.

VLARobot Manipulation

Yibo Qiu, Haoliang Ye, Shu'ang Sun · 77 views · Aug 27, 2026

SERF: Spatiotemporal Environment and Robot Feature Map for Long-Horizon Mobile Manipulation

SERF represents the environment and the articulated robot body as neural points in one shared latent space and updates them online. Environment points follow object-level rigid registration, while robot surface points are posed by forward kinematics. A shared residual decoder maps local point features to DINOv3 patch embeddings, and a two-stage Point Transformer produces eight map tokens from three robot-centered radii, both end effectors, robot-only, environment-only, and global branches. These tokens condition a VLA policy together with images, proprioception, and language. Across three long-horizon BEHAVIOR-1K tasks, SERF raises average progress from 44.0% for fine-tuned PI0.5 to 58.7%, and the paper also reports gains when labels and base poses are estimated onboard, when object-drop recovery is required, and in unvisited regions.

VLAMobile Manipulation

Sunghwan Kim, Byeonghyun Pak, Kehan Long · 70 views · Jun 11, 2026

RoboColosseum: Evaluating Physical Intelligence at Scale In-The-Wild

General-purpose robot policies must work across robots, tasks, and environments, yet the variation that makes physical trials informative also makes them difficult to reproduce. We introduce RoboColosseum, a distributed evaluation framework that combines unified policy serving and standardized protocols across a network of locally operated robots in diverse environments. Our initial study evaluates five vision-language-action (VLA) policies and three frontier AI agents across four embodiments (SO-101, Franka, Bimanual-YAM, and G1 Humanoid). The Open Track compares models without task-specific adaptation in blinded, pairwise trials under matched local conditions. The Fine-tune Track measures how well VLA policies adapt to 20 tasks across all four embodiments when each policy receives the same 100 demonstrations per task. Across 5,699 pairwise Open Track comparisons at 124 sites, frontier agents rival specialized VLA policies, while all systems struggle to complete open-world tasks reliably. After fine-tuning, GR00T N1.7 leads on SO-101 and pi0.5 leads on Franka and Bimanual-YAM. Policy rankings show weak agreement across embodiment-specific task suites (Kendall W = 0.24). Open Track rankings poorly predict relative performance after fine-tuning, with substantial rank reversals on all three embodiments shared between tracks. Measuring partial progress reveals differences that task success alone misses. On the G1 Humanoid, no policy completes any of the five tasks, yet average progress ranges from 13.3% to 23.2% across policies. Open Track rankings stabilize as trials accumulate, and active matchmaking achieves comparable ranking agreement with substantially fewer physical trials than uniform sampling. These results show how distributed physical evaluation can reveal where robot policies generalize, how well they adapt to new tasks, and how far they progress on tasks they fail to complete.

Physical AIVLA

Tan Sang Nguyen, Yi Ru Wang, Shailesh Shailesh · 140 views · Nov 2, 2026

Democratizing Robot-Use Agents: A Unified Harness with Open Models

Robot-use Agents (RUAs) have shown promising zero-shot manipulation capabilities, but reliance on frontier, proprietary models limits research into further training. To democratize RUA research, we introduce uni-robot-agent, a unified harness for competitive open-weight robot use that supports further training. Without privileged inputs, an out-of-the-box Qwen3.8-Flash-Next achieves a RoboDojo Score of 31.39, surpassing the published GPT-6 Astra-RoboProbe result of 28.97 and many VLA/WAM baselines, and performs competitively on 19 real-world tasks. Our analysis across model generations, families, and sizes links effective robot use to four general capabilities, suggesting that future RUA training should develop them jointly.

Robot-Use AgentsOpen models

Jiarui Yuan, Yuxin Zuo, Xiaoyu Huang · 160 views · Nov 1, 2026

Real2Sem2Real: 3D Semantic Augmentations for Robot Learning from a Single Human RGB Demonstration

Human video scales easily and does not need robot hardware, but existing methods require depth sensing, overlay the robot onto 2D video, or generate data only for the demonstrated objects. We present Real2Sem2Real (R2Sem2R), a real-to-semantic-to-real pipeline that turns one monocular RGB video of a human demonstration into thousands of robot demonstrations across the semantic object instances of a category. R2Sem2R tracks the SE(3) pose of each object with DINOv3-distilled Gaussian splats, outperforming existing methods and FoundationPose with estimated metric depth. R2Sem2R then aligns digital cousins, each generated from one or a few photographs, to the demonstrated objects with a semantic alignment. This alignment matches whole objects and the functional regions of target objects, such as a coffee machine's tray. R2Sem2R maps the human hand to a feasible robot grasp that extends to the cousins. Robot demonstrations are generated either in IsaacSim, for photorealistic rendering, or MuJoCo, for contact physics, and R2Sem2R can generate over 2,000 episodes in about two hours. On two real robots and four tasks, we train a vision-language-action policy on data from one human video per task, and policies trained on our data match or exceed policies trained on teleoperation across diverse object instances. The project page is available at https://real2sem2real.github.io/.

Robot LearningHuman demonstrations

Matthew Strong, Ankush K. Dhawan, Julien Poffet · 180 views · Oct 20, 2026

RoboFoundry: System-as-Policy Evolution for Self-Learning Embodied Agents

RoboFoundry freezes the foundation model and treats the surrounding context, skill, and semantic-binding system as the evolving policy, allowing embodied agents to diagnose failures, validate persistent repairs, and transfer improvements across benchmarks and robots.

Embodied AgentSystem-as-Policy

Jingsong Liang, Shuhao Liao, Shizhe Zhang · 159 views · Sep 26, 2026

EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data

EgoScale is a human-to-dexterous-manipulation transfer framework. A VLA policy is pretrained on 20,854 hours of action-labeled egocentric human video, revealing a log-linear relationship between data scale and validation loss. Lightweight aligned human-robot mid-training then grounds the learned motor prior in robot control, improving average success by 54% over a no-pretraining baseline on a 22-DoF dexterous hand and transferring to lower-DoF hands.

Dexterous ManipulationVLA

Ruijie Zheng, Dantong Niu, Yuqi Xie · 198 views · Feb 18, 2026

Self-Adaptive VLA for Robust Robot Deployment

This post-training method turns a policy's own shifted rollouts into latent context, uses a lightweight encoder and adaptive layer normalization to adapt a vision-language-action policy, and recovers over 80% of base performance across four precision-critical manipulation tasks.

VLATest-Time Adaptation

Hongxin Zhang, Chunru Lin, Tsun-Hsuan Wang · 190 views · Sep 24, 2026

PACE: Temporal-Role Tactile Conditioning for Chunk-Based Insertion Policies

Contact-rich insertion requires a robot to combine visual alignment with tactile adjustment. Existing visuotactile chunk-based policies often collapse tactile feedback into one generation-time pathway, so persistent contact state, abrupt contact events, and the future contact consequences of an action are modeled implicitly in a single representation. PACE assigns one tactile stream three temporal roles. An exponential-smoothed contact state modulates action-chunk generation through layer-wise feature conditioning. A short-window tactile difference forms an event cue and drives a gated residual correction over only the first steps of the chunk. A training-only head predicts future tactile increments from decoder states as an auxiliary representation-learning objective. This preserves chunk-level action continuity while giving slowly varying contact phases and fast collision or jamming events separate pathways. Across five real insertion tasks, PACE reaches 90.7 percent average success, more than 44 percentage points above vision-only and single-path tactile-fusion baselines. It also averages 88.7 percent on six ManipulationNet geometry and clearance settings, including 0.1 mm clearance. Ablations show complementary contributions from Phase, Event, and Pred., while cross-signal and cross-architecture tests extend the principle to force/torque, joint current, and diffusion-policy backbones.

Tactile SensingRobot Manipulation

Zhiyuan Chen, Shixiong Xu, Shuai Chen · 286 views · Oct 12, 2026

Can World and Action Modeling Benefit Each Other under Unified Causal Training?

World and action modeling are closely related, yet their reciprocal benefits under unified causal training remain unclear. We introduce OneWM, a simple framework for unified causal training that treats actions as both conditions for world prediction and targets for policy learning within a shared video-action architecture. Using this framework, we investigate whether learning forward dynamics conditioned on specified actions improves action learning, and whether learning to generate actions in turn improves action-conditioned world modeling. We use LIBERO, LIBERO-Plus, and RoboCasa365 as embodied testbeds to evaluate both policy learning and world modeling, complemented by real-world policy experiments. Learning the visual consequences of specified actions improves policy performance and generalization across changing scenes. In turn, action learning improves multi-view world-prediction fidelity under the same architecture. Together, these results answer both questions positively, supporting reciprocal benefits between world modeling and action generation under unified causal training.

World ModelsRobot Manipulation

Yue Su, Bokai Lin, Xinkai Wang · 230 views · Oct 15, 2026

Training Generalist Navigation Policies via Action Self-Distillation

Action Self-Distillation turns future trajectory feedback into an inferred language goal, then trains one open-source vision-language model to predict reasoning and ego-frame paths from onboard observations without human language or reasoning labels.

NavigationVLA

Bowen Feng, Keqin Wang, Yulong Yang · 302 views · Oct 13, 2026

As an Amazon Associate, we earn from qualifying purchases.