Papers

WaterfallPagination

AllShow tags

Humanoid106VLA77Reinforcement Learning76Embodied AI68World Models62Whole-Body Control40Research28Mobile Manipulation253dgs243D Gaussian Splatting22Imitation Learning22vision-language-action22Quadruped robots21Dexterous Manipulation20Gait optimization17Vision-language navigation16Loco-manipulation12SLAM12Simulation12Vision-language models12

OmniBrain-0: A Unified Agentic Model for Embodied Intelligence

General-purpose embodied intelligence requires robots to coordinate diverse skills over long horizons, track task progress, and recover from failures. Specialist policies alone do not ensure task-level reliability, while powerful multimodal agents rely on costly reasoning and external experience that compact models struggle to reproduce. We introduce OmniBrain-0, a lightweight unified embodied agent that coordinates manipulation and navigation through a shared high-level decision model, integrating task understanding, planning, progress tracking, and feedback-driven recovery. Our progressive training framework combines embodied pretraining, domain-specialized agent distillation, and multi-teacher on-policy distillation (MOPD). It distills powerful teachers' interaction experience into manipulation and navigation experts, then consolidates their decision capabilities into OmniBrain-0. Unification therefore occurs within the learned decision model rather than only at the tool interface. We further introduce Rubric-Guided Recursive Self-Improvement (RSI), a data flywheel in which a Meta-Agent turns execution evidence into refined rubrics, step-level reminders, and verified trajectories for domain-expert updates before their reconsolidation into OmniBrain-0 through MOPD. We evaluate OmniBrain-0 on LIBERO-Pro, LIBERO-Plus, RoboCasa365, RoboTwin 2.0, and R2R-CE Val-Unseen, as well as multiple real-world manipulation tasks. OmniBrain-0 combines strong cross-domain task performance with substantially lower inference and execution overhead than its closed-source teacher, demonstrating efficient transfer and consolidation of embodied capabilities into a compact unified agent.

Embodied AIVLA

Xiaomi Embodied Intelligence Team, Yuechen Luo, Chong He · 82 views · Oct 26, 2026

RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments

RobotWorld introduces a cross-embodiment robot-use benchmark spanning 84 tasks and evaluates whether multimodal agents can convert perception and computation into verifier-confirmed physical execution.

Robot BenchmarkEmbodied AI

Zhiqin Yang, Chenxin Li, Xiaomeng Hu · 70 views · Oct 7, 2026

SERF: Spatiotemporal Environment and Robot Feature Map for Long-Horizon Mobile Manipulation

SERF represents the environment and the articulated robot body as neural points in one shared latent space and updates them online. Environment points follow object-level rigid registration, while robot surface points are posed by forward kinematics. A shared residual decoder maps local point features to DINOv3 patch embeddings, and a two-stage Point Transformer produces eight map tokens from three robot-centered radii, both end effectors, robot-only, environment-only, and global branches. These tokens condition a VLA policy together with images, proprioception, and language. Across three long-horizon BEHAVIOR-1K tasks, SERF raises average progress from 44.0% for fine-tuned PI0.5 to 58.7%, and the paper also reports gains when labels and base poses are estimated onboard, when object-drop recovery is required, and in unvisited regions.

VLAMobile Manipulation

Sunghwan Kim, Byeonghyun Pak, Kehan Long · 70 views · Jun 11, 2026

Route Anything: A Study of Model and Policy Routing for General-Purpose AI Systems

Route Anything unifies model and policy routing across text, images, video, speech, code agents, and robot manipulation. Its 25-benchmark trajectory matrix shows 5.38 points of per-input oracle headroom over the best fixed candidate; learned routers gain 0.30 points overall, while priced routes raise efficiency by 8.82x on average and a certified zero-loss code route holds its quality floor in all 200 splits.

LLMModel Routing

Pengfei Zhou, Yixing Ma, Hexin Wang · 94 views · Nov 3, 2026

Future Representation Distillation for Imagination-Free World Action Models

DreamFree turns a pretrained video diffusion model into a predictive visual encoder. A frozen teacher observes current and future frames, while a student predicts the teacher future representation from the current observation alone, so deployment avoids future-video denoising. Two-stage training combines future representation distillation with native flow matching, then learns an action readout and an action DiT. The 14B model reaches strong results on LIBERO, RoboCasa-Kitchen, and real DROID manipulation while running closed-loop control at 11 Hz, 17.8 times faster than DreamZero.

World-action modelsVideo Diffusion

Jaehyeong Jo, Sangwon Jang, Sihyun Yu · 86 views · Nov 3, 2026

HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness

Embodied navigation requires an agent to ground instructions or object goals in spatial observations and turn plans into execution. HarnessVLN is a zero-shot, training-free agent harness that coordinates perception, hierarchical event memory, a persistent spatiotemporal graph, and navigation tools through a shared interface. Before dispatch, the harness checks evidential support, geometric feasibility, and subgoal consistency; structured execution feedback then updates task progress, spatial evidence, and failure records for subsequent planning, recovery, and stopping. HarnessVLN reaches success rates of 60.8% on R2R, 53.9% on RxR, 76.0% on HM3D-v2, and 59.3% on HM3D-OVON, outperforming the training-free methods compared in the paper. Three real-world scenarios on an AgiBot A3U humanoid demonstrate deployment feasibility across both navigation families. Project page: https://agibot-harnessvln.netlify.app/.

Vision-language navigationEmbodied AI

Yang Chen, Lirong Che, Zhenyu Huang · 106 views · Sep 14, 2026

RoboColosseum: Evaluating Physical Intelligence at Scale In-The-Wild

General-purpose robot policies must work across robots, tasks, and environments, yet the variation that makes physical trials informative also makes them difficult to reproduce. We introduce RoboColosseum, a distributed evaluation framework that combines unified policy serving and standardized protocols across a network of locally operated robots in diverse environments. Our initial study evaluates five vision-language-action (VLA) policies and three frontier AI agents across four embodiments (SO-101, Franka, Bimanual-YAM, and G1 Humanoid). The Open Track compares models without task-specific adaptation in blinded, pairwise trials under matched local conditions. The Fine-tune Track measures how well VLA policies adapt to 20 tasks across all four embodiments when each policy receives the same 100 demonstrations per task. Across 5,699 pairwise Open Track comparisons at 124 sites, frontier agents rival specialized VLA policies, while all systems struggle to complete open-world tasks reliably. After fine-tuning, GR00T N1.7 leads on SO-101 and pi0.5 leads on Franka and Bimanual-YAM. Policy rankings show weak agreement across embodiment-specific task suites (Kendall W = 0.24). Open Track rankings poorly predict relative performance after fine-tuning, with substantial rank reversals on all three embodiments shared between tracks. Measuring partial progress reveals differences that task success alone misses. On the G1 Humanoid, no policy completes any of the five tasks, yet average progress ranges from 13.3% to 23.2% across policies. Open Track rankings stabilize as trials accumulate, and active matchmaking achieves comparable ranking agreement with substantially fewer physical trials than uniform sampling. These results show how distributed physical evaluation can reveal where robot policies generalize, how well they adapt to new tasks, and how far they progress on tasks they fail to complete.

Physical AIVLA

Tan Sang Nguyen, Yi Ru Wang, Shailesh Shailesh · 142 views · Nov 2, 2026

See What I Need, Dream What I Want: A Navigation Foundation Model on a Self-Constructed Representation Space

RepresentNav learns which features a navigation policy needs across three frozen vision backbones, predicts four future states in that same latent space, and emits eight waypoint poses; simulated benchmarks, latent probes, and physical-robot trials test whether task-shaped seeing and dreaming improve control.

Robot navigationEmbodied AI

Jiawei Chen, Yinxi Wang, Zihan Yang · 136 views · Oct 22, 2026

D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models

Vision-Language Models (VLMs) hold promise for zero-shot video task-progress estimation in robot learning, yet existing frameworks frequently suffer from confirmation bias, hallucinating progress in failed executions due to implicit success assumptions or relative temporal ranking designs. Naive prompting of frontier models collapses into a binary success detector that fails to capture nuanced, non-linear progress and failures. While post-training approaches can mitigate this through explicit training, they demand expensive real-world robotics data and require computationally intensive post-training for every individual base model. To resolve these limitations, we introduce D&A (Describe & Assess): our approach decouples progress estimation into an objective, task-grounded textual description of scene dynamics followed by an assessment phase using explicit rubrics. By compelling the VLM to synthesize an explicit, text-based definition of absolute success before scoring, the framework anchors evaluations to an external, global criterion rather than the video's internal timeline. Empirical results demonstrate that our method calibrates zero-shot estimates and mitigates confirmation bias, providing fine-grained, partial progress metrics that accurately penalize failed executions while correctly interpolating successful runs. Being purely prompt-based, D&A generalizes across tasks, embodiments, and benchmarks—matching or surpassing post-trained reward models—and its reward estimates enable efficient downstream reinforcement learning.

VLMRobot Reward Modeling

Marius Memmel, Shawn Yin, Urvi Rutia · 126 views · Nov 2, 2026

T2-z: Test-Time Optimization of Environment Latents for Robotic World Models

Predicting how the world responds to an action requires more than what can be seen: robots must account for environmental factors that are neither directly observable nor explicitly specified, from physical properties such as friction and mass to human-defined environmental rules. We introduce T2-Z for test-time optimization of environment latents, which augments a robotic world model with a learnable environment latent to condition world predictions. This compact latent captures underlying environmental variation without supervision of what environmental factors exist or what values they take. During training, T2-Z alternates between two stages: a world-model optimization stage that fixes the latents and learns general action-conditioned dynamics across environments; and an environment-latent optimization stage that freezes the world model and optimizes the conditioning latents, allowing latent structure to emerge purely through world prediction. At test time, the world model remains frozen and a newly initialized environment latent is optimized from one or more test-time interactions, adapting the model to previously unseen environmental configurations. Across 7 simulated settings and 4 real-world robotic tasks, T2-Z reduces object prediction error metric from 15.20 to 7.00 and raises action success from 33.73% to 72.96%. Its emergent latents capture structured representations of individual and compounded factors across diverse visual scenes. Ablations further show that T2-Z can efficiently infer environment latents from as little as a single test-time interaction trajectory.

Robotic World ModelsTest-Time Optimization

Chenyuan Zhou, Yining Hong, Huang Huang · 137 views · Oct 15, 2026

GLaM: Training a Latent World Model over Global Spatiotemporal Memory for Active Exploration and Navigation

GLaM is a goal-conditioned latent world model over an online spatiotemporal map that jointly predicts future map representations and robot-centric waypoint latents, with GLaM Nav adding observation-based verification, retrieval, and geometric control. On a controlled 50% HM3D-ObjectNav subset, it raises success rate from 78.50% to 86.89% and SPL from 47.70% to 48.35%; qualitative Astribot S1 demonstrations cover exploration and memory-based return in an unknown office.

Embodied AIObject Navigation

I-Tak Ieong, Ruizhi Feng, Zhaoyang Lu · 172 views · Sep 13, 2026

UniWAM Technical Report: Unified Mobile Manipulation via Mixed-Stream World-Action Modeling and Manipulation Anchor Pose Supervision

UniWAM models navigation and manipulation as independently sampled control streams over shared video and action diffusion transformers. It introduces automatically generated Manipulation Anchor Pose supervision at the scale of 1.5 million episodes, improving manipulation-ready docking accuracy and mobile-manipulation success on MAP-Bench and 24 real-robot tasks.

Mobile ManipulationWorld-action models

Wei Xue, Keliang Liu, Mingzhang Cui · 174 views · Sep 30, 2026

As an Amazon Associate, we earn from qualifying purchases.