Papers

GeoFlow-SLAM: A Robust Tightly-Coupled RGBD-Inertial and Legged Odometry Fusion SLAM for Dynamic Legged Robotics

GeoFlow-SLAM targets the feature-matching, initialization, and texture-robustness failures that arise when legged robots move aggressively at high frequency. It tightly couples dual-stream optical flow, legged odometry, PnP, GICP, and depth-to-map geometric constraints in one optimization framework. Experiments on Unitree Go2 and G1 robots plus OpenLORIS, ScanNet, and TUM datasets show state-of-the-art robustness and accuracy.

SLAMLegged RoboticsRGBD-InertialXiao, Tingyang, Zhou, Xiaolin, Liu, Liu·Mar 18, 2025

Show-Harness: Just a VLM Agent Can Play Robots

Foundation vision-language models (VLMs) exhibit broad intelligence about the world, yet translating this intelligence into robot control remains challenging. We present Show-Harness, an Embodied Harness that enables VLMs to "play" robots through a compact semantic interface linking intent to action. Show-Harness exposes discrete semantic action units that VLMs can naturally reason over, while embodiment-specific interpreters deterministically ground them into local robot actions, keeping the VLM directly responsible for fine-grained physical decisions. Through the same interface, Show-Harness demonstrates the feasibility of (1) directly unlocking closed-source frontier VLMs for zero-shot robot control, and (2) adapting small-scale open-source VLMs for low-cost deployment with just a few GPU-hours of fine-tuning. We further develop GUMI (GUI Manipulation Interface), which extends the same semantic action space to GUI-based demonstration collection, allowing humans and agents to "play" robots across embodiments without specialized teleoperation hardware. Extensive experiments show that Show-Harness-equipped VLM agents generalize robustly across tasks, embodiments, and environments, outperforming representative agentic and VLA paradigms. These results suggest that the right interface can unlock substantial embodied capability from foundation VLMs, without requiring additional model capacity or costly embodiment-specific pretraining.

Vision-language modelsAgentic HarnessSemantic Action InterfaceYanzhe Chen, Zechen Bai, Zhijun Cao·Sep 9, 2026

Memory as Plans: World-Action Modeling with Memory-Grounded Planning

Most robot policies predict actions from the current observation or a short fixed window, yet long-horizon manipulation is non-Markovian: the evidence a decision needs may already be out of view. MaP-WAM treats memory as planning-time evidence instead of executor input. Completed segments are stored as structured multimodal records pairing a language instruction with sparse visual context (8 uniformly sampled frames per segment); a fine-tuned Qwen3.5-4B language planner proposes the next segment-level language plan, and a causal world model initialized from WAN-2.2-5B turns the long-term visual context into a matching visual plan. The two form a memory-grounded plan that conditions a World-Action-Progress executor, a Mixture-of-Transformers extension of a video DiT that jointly predicts action chunks, future visual latents, and execution progress, with progress as a first-class modality. Because the executor only sees a fixed-length plan prefix, its context stays constant as history grows, and block-causal attention makes both planning and execution KV-cacheable; plan-observation alignment calibrates recursively predicted progress against visual-plan frames, and a progress gate (threshold 0.95) triggers segment transitions that write resampled real observations back into memory. On RMBench MaP-WAM reaches 83.3% average success over nine memory-dependent tasks (best baseline LingBot-VA 77.1%), and 78.0% on two real Franka Research 3 tasks (88% Find Button, 68% Press Buttons), while a full-context executor runs out of memory at 1700 history frames and WAP holds an approximately constant 827 ms per action chunk. Limitations: segment structure is taken from benchmark annotations rather than discovered automatically, and plan-observation alignment uses a lightweight training-free pixel-difference metric.

World-action modelsLong-Horizon MemoryMemory-Grounded PlanningSizhe Zhao, Haozhe Xie, Weiyu Zhao·Sep 10, 2026

World-Model-Augmented Visual Locomotion for Humanoids on Foothold-Constrained Terrain

Foothold-constrained terrain is characterized by sparse, discontinuous, or geometrically restricted feasible foot contacts, as encountered on stepping stones, across gaps, and on narrow stair treads. On such terrain, a single misstep often leaves little room to recover, so policies that base foot-placement decisions primarily on the immediately visible terrain are prone to failure. We ask whether a learned predictive summary of near-future observations and rewards can provide the anticipatory information required in such settings. We present World-Model-Augmented Visual Locomotion (WM-LOCO), which jointly trains a recurrent world model and a PPO policy. Conditioned on proprioception and a single onboard depth image, the world model produces a predictive recurrent feature that guides the policy, without explicit foothold labels. In simulation, WM-LOCO succeeds on gaps and stepping stones where a matched baseline fails completely, and matches the baseline's success rate on stairs while improving stride efficiency and reducing pelvis acceleration. We deploy the same policy onboard a physical Unitree G1 humanoid using onboard proprioception and a single depth stream; it traverses all three terrain classes with an average success rate of 93.3%.

HumanoidLegged LocomotionWorld ModelsLiu, Yuxi, Han, Lijun, Wang, Ziming·Sep 2, 2026

IMLE-VLA: Fast Single-Step Action Generation for Vision-Language-Action Policies

Vision-language-action (VLA) policies leverage pretrained vision-language backbones to achieve strong cross-task generalization. A leading design couples this backbone with a dedicated continuous action head trained via diffusion or flow matching. However, such heads rely on iterative multi-step sampling, for example 10 Euler steps in π_0.5. This creates an inference bottleneck that produces stop-and-go movement in the robot and slower task completion. We introduce IMLE-VLA, which replaces the iterative action head with a single-step conditional generator trained via conditional Implicit Maximum Likelihood Estimation (cIMLE). The cIMLE objective promotes multimodal action coverage, avoiding the mode collapse of naive regression heads while eliminating multi-step sampling entirely. When IMLE-VLA is applied to π_0.5, it increases inference frequency 3.67x (55 Hz vs. 15 Hz), enabling up to 11x higher action throughput. On the 40-task LIBERO benchmark, IMLE-VLA achieves the highest average success rate (98.0%) among all baselines while leading in inference frequency. Under the test-time perturbations of LIBERO-plus, IMLE-VLA retains π_0.5's robustness while other baselines degrade sharply, confirming that the cIMLE head preserves generalization. Real-world experiments on a Franka Emika Panda across four tasks demonstrate smoother motion (2.2x to 3.0x lower jerk) and faster task completion, with IMLE-VLA outperforming π_0.5 on every task and reducing average VLA inference time per episode by 3.9x to 6.6x. Videos and code are available at https://kianhk6.github.io/IMLE-VLA/

VLAEfficient InferenceSingle-Step Action GenerationHosseinkhani, Kian, Peng, Qinhe, Shramko, George·Sep 10, 2026

WARP-RM: Warp-Augmented Relative Progress Reward for Data Curation

WARP learns dense signed relative progress from successful demos via time-warp augmentations, then WARP-BC filters and reweights action chunks for behavior cloning. On bimanual T-shirt folding, throughput rises up to ~18× vs vanilla BC as suboptimal demos increase; in 512 paired sim bottle scenes it reaches 290 bottles/hr.

Imitation LearningData CurationBehavior CloningJustin Yu, Andrew Goldberg, Kavish Kondap·Jun 26, 2026

GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation

GE-Act 2.0 combines a control-oriented autoencoder, a single-step visual planner, an inverse dynamics model, and knowledge-aligned selective optimization to pretrain a world-action policy from scratch on manipulation data. Scaling co-training data from 300 to 30,000 hours raises zero-shot OOD success to 44.1% on G1-OP and 31.1% on G2-90D without task-specific fine-tuning.

World-action modelsRobot ManipulationEmbodied AI AgiBot Research Team, Renhang Liu, Wenzhi Zhao·Sep 4, 2026

LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World

Training vision-language-action (VLA) policies for humanoid loco-manipulation is constrained by the high cost and complexity of collecting human teleoperation demonstrations. VLA policies fine-tuned in simulators have, until now, failed to transfer effectively in humanoid loco-manipulation tasks. We present LEGS (Loco-manipulation via Embodied Gaussian Splatting), a hybrid simulator that composites a mesh foreground (robot, objects, props) over a photorealistic 3D Gaussian Splatting (3DGS) background reconstructed from a handheld scene capture. LEGS uses a procedural motion-primitive generator to synthesize labeled demonstrations at scale without human teleoperation, and a deterministic two-stage color calibration to align the rendered 3DGS image to the robot's deployment camera. On a Unitree G1 humanoid robot, across three pick-and-place tasks of increasing whole-body difficulty and three VLA backbones (psi_0, pi_0.5, GR00T N1.6), a policy trained purely on LEGS data matches or exceeds one trained on human teleoperation demos on every experiment. It also outperforms a mesh-only simulation baseline that ablates the effect of the 3DGS background, showing that photorealistic rendering is a key enabler for synthetic data transfer. Humanoid motion is recorded independently of scene appearance in LEGS, allowing the same auto-generated demonstrations to be re-rendered under new backgrounds and object meshes–covering a new scene at more than 15x lower cost than teleoperation–to augment training data for robustness to scene variations. Under combined object-and-scene appearance shift, the policy trained on re-rendered LEGS-AUG data maintains task success while the baseline trained on teleoperation data fails entirely. Our project page is located at https://legsvla.github.io/.

HumanoidVLA3D Gaussian SplattingKim, Hojune, Chen, Timothy, Sun, Jiankai·May 31, 2026

FADA: Few-Shot Domain Adaptation via Dynamics Alignment for Humanoid Control

High-precision humanoid control is limited by target-domain dynamics mismatch, where the same control objective can induce different realized motions under changes in terrain, payload, or actuator response. Existing methods either pursue zero-shot transfer through domain randomization or in-context adaptation without target-domain specialization, or require heavy adaptation pipelines that leverage target-domain data, such as model calibration, residual learning, or policy retraining. In this paper, we present FADA (Few-Shot Domain Adaptation via Dynamics Alignment), a three-stage Planner-Inverse Dynamics Model (Planner-IDM) framework for few-shot adaptation in humanoid control. FADA first trains an oracle policy with privileged information and then distills the oracle behavior into a deployable Planner-IDM student through DAgger. At deployment, FADA freezes the planner and finetunes only the IDM using approximately 2 minutes of target-domain rollouts with standard supervised learning. Rather than requiring optimal demonstrations or rewards, FADA uses the paired actions and observations that are observed during these rollouts as supervision, aligning the IDM's action generation with target-domain dynamics. Experiments show that FADA outperforms both in-context and end-to-end adaptation baselines, improving task performance under dynamics shifts and enabling real humanoid robots to execute diverse high-precision whole-body tasks. Implementation details and qualitative hardware rollout videos are available at https://lecar-lab.github.io/FADA-humanoid/.

HumanoidFew-Shot LearningDomain AdaptationAngchen Xie, Nikhil Sobanbabu, Ishayu Shikhare·Jun 26, 2026

A4A: Cross-Embodiment Transfer of Action-Oriented 4D Affordances from Human Demonstrations

Human demonstrations contain rich manipulation knowledge, but it remains unclear what information can be transferred effectively to robot control. Existing affordance representations are typically formulated as 2D masks, 3D regions, contact points, or actionability scores, and therefore primarily identify where interaction may occur. However, effective manipulation also requires modeling how interaction-relevant geometry evolves during task execution. To bridge this gap, we introduce action-oriented 4D affordances, which represent the language-conditioned future trajectories of interaction-relevant 3D points. These trajectories capture task-conditioned geometric evolution rather than embodiment-specific actions, enabling transferable interaction priors across humans and robots. Based on this representation, we construct a large-scale action-oriented 4D affordance dataset from existing human–object interaction video data and complementary RGB-D demonstrations, and introduce A4A, an affordance-to-action framework that uses 4D affordance trajectory prediction to pretrain robot policies before manipulation fine-tuning. Experiments in both simulation and the real world validate the effectiveness of A4A, showing that pretraining with action-oriented 4D affordance data consistently improves the manipulation performance of diverse VLA policies. These results establish action-oriented 4D affordances as an effective cross-embodiment representation for transferring manipulation knowledge from human demonstrations to robot control.

VLAAffordance4D AffordanceYifan Han, Litao Liu, Yuqi Gu·Sep 5, 2026

DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation

Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification. While recent parallel drafters efficiently propose long token sequences in a single forward pass, they suffer from rapid acceptance decay due to a lack of inter-token dependencies. Furthermore, indiscriminately verifying these extended blocks wastes critical batch capacity on tokens with high rejection risks, severely degrading throughput in high-concurrency serving systems. We introduce DSpark, a speculative decoding framework that unifies high-throughput parallel generation with adaptive, load-aware verification. To maintain draft quality, DSpark utilizes a semi-autoregressive architecture, coupling a parallel backbone with a lightweight sequential module, to introduce intra-block dependency modeling and mitigate suffix decay. To optimize system efficiency, DSpark employs confidence-scheduled verification, dynamically tailoring the verification length for each request based on estimated prefix survival probabilities and engine-specific throughput profiles. On offline benchmarks across diverse domains, DSpark substantially improves the accepted length over state-of-the-art autoregressive and parallel drafters. When deployed within the DeepSeek-V4 serving system under live user traffic, DSpark successfully mitigates verification waste. Compared to the established production baseline (MTP-1), DSpark accelerates per-user generation speeds by 60 to 85 percent at matched throughput levels. More importantly, by preventing severe throughput degradation under strict interactivity constraints, it enables performance tiers that were previously unattainable, shifting the Pareto frontier of our serving system.

Speculative DecodingLLM InferenceLLM ServingXin Cheng, Xingkai Yu, Chenze Shao·Jul 6, 2026

Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents

Language-conditioned manipulation requires both precise contact-rich control and robust reasoning over language, scenes, and long horizons. End-to-end Vision-Language-Action (VLA) models provide strong local visuomotor skills, but they are trained on in-distribution task trajectories and often fail under deployment perturbations such as semantic retargeting, goal re-binding, spatial-layout shifts, and unstable local contacts. LLM coding agents provide complementary semantic and compositional reasoning, but purely analytic primitives struggle with irregular grasping, constrained placement, and articulated-object interaction. We present Harness VLA, a memory-augmented agentic framework that exposes a frozen VLA as a retryable contact-rich primitive and composes it with a small fixed library of analytic primitives for grounding, staging, transport, navigation, and release. Rather than expanding the skill library, the harness learns the operating range of these fixed primitives from task-specific execution traces, global success rules, and failure models. By lifting semantic re-grounding, non-contact execution, and VLA re-staging to the planner while reserving the frozen VLA for local contact-rich phases, Harness VLA extends pretrained VLAs beyond their original trajectory distribution without finetuning. Across perturbed tabletop, household kitchen, and clean-to-randomized bimanual manipulation, Harness VLA improves over the strongest relevant baselines by 38.6 and 27.1 percentage points on LIBERO-Pro and RoboCasa365, respectively, and reaches 58.4% on RoboTwin C2R. Code is available at https://github.com/RLinf/RPent.

VLARobot ManipulationLLM AgentYixian Zhang, Huanming Zhang, Feng Gao·Jul 9, 2026