#Egocentric Vision (13)

OmniEgoCap: Device-Agnostic Sequence-Level Egocentric Motion Reconstruction

Commercial egocentric devices capture human behavior in everyday settings, yet full-body motion reconstruction must generalize across diverse cameras and mountings. This is challenging because head trajectories under-determine body motion, while hand observations are intermittent and camera-dependent: a hand may disappear simply because it has left the camera's field of view. Existing methods treat such absences as missing data, relying on fixed visibility regimes or post-hoc optimization, and fail to generalize across devices. We present OmniEgoCap, a sequence-level diffusion framework for device-agnostic egocentric full-body motion reconstruction. Our key insight is that egocentric videos contain sequence-level evidence about both the wearer and the camera-induced visibility structure, enabling OmniEgoCap to infer consistent body shape and coherent motion from sparse head and hand cues. To prevent overfitting to a single camera setup, we further introduce geometry-aware visibility augmentation, which synthesizes visibility patterns from realistic variations in camera geometry. We also present OmniEgoDB, the first mocap benchmark that pairs ground-truth motion with recordings from multiple consumer egocentric devices. Experiments on synthetic, real-device, and in-the-wild settings demonstrate superior motion reconstruction, robust cross-device generalization, and coherent in-the-wild results.

Kyungwon Cho, Jeonghyeon Na, Hanbyul Joo · 51 views · Dec 22, 2025

Egocentric VisionMotion Capture

Dec 22, 2025

0:10

OmniEgoCap: Full-Body Motion From Head and Hands

@tokufxug · 44 views · 2026-10-30

Egocentric VisionMotion Capture

0:12

EgoOne Lite: A Wearable Egocentric Data Collection Rig

@shenzhenfoundry · 69 views · 2026-11-02

Egocentric VisionData Collection

EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data

EgoScale is a human-to-dexterous-manipulation transfer framework. A VLA policy is pretrained on 20,854 hours of action-labeled egocentric human video, revealing a log-linear relationship between data scale and validation loss. Lightweight aligned human-robot mid-training then grounds the learned motor prior in robot control, improving average success by 54% over a no-pretraining baseline on a 22-DoF dexterous hand and transferring to lower-DoF hands.

Ruijie Zheng, Dantong Niu, Yuqi Xie · 152 views · Feb 18, 2026

Dexterous ManipulationVLA

Feb 18, 2026

2:40

EgoXperience: Learning World-Grounded Egocentric Representations from Human Experience

@liuziwei7 · 151 views · 2026-10-20

Egocentric Vision3D reconstruction

Social-WM: Safety-Aware Latent World Models for Robot Social Navigation

Safe social navigation requires a robot to anticipate not only the future consequences of its actions, but also whether a nominal action can actually be executed under surrounding physical and social constraints. We present Social-WM, an efficient latent world-model planning framework trained from egocentric RGB video sequences. Our key observation is that social-navigation experience contains a systematic discrepancy between the nominal action and the realizable action: a nominal forward action may be fully executed in free space, but needs to be constrained when heading towards a pedestrian or obstacle. Social-WM learns these safety-relevant consequences directly through action-conditioned future prediction, where the target is the actual observed future following each command. We further introduce a realizable inverse-dynamics objective that associates observed latent transitions with the action actually realized rather than the nominal one. At deployment, candidate actions are imagined through the latent world model, and the inverse dynamics model estimates their realizability; nominal–realizable discrepancy then provides a safety signal before execution. The learned dynamics and realizability model remain goal-independent and support both position- and image-goal navigation. On Social-HM3D, Social-WM achieves 63.77% success while reducing human collisions to 21.67%, and maintains strong performance under zero-shot transfer to Social-MP3D, without explicit pedestrian tracking, privileged human state, or online reinforcement learning.

Zhihao Zheng, Mooi Choo Chuah · 250 views · Sep 30, 2026

Social NavigationWorld Models

Sep 30, 2026

Exposing the Long-tail in Embodied Urban Navigation via Scalable Learning from In-the-Wild Videos

Learning embodied urban navigation policies from real-world data is constrained by the cost of task-specific data collection and the limited coverage of rare yet safety-critical scenarios. To address these challenges, we present a scalable framework for learning point-goal urban navigation from web-scale in-the-wild egocentric videos while systematically exposing its long tail. The framework automatically annotates uncurated web videos with metric trajectories and structured navigation semantics, which are then used to train a vision-language-action policy for interpretable navigation planning. We characterize the long tail based on model performance and the distribution of perception-motion patterns, and employ reflection-based analysis to diagnose recurring failure modes. Experiments on web-video data and real-world urban navigation tasks demonstrate effective knowledge transfer from unconstrained videos and reveal coherent long-tail structures beyond aggregate navigation performance.

Bingyi Xia, Han Bao, Zhewei Chen · 306 views · Aug 17, 2026

Urban NavigationVLA

Aug 17, 2026

0:15

MobileVISTA: Pose Generalization from Single-Pose Demos

@s_wistreich · 221 views · 2026-10-07

Mobile ManipulationData Augmentation

EgoLAP: Learning from Egocentric Human Data through Language-Action Reasoning

Egocentric human data offer a path to scaling robot learning beyond costly robot demonstrations, yet the embodiment gap makes raw human trajectories a poor supervisory target for control. Our key insight is that, although low-level actions are embodiment-specific, their underlying motion intent can capture task-relevant structure that transfers across humans and robots. We introduce EgoLAP, a VLA pre-training framework that jointly learns from human and robot trajectories through a shared language-based action chain-of-thought. EgoLAP expresses motion intent as structured, temporally abstracted language actions and pairs them with motion-level reasoning grounded in scene geometry, physics, and object affordances. Across extensive real-world and simulated experiments, EgoLAP transfers human experience to robot control more effectively than alternative action representations and reaches 80.1% mean real-world task progress, a 2.3x performance gain over alternative action representations. Motion-level reasoning also outperforms a composite reasoning format that combines subtask, object-box, and visual-trace reasoning.

Lihan Zha, Shresth Grover, Tenny Yin · 581 views · Oct 6, 2026

VLAManipulation

Oct 6, 2026

4:14

Gen-HumanEgo: Open-Source Egocentric Human Data for Robot Learning

@GenrobotAI · 1.1k views · 2026-09-23

Egocentric VisionRobot Learning

SSR: Scaling Surefooted and Symmetric Humanoid Traversal to the Open World

Extending humanoid traversal to the open world is key to practical deployment in human environments, but remains challenging. The robot must use vision to ensure safe and reliable foot placement on heterogeneous terrain under highly dynamic motion, while producing coordinated, natural whole-body behaviors. We propose SSR, an efficient end-to-end framework for egocentric vision-based humanoid traversal that jointly learns these capabilities. SSR introduces imagined foothold guidance, which learns to model forthcoming swing-foot contacts and evaluates their support to guide pre-touchdown swings toward stable regions, reducing edge slips. It further employs equivariant latent-space symmetry augmentation to efficiently induce bilateral coordination under high-dimensional visual observations, and uses terrain-specific multi-discriminator motion priors to encourage human-like behavior across scenes. Extensive experiments show that SSR achieves safe, stable, and high-quality locomotion on diverse real-world terrains, including stairs with varied structures and extreme challenges such as wide gaps and high platforms, while enabling reliable long-horizon traversal in open outdoor environments.

Ruiqi Yu, Yiwen Wang, Yuan Hao · 898 views · May 29, 2026

HumanoidLegged Locomotion

May 29, 2026

0:17

Boxer Lifts 2D Detections into Fused 3D Boxes

@ddetone · 621 views · 2026-09-12

3D Object DetectionOpen-Vocabulary Perception

1:22

MINT open-source egocentric hand motion reconstruction

@wuji_global · 608 views · 2026-09-10

Motion CaptureEgocentric Vision

As an Amazon Associate, we earn from qualifying purchases.