EgoXperience: Learning World-Grounded Egocentric Representations from Human Experience
EgoXperience turns egocentric human experience into world-grounded representations for embodied intelligence. Its EgoX-Engine captures first-person video with the HOMIE suite and emits eight synchronized modalities: RGB, depth, camera poses, MANO hands, object masks, object 6DoF trajectories, metric object meshes and task captions, using first-mask anchoring with SAM3 propagation and evidence-verified FoundationPose 6DoF. EgoX-Data-6M spans 50 hours and 5.4M frames over 270 tasks and 101 objects across workspace, cross-room and outdoor settings, plus a quality-controlled one-hour EgoX-Bench. The 1.2B-parameter EgoX extends VGGT feed-forward reconstruction from static scenes to egocentric video, jointly reconstructing hands, object 6DoF, depth and camera poses in a shared metric world frame from monocular input and a first-frame object mask in a single forward pass.
