#Self-Supervised Learning (8)

Waterfall Pagination

0:35

DeFM: Learning Foundation Representations from Depth for Robotics

Depth sensors are widely deployed across robotic platforms, and advances in fast, high-fidelity depth simulation have enabled robotic policies trained on depth observations to achieve robust sim-to-real transfer for a wide range of tasks. Despite this, representation learning for depth modality remains underexplored compared to RGB, where large-scale foundation models now define the state of the art. To address this gap, we present DeFM, a self-supervised foundation model trained entirely on depth images for robotic applications. Using a DINO-style self-distillation objective on a curated dataset of 60M depth images, DeFM learns geometric and semantic representations that generalize to diverse environments, tasks, and sensors. To retain metric awareness across multiple scales, we introduce a novel input normalization strategy. We further distill DeFM into compact models suitable for resource-constrained robotic systems. When evaluated on depth-based classification, segmentation, navigation, locomotion, and manipulation benchmarks, DeFM achieves state-of-the-art performance and demonstrates strong generalization from simulation to real-world environments. We release all our pretrained models, which can be adopted off-the-shelf for depth-based robotic learning without task-specific fine-tuning. Webpage: https://de-fm.github.io/

Manthan Patel, Jonas Frey, Mayank Mittal · 156 views · Jan 26, 2026

DepthSelf-Supervised Learning

Jan 26, 2026

0:49

SSIO: Self-Supervised Inertial Odometry Without Labels

@QiuYuhengQiu · 122 views · 2026-10-28

SSIOInertial Odometry

Long Range Navigator (LRN): Extending robot planning horizons beyond metric maps

A robot navigating an outdoor environment with no prior knowledge of the space must rely on its local sensing to perceive its surroundings and plan. This can come in the form of a local metric map or local policy with some fixed horizon. Beyond that, there is a fog of unknown space marked with some fixed cost. A limited planning horizon can often result in myopic decisions leading the robot off course or worse, into very difficult terrain. Ideally, we would like the robot to have full knowledge that can be orders of magnitude larger than a local cost map. In practice, this is intractable due to sparse sensing information and often computationally expensive. In this work, we make a key observation that long-range navigation only necessitates identifying good frontier directions for planning instead of full map knowledge. To this end, we propose the Long Range Navigator (LRN), that learns an intermediate affordance representation mapping high-dimensional camera images to affordable frontiers for planning, and then optimizing for maximum alignment with the desired goal. LRN notably is trained entirely on unlabeled ego-centric videos making it easy to scale and adapt to new platforms. Through extensive off-road experiments on Spot and a Big Vehicle, we find that augmenting existing navigation stacks with LRN reduces human interventions at test-time and leads to faster decision making indicating the relevance of LRN.

Matt Schmittle, Rohan Baijal, Nathan Hatch · 195 views · Apr 17, 2025

Long-Range NavigationOff-Road Navigation

Apr 17, 2025

SiamJEPA: On the Role of Siamese Student Encoders in JEPA

Investigates the effect of Siamese student encoders in JEPA-based self-supervised representation learning. SiamJEPA uses masked Siamese student encoders with an EMA teacher, acting as an effective regularizer that improves representation separability and accelerates early-stage learning, outperforming single-encoder JEPA variants under limited budgets.

Makoto Yamada · 567 views · Jul 4, 2026

JEPASiamese

Jul 4, 2026

PhiZero: A World Model Built Around Physical Language

We introduce PhiZero, a physical world model built around physical language, a compact discrete representation of world-state transitions. Existing physical world models typically predict future videos directly in pixel space, leaving the underlying world dynamics implicit within high-dimensional visual predictors. Motivated by humans' ability to abstract predictive structure from visual experience and organize it in natural language for explicit reasoning, we learn physical language from in-the-wild videos through self-supervision and use it to explicitly reason about how the physical world evolves. Accordingly, PhiZero adopts a reason-then-render paradigm: it first infers future world evolution as a physical-language sequence and then renders the inferred transitions into videos. Extensive experiments across generation and understanding benchmarks validate the ability of PhiZero to model physically coherent world evolution. We further show its potential for realistic and interactive world modeling, fine-grained action-conditioned simulation, and zero-shot motion transfer.

Shuyao Shang, Yuqi Wang, Ruopeng Gao · 454 views · Jul 30, 2026

World ModelsPhysical Language

Jul 30, 2026

As an Amazon Associate, we earn from qualifying purchases.