Training Generalist Navigation Policies via Action Self-Distillation

Authors: Bowen Feng, Keqin Wang, Yulong Yang, Amogh Joshi, Christine Allen-Blanchette, Dhruv Shah, Felix Heide

Affiliation: Princeton University

Venue: CoRL 2026

Paper: arXiv:2610.17593 · Project page: asd-nav.github.io

Code status: At the time of this review, the project page's Code button still points to an empty anchor. The arXiv source archive contains LaTeX and figures, but not the data.py, prompt.py, and train.py files described in Appendix B.4. The method can therefore be audited from the paper, but a line-by-line paper-to-code comparison is not yet possible.

One-sentence summary: Action Self-Distillation treats a robot's recorded future trajectory as privileged feedback during training, allowing one vision-language model to infer a language goal and then learn to produce reasoning and future motion from onboard observations without human language labels.

1. The annotation gap in robot navigation

Robot navigation datasets already contain a valuable pair of signals: images show what the robot sees, and trajectories show what the robot did. The missing signal is language. A trajectory does not automatically say whether the robot was heading toward the right chair, following a sidewalk, avoiding a pedestrian, or preparing for a later right turn. Training a language-conditioned vision-language-action policy in the usual way therefore adds another annotation layer: goal descriptions, scene semantics, and explanations of why the demonstrated action was appropriate.

Visual navigation foundation models avoid some of this cost by conditioning on goal images or goal coordinates. That formulation scales well across heterogeneous robot data, but the resulting interface is not natural language. Language-conditioned navigation systems offer the desired interface, yet many rely on human-authored instructions and reasoning, while others use a pipeline of external models to synthesize goals, captions, or rationales before supervised fine-tuning.

This paper asks a different question: can trajectory-only data supervise its own language grounding? The authors observe that a capable VLM can reason more effectively when it receives corrective or task-relevant feedback about a candidate behavior. During training, the demonstrated future trajectory can play that role. If the trajectory is projected onto the current image, the model can see not only the scene but also the path that a human selected through it. The augmented image is the privileged observation $o_0^*=f(o_0,y)$, where $y$ is the ground-truth future trajectory and $f$ renders its projection.

Action Self-Distillation, or ASD, uses this difference in information to create an internal teacher. The teacher sees the trajectory overlay and generates an inferred goal plus structured reasoning. The student sees only deployable observations and uses the inferred goal as an additional condition. No human writes the goal and no external annotator produces the reasoning. The same VLM supplies both roles, with the teacher maintained as an exponential moving average of the student.

2. Problem formulation and representation

At time $t$, the robot receives a front-camera image $o_t$ and occupies a two-dimensional bird's-eye-view position $s_t=[x,y]\in\mathbb{R}^{2}$. The input history contains $T$ past states, $\tau_{-T:0}=\{s_{-T},\ldots,s_0\}$. The policy predicts future positions $\tau_{0:H}=\{s_0,\ldots,s_H\}$. Trajectory coordinates are serialized as structured text, so the language model predicts the path through the same autoregressive next-token interface used for words.

The desired policy emits both a path and an explicit reasoning trace. Given language goal $g$, the target distribution is:

$$ \hat{\tau}_{0:H},R\sim\pi(\cdot\mid o_{-T:0},\tau_{-T:0},g). $$

Equation (1) captures the final interface, but the training data contain neither $g$ nor $R$. The central technical problem is therefore to construct a useful conditional distribution from image-trajectory triples alone.

ASD uses eight past waypoints sampled at 5 Hz and predicts sixteen future waypoints at 2.5 Hz. The resulting prediction horizon is 6.4 seconds. A predicted path is passed to the robot controller, which converts it into linear and angular velocity commands. The language output is not a detached explanation added after control; both reasoning and trajectory tokens are generated by the same policy.

3. Two passes, one model

The student is parameterized by $\theta$, and the self-teacher by $\phi$. Both begin from the same pretrained VLM, Qwen2.5-VL-3B-Instruct. The teacher is not a frozen external model. It follows the student through an EMA update, so it retains most of the original pretrained knowledge while gradually incorporating what the student learns from navigation data.

In the teacher pass, the VLM receives the original image with the true future trajectory projected onto it, the state history, and a teacher prompt. It produces an inferred language goal, structured reasoning, and a trajectory:

$$ \hat{g},R,\hat{\tau}_{0:H} \sim\operatorname{sg} \left(\pi_{\phi}(\cdot\mid P,o_0^*,\tau_{-T:0})\right). $$

The stop-gradient operator $\operatorname{sg}$ prevents the reasoning objective from directly training the teacher. The teacher output is a target, not a second jointly optimized policy.

The inferred goal is extracted and inserted into the student prompt. The student receives the original image, history, and goal, but not the trajectory overlay:

$$ \tilde{R},\tilde{\tau}_{0:H} \sim\pi_{\theta} (\cdot\mid\tilde{P},o_0,\tau_{-T:0},\hat{g}). $$

The information asymmetry is deliberate. During training, the teacher can see what path the demonstrator chose. During deployment, the student cannot see the future and must rely on the human-provided goal. The teacher's goal is a bridge between trajectory evidence and deployable language conditioning.

flowchart TD
    D["Trajectory-only data
image past future"] --> T["Self-teacher
EMA VLM"]
    D --> S["Student VLM"]
    F["Future trajectory y"] --> O["Projection on image
privileged view"]
    O --> T
    T --> G["Extracted goal"]
    T --> R["Structured teacher reasoning"]
    G --> S
    S --> SR["Student reasoning"]
    S --> P["Predicted future path"]
    R --> K["Forward KL on reasoning tokens"]
    SR --> K
    F --> C["Cross entropy on path tokens"]
    P --> C
    K --> L["Combined loss"]
    C --> L
    L --> U["Student update"]
    U --> E["EMA teacher update"]
    E --> T
  

Figure 2 gives the corresponding paper diagram. The teacher is on the privileged branch, the student is on the deployable branch, and the two losses are separated by output span. Deployment removes the left branch entirely.

4. Structured reasoning as an interface

The teacher is constrained to emit a fixed sequence. The Task field states the navigation goal in one sentence. Perception describes visible terrain, boundaries, structures, and clear space. Agents lists moving entities that are actually present or says none. Prediction describes their expected motion. Planning chooses one action from a fixed vocabulary and ties the choice to named scene features. Trajectory then contains the executable path.

This format serves more than presentation. It limits unsupported hallucination by instructing the teacher to describe only what is visible in the raw image. It also creates a clean boundary for different loss functions. The reasoning span can use distribution matching, while the trajectory span uses supervised cross entropy against the true waypoints.

The Task span is excluded from the student's reasoning loss. In goal-conditioned training, the trainer extracts the teacher's Task sentence and injects it into the student context. The student never regenerates that sentence. This distinction matters because the experiment compares full ASD with goal-only supervision. In full ASD, the student learns both the inferred goal condition and the reasoning that connects visible scene structure to planning. In the goal-only ablation, it receives the goal but loses the reasoning supervision.

The reasoning is still a generated artifact, not a verified causal account of every action. The paper does not claim that each sentence is a faithful decomposition of the network's hidden computation. It claims something weaker and more testable: the text is grounded in visible features, and removing it substantially reduces real-world success.

5. Reasoning loss and trajectory loss

The reasoning span is trained with a forward Kullback-Leibler divergence:

$$ \mathcal{L}_{\mathrm{reason}} =D_{\mathrm{KL}}\left(R\Vert\tilde{R}\right). $$

In implementation, this is a token-level comparison between teacher and student logits before sampling. The expanded expression is:

$$ \mathcal{L}_{\mathrm{reason}} =D_{\mathrm{KL}}\left( \operatorname{sg}\left[ \pi_{\phi}(\cdot\mid P,o_0^*,\tau_{-T:0}) \right] \Vert \pi_{\theta}(\cdot\mid\tilde{P},o_0,\tau_{-T:0},\hat{g}) \right). $$

The student's future path is supervised directly:

$$ \mathcal{L}_{\mathrm{traj}} =\operatorname{CE} (\tilde{\tau}_{0:H},y). $$

The combined objective is:

$$ \mathcal{L} =\mathcal{L}_{\mathrm{traj}} +\mathcal{L}_{\mathrm{reason}}. $$

Appendix A clarifies why the KL term is more than a convenient pseudo-label loss. With a stop-gradient on the teacher, the divergence is equivalent, up to a constant, to teacher-weighted cross entropy:

$$ \mathcal{L}_{\mathrm{reason}} \equiv -\mathbb{E}_{R\sim\pi_{\phi}} \left[ \log\pi_{\theta} (R\mid\tilde{P},o_0,\tau_{-T:0},\hat{g}) \right] +\mathrm{const}. $$

Standard supervised fine-tuning usually has one human rationale per sample. The empirical target collapses to a hard label, and the student is pushed toward that exact wording. The teacher supplies a distribution over plausible next tokens. If two descriptions of the same scene are correct, both can receive probability mass. The authors argue that this soft target better preserves multiple valid reasoning modes and avoids overfitting to one offline annotation.

6. Why an EMA teacher is used

The teacher parameters are updated as:

$$ \phi\leftarrow\alpha\phi+(1-\alpha)\theta. $$

The training configuration sets $\alpha=0.999$ and updates the teacher every ten student steps. A frozen teacher would make offline label generation straightforward, but it would not adapt as the student changes. An exact copy of the current student would inherit noisy updates and could destabilize the target distribution. The EMA path keeps most of the pretrained model while allowing a slower task adaptation.

The appendix reports a stress test with a more aggressive 0.99 rate and an update every step. Across 500 scenes, the KL between the original Qwen checkpoint and the adapted checkpoint rises smoothly from 0.34 to 0.68 nats per token. The p95 surprisal rises from 4.8 to 6.7, and the maximum from 16.8 to 18.0. Reasoning entropy changes from 1.36 to 1.67 nats, far from the repetitive limit of zero and the random-text limit of roughly 12.4 nats. These measurements support the authors' claim that the teacher adapts without collapsing into repetition or uniform noise.

7. Training and inference algorithms

Each training iteration samples an image, history, and future trajectory from a trajectory-only dataset. The future trajectory is projected onto the current image, the teacher generates a goal and reasoning, and the extracted goal is injected into the student prompt. The student generates its own reasoning and path. The reasoning token positions receive KL supervision, the path token positions receive cross entropy, and the student is updated. The teacher then receives the EMA update.

At deployment, the teacher, the projected trajectory, and the ground-truth future all disappear. The non-goal-conditioned policy is:

$$ \tilde{R},\tilde{\tau}_{0:H} \sim\pi_{\theta}^{\star} (\cdot\mid\tilde{P},o_0,\tau_{-T:0}). $$

In the language-conditioned setting, an operator supplies a free-form goal $g^{\circledast}$:

$$ \tilde{R},\tilde{\tau}_{0:H} \sim\pi_{\theta}^{\star} (\cdot\mid\tilde{P},o_0,\tau_{-T:0},g^{\circledast}). $$

The robot executes the first eight of the sixteen predicted actions, receives a new observation and updated history, and plans again. ASD has no learned stopping criterion. In the real-world protocol, a method-blinded human evaluator ends the episode when the instruction is satisfied. Pauses in multi-stage runs may therefore reflect the latency of observation, inference, and replanning rather than a learned pause behavior.

8. Training data and experimental setup

The training mixture combines RECON and FrodoBots-2K. RECON contains outdoor navigation from a Clearpath Jackal in an unstructured campus environment, including sidewalks, grass, parking lots, and shared pedestrian spaces. FrodoBots-2K contains sidewalk data from an Earth Rover Zero across more than ten cities. Because the full collection was too large for the training window, the authors randomly select 25 percent of the scenes from the online splits and use a 90/10 train-validation split, amounting to roughly 100 hours of demonstration data.

The base model is Qwen2.5-VL-3B-Instruct. Training uses eight A6000 Pro GPUs, 256 GB of RAM, and an AMD Threadripper 7980X. Each GPU handles a batch of one sample with gradient accumulation of two. The optimizer is AdamW fused, the learning rate is $1\times10^{-5}$, the schedule is cosine, warmup is five steps, and training lasts two epochs. Images use dataset-native resolution and the maximum output is 512 tokens.

Configuration	Value	Role
Backbone	Qwen2.5-VL-3B-Instruct	Shared initialization for teacher and student
History	8 waypoints at 5 Hz	Past motion context
Prediction	16 waypoints at 2.5 Hz	6.4 second future horizon
Data	RECON + FrodoBots-2K	25 percent of scenes, about 100 hours
Optimization	2 epochs, learning rate 1e-5	AdamW fused with cosine schedule
Teacher update	EMA rate 0.999 every 10 steps	Slow adaptation from pretrained weights

Real-world evaluation uses one FrodoBot EarthRover Mini with a front-facing camera. There are nine indoor tasks, covering directions, objects, and multi-stage navigation, and four outdoor tasks covering a sidewalk, statue, tree, and car. Each task uses a fixed prompt and ten trials with varied initial pose. The evaluator allows at most three heuristic replans per episode. A collision or loss of the goal from the field of view counts as failure.

9. Indoor results

The indoor setting is a laboratory with chairs, trash bins, boxes, doors, a monitor, and a hallway. Both RECON and FrodoBots-2K are primarily outdoor datasets, so the indoor tests combine a visual domain shift with new objects and new language goals. Qwen2.5-VL-ASD reaches 84 percent average success, compared with 54 percent for OmniVLA and 27 percent for NaVILA. The Qwen3.5-VL-ASD variant reaches 94 percent.

Method	3 directional tasks	4 object tasks	2 multi-stage tasks	All 9 tasks
OmniVLA	0.633	0.575	0.350	0.54
NaVILA	0.200	0.250	0.400	0.27
Qwen2.5-VL-ASD	0.900	0.875	0.700	0.84
Qwen3.5-VL-ASD	0.967	0.975	0.850	0.94

The hard tasks reveal why average success is not enough. In the right-chair task, three identical chairs require relational grounding. In the monitor task, the robot must distinguish a monitor from a trash bin and chair. In the hallway task, it must pass through a door, turn right, and continue along the corridor. Qwen2.5-VL-ASD obtains 80, 70, and 60 percent on these tasks, while OmniVLA obtains 20, 20, and 0 percent.

The paper also compares foundation-model behavior on the left-chair task. OmniVLA reaches 90 percent in 25.3 seconds and NaVILA reaches 20 percent in 28.9 seconds. Vanilla Qwen2.5-VL produces physically infeasible actions and reaches 0 percent. Claude Opus 4.7 identifies the target and reaches 70 percent, but takes 846.1 seconds on average. ASD reaches 100 percent in 15.3 seconds. The comparison supports the paper's premise that semantic reasoning must be grounded into a low-latency action distribution rather than simply invoked from a larger model at test time.

10. Outdoor results and steerability

The outdoor tasks ask the robot to follow a sidewalk or navigate to a statue, tree, or car in a campus environment. Qwen2.5-VL-ASD reaches 95 percent average success and the Qwen3.5 variant reaches 98 percent. The statue is an out-of-distribution category absent from training. ASD reaches 80 percent, OmniVLA reaches 60 percent, and NaVILA reaches 10 percent.

Method	Sidewalk	Statue	Tree	Car	Average
OmniVLA	1.00	0.60	1.00	1.00	0.90
NaVILA	0.90	0.10	0.00	0.30	0.33
Qwen2.5-VL-ASD	1.00	0.80	1.00	1.00	0.95
Qwen3.5-VL-ASD	1.00	0.90	1.00	1.00	0.98

The steerability experiment is important because each training observation is normally paired with one demonstrated trajectory. ASD is not trained with counterfactual pairs such as "same image, go to the left chair" and "same image, go to the right chair." Figure 3 shows that it can nevertheless follow color, direction, and object prompts in one physical setting. This makes a simple nearest-demonstration explanation less plausible: the model must condition its generated path on the requested target.

11. Multi-stage navigation

The Doorway task requires the robot to approach and pass through a door. The Hallway task adds a right turn and continued motion along the corridor. Qwen2.5-VL-ASD reaches 80 and 60 percent, respectively. OmniVLA reaches 70 and 0 percent, while NaVILA reaches 30 and 50 percent. The baselines therefore exhibit complementary failure modes rather than a single general disadvantage.

The result suggests that the teacher's inferred goal may preserve intent over a longer trajectory. If the demonstration passes a doorway and then turns, a goal sentence can describe the higher-level destination, while the student's reasoning can still mention the doorway and the right-hand opening. This is a plausible mechanism, but the paper does not isolate which reasoning tokens causally drive which path segment. The claim should therefore be limited to observable behavior and ablation evidence.

The multi-stage evaluation uses ten trials per method. A 60 percent versus 50 percent difference in Hallway is one trial, and the paper does not report confidence intervals. The appropriate conclusion is that ASD demonstrates a viable multi-stage behavior and a large improvement over one baseline, not that it has established a statistically stable lead for complex building navigation.

12. Ablations and what they rule out

Table 4 separates several explanations for the performance gain. Full Qwen2.5-VL-ASD reaches 80 percent average success on five representative tasks. Goal-only supervision falls to 26 percent. Training with reasoning but disabling reasoning at inference falls to 14 percent. Offline labels from Claude Opus 5 followed by SFT reach 36 percent, offline labels from Qwen2.5-VL reach 22 percent, and NaVILA fine-tuned on the same data reaches 16 percent.

Method	Right chair	Chair	Monitor	Hallway	Car	Average
ASD, goal-only supervision	0.40	0.20	0.20	0.10	0.40	0.26
ASD, reasoning off at inference	0.10	0.20	0.20	0.00	0.20	0.14
Claude Opus 5 labels + SFT	0.30	0.50	0.30	0.30	0.40	0.36
Qwen2.5-VL labels + SFT	0.40	0.30	0.30	0.00	0.10	0.22
NaVILA with same-data SFT	0.00	0.10	0.20	0.30	0.20	0.16
Qwen2.5-VL-ASD	0.80	0.90	0.70	0.60	1.00	0.80
Qwen3.5-VL-ASD	0.90	1.00	0.90	0.80	1.00	0.92

The first two rows rule out two narrow explanations. If the only useful signal were the inferred goal, goal-only supervision should remain competitive; it does not. If reasoning were merely scaffolding that could be removed after training, inference-time disabling should be harmless; it collapses performance. The comparison with Claude Opus 5 labels also rules out the simplest "use a stronger teacher offline" explanation. A stronger annotator still produces one static sample per input, and the student receives a hard target rather than a privileged, adaptive distribution.

The ablations do not fully isolate semantic correctness from generic regularization. KL matching may transfer formatting, scene priors, action tendencies, or output structure in addition to grounded reasoning. The paper narrows this ambiguity but does not measure the causal contribution of individual reasoning fields. A useful follow-up would intervene on Perception, Agents, and Planning separately rather than treating the entire reasoning span as one unit.

13. Open-loop trajectory prediction

ASD is also evaluated on the validation splits of RECON and FrodoBots-2K with a two-second open-loop protocol. On RECON, its average displacement error is 0.174, compared with 0.180 for Diffusion Policy and 0.192 for Qwen2.5-VL+SFT. On FrodoBots-2K, the average displacement error is 0.031. The paper reports the best result on seven of eight error comparisons, not all eight. For FrodoBots-2K MSE, ASD records 0.004 while the Gaussian and Gaussian-mixture baselines record 0.002.

The open-loop comparison is useful because Qwen2.5-VL+SFT shares the backbone, training data, and much of the optimization recipe with ASD. The gap between them is therefore more consistent with the reasoning supervision than with model capacity alone. The result does not replace closed-loop testing. Open-loop error does not capture accumulated drift, collision recovery, or the cost of replanning, so the real-world tasks remain the primary evidence.

14. Why the result matters

The most important contribution is the supervision mechanism. In a conventional pipeline, trajectories train actions, but language labels train language behavior. ASD makes the trajectory itself the source of the language condition. The teacher asks what goal would explain the demonstrated future, then grounds a rationale in the current image. The EMA teacher keeps this inference inside the training loop instead of freezing a single pseudo-label set.

The result also clarifies the role of pretrained VLM capability. Vanilla Qwen2.5-VL fails on the action task despite containing useful semantic knowledge. Claude Opus 4.7 identifies the target but is far too slow for control. A larger or more capable model does not automatically solve grounding or latency. ASD uses about 100 hours of trajectory data to turn a smaller open model into a deployable policy, which is a more practical claim than "scale the VLM and hope action ability emerges."

The data-efficiency comparison with OmniVLA is suggestive but not definitive. The paper notes that OmniVLA uses roughly 9,500 hours across ten platforms, while ASD trains on a much smaller mixed dataset. Differences in pretraining, annotation, architecture, and evaluation prevent a controlled efficiency ratio. The defensible conclusion is that trajectory-conditioned self-distillation offers a useful learning signal for language grounding, not that it dominates every multimodal pretraining recipe.

15. Limitations and residual risk

The first author-stated limitation is dependence on demonstration trajectories as privileged feedback. ASD removes human language and reasoning labels, but it still requires high-quality trajectories. If the intended goal can only be expressed through interaction, correction, or task-level reward, a trajectory may be insufficient to infer what the operator actually wanted.

The second author-stated limitation is teacher quality. Goals and reasoning inherit the biases and failures of the pretrained VLM. Ambiguous scenes, small targets, and fine-grained spatial instructions can cause the self-teacher to generate a wrong or incomplete rationale. Because the student is trained on that output, systematic teacher errors become systematic supervision errors rather than being filtered by independent human review.

An independent limitation is the breadth of the real-world evidence. All deployments use one FrodoBot EarthRover Mini and one main laboratory plus one campus environment. Each task is run ten times, and no confidence intervals are reported. The 60 versus 50 percent Hallway result is a single-trial difference. Long-term operation, different cameras, other robot dynamics, and high-density crowds remain untested.

A second independent limitation concerns execution and autonomy. ASD executes the first eight of sixteen predicted steps, and the paper reports deviations on uneven terrain even when the plan is reasonable. It also has no learned stopping criterion, so a human evaluator decides when the instruction has been satisfied. Self-generated goals may contain dataset or model biases that look plausible in text but are not externally auditable. Production use would need independent safety checks, fallback behavior, and a reliable stop policy.

16. Conclusion

ASD shows that trajectory-only robot data can supervise more than low-level imitation. By giving the teacher a projected future trajectory, the method elicits a language goal and scene-grounded reasoning inside the training loop. The student then learns a deployable mapping from images and language to reasoning and motion. The EMA teacher avoids the static-label limitation of a frozen annotator, and the KL term transfers a distribution rather than a single rationale.

The strongest evidence combines 84 percent indoor success, 95 percent outdoor success, 80 percent on the out-of-distribution statue task, and a large ablation gap against goal-only and offline-label SFT. The work is not yet a complete autonomous navigation system: the test fleet, trial count, stopping behavior, and terrain robustness remain limited. Its main contribution is nevertheless clear: demonstrated motion can be used not only as an action target, but also as the evidence from which a robot policy learns why an action should be taken.

Golden line: ASD does not add a human answer to every trajectory; it makes the path already taken explain why it was worth following.