StageVLN: Spatial and Trajectory Auxiliary Guidance for Efficient Vision-Language Navigation

Paper: Anh Dao, Quan-Dung Pham, Le Danh Vinh, and colleagues. arXiv:2610.05664v1, submitted October 5, 2026, with computer vision as the primary category and robotics as a cross-list.

Code status: The paper does not provide a repository link. A GitHub search by method name and exact title, performed on October 26, 2026, did not identify an official StageVLN release. This analysis therefore relies only on the paper text, equations, tables, figures, and implementation details; it makes no claims about unavailable source code.

One-Sentence Summary

StageVLN trains a 4B vision-language navigation policy with a frozen VGGT-Ω geometry teacher, relative-heading targets, and expert-route progress targets, then removes every auxiliary branch at deployment so inference still uses only RGB observations and language.

1. The Problem: Semantic Competence Is Not Navigation State

Continuous vision-and-language navigation requires an agent to interpret an instruction while relating each egocentric frame to traversable space, body orientation, and progress through a multi-step route. Action-only imitation can produce a working policy, but it does not explicitly require the hidden representations to preserve those spatial and temporal relationships.

Large vision-language models bring strong object recognition and instruction-following priors. Recognizing a kitchen, a bed, or a doorway is nevertheless different from knowing how far the agent has rotated since an earlier observation, or how much of the expert route remains. Semantic pretraining alone does not guarantee that such state remains linearly accessible before an action token is generated.

The authors divide existing remedies into explicit 3D inputs and runtime geometry fusion. Explicit depth, maps, point clouds, or reconstructed scene structure provide direct spatial evidence. Runtime fusion connects a geometry foundation model to the navigation policy. Both can improve spatial reasoning, but both add sensors, memory, latency, and architectural dependencies at deployment.

StageVLN asks a different question: can spatial and trajectory knowledge be used as privileged supervision during training and then absorbed into the original RGB-language policy? The work combines ideas from intermediate representation distillation and auxiliary navigation objectives, but its specific contribution is the choice to keep every auxiliary component outside the inference pathway.

This distinction matters for embodied systems. The real-world setup uses a Unitree G1, a ZED X Mini camera, and a Jetson AGX Orin, but policy inference runs on a remote RTX 5090 workstation. A second large geometry encoder on that path would increase compute and system coupling. Training-only guidance avoids an additional permanent module even if it does not eliminate the cost of the backbone itself.

2. Background: Teachers, Heading, and Route Progress

The policy $\pi_\theta$ receives an instruction $l$ and the observation history $O_t=(o_0,\ldots,o_t)$, then selects from forward, left, right, and stop. Standard supervised navigation maximizes the likelihood of the expert action:

$$\mathcal{L}_{\mathrm{nav}}=-\mathbb{E}_{\mathcal{D}}\left[\log\pi_{\theta}\left(a_{t}^{\star}\mid l,O_{t}\right)\right].$$

This objective constrains the output token rather than the intermediate state. A large autoregressive navigator may encode the current scene in multiple possible ways while still fitting the action labels. StageVLN adds explicit pressure on local geometry, inter-frame orientation, and route-level translation.

The three signals are complementary. Spatial distillation targets the structure inside the current image. Relative heading targets relations between historical and current views. Progress prediction targets the global state of the episode. Together they span local, relational, and global scales.

3. Method Overview

Figure 1 frames the deployment difference. Explicit 3D methods require extra sensing or maps at inference. Runtime geometry fusion keeps a geometry model on the inference path. StageVLN confines spatial and trajectory guidance to training and leaves the navigator alone after optimization.

Figure 1: StageVLN preserves an RGB-language inference path while moving geometry and trajectory guidance into training.

Figure 2 gives the full pipeline. The current RGB image is passed to a frozen geometry teacher, while the navigator receives up to eight historical frames, the current frame, and the instruction. The teacher processes only the current frame. The navigator's early, middle, and late layers are aligned with different teacher layers through separate projectors.

Figure 2: Training computes four losses; deployment removes the teacher, projectors, and trajectory heads.

flowchart TB
    subgraph TRAIN["Training"]
        RGB["Current RGB"] --> TEACHER["Frozen VGGT-Ω Teacher"]
        TEACHER --> FEAT["Multi-level Geometry Features"]
        HIST["History and Current Frames"] --> POLICY["Qwen3.5-4B Navigator"]
        INSTR["Language Instruction"] --> POLICY
        POLICY --> HIDDEN["Visual Tokens at Layers 8 16 24"]
        FEAT --> DISTILL["Multi-level Cosine Distillation"]
        HIDDEN --> DISTILL
        HIDDEN --> HEAD["Relative Heading Head"]
        HIDDEN --> PROG["Route Progress Head"]
        POLICY --> NAVLOSS["Action Loss"]
        DISTILL --> TOTAL["Joint Objective"]
        HEAD --> TOTAL
        PROG --> TOTAL
        NAVLOSS --> TOTAL
    end
    TOTAL --> UPDATE["Update Navigator and Training-only Modules"]
    subgraph DEPLOY["Inference"]
        DRGB["RGB and Instruction"] --> DPOLICY["Navigator Only"]
        DPOLICY --> ACTION["Forward Left Right Stop"]
    end
    UPDATE --> DPOLICY
  

4. Hierarchical Geometric Representation Distillation

Let $g_\phi$ be a frozen geometry teacher. For the current observation $o_t$, it produces features at $M$ selected layers:

$$\left\{G_{t}^{(r_m)}\right\}_{m=1}^{M}=g_{\phi}(o_t).$$

The index $r_m$ identifies the teacher layer used for alignment pair $m$. The implementation maps navigator layers 8, 16, and 24 to VGGT-Ω aggregator outputs at zero-based indices 11, 17, and 23. This creates supervision across the depth of the network rather than concentrating everything at the final representation.

The teacher patch features receive positional encodings. They are then reshaped into a two-dimensional grid, resized to the student's current-image grid, and flattened in the corresponding spatial order. The detached target is

$$Z_{t}^{(m)}=\operatorname{sg}\left[R_{t}\left(G_{t}^{(r_m)}+E_{t}^{(r_m)}\right)\right].$$

Here $E_t^{(r_m)}$ is the positional encoding, $R_t$ is the spatial reshaping operation, and $\operatorname{sg}$ stops gradients. Non-patch tokens are excluded. The teacher remains frozen, so the target cannot drift toward the student architecture during training.

On the student side, StageVLN extracts only current-image visual tokens $V_{t,n}^{(\ell_m)}$ from the selected navigator layer. Historical visual tokens, language tokens, action tokens, and padding are excluded. A separate trainable projector $P_m$ maps the student representation to the teacher feature dimension.

The alignment uses mean cosine distance:

$$\mathcal{L}_{\mathrm{spatial},t}^{(m)}=\frac{1}{N_{t}}\sum_{n=1}^{N_{t}}\left[1-\cos\left(P_{m}\left(V_{t,n}^{(\ell_m)}\right),Z_{t,n}^{(m)}\right)\right].$$

$N_t$ is the number of aligned tokens. Cosine distance compares directions rather than raw activation ranges, which is useful when teacher and student use different pretrained parameter scales. The full spatial objective averages the $M$ alignment losses:

$$\mathcal{L}_{\mathrm{spatial}}=\mathbb{E}_{\mathcal{D}}\left[\frac{1}{M}\sum_{m=1}^{M}\mathcal{L}_{\mathrm{spatial},t}^{(m)}\right].$$

Each projector consists of LayerNorm followed by a two-layer MLP with dimensions $2560\rightarrow4096\rightarrow2048$ and a GELU activation. Gradients update the projectors and navigator while the teacher target is detached. The spatial loss has weight 0.3, with equal averaging over the three levels.

This is not explicit geometry prediction. The student is not required to decode depth, camera pose, or a map. It learns an implicit spatial bias through feature alignment. That distinction explains the deployment benefit, but it also identifies a risk: if the teacher representation is systematically poor in a new environment, the navigator has no explicit geometric output that can be inspected or corrected at runtime.

5. Relative Heading Across Observations

Current-frame distillation does not specify that the agent has moved or rotated between frames. StageVLN therefore selects historical frames and forms a relative-heading target. For observation time $\tau$, visual tokens at layer $\ell_h$ are mean-pooled and passed through a shared frame projector $F$:

$$e_{\tau}=F\left(\frac{1}{N_{\tau}}\sum_{n=1}^{N_{\tau}}V_{\tau,n}^{(\ell_h)}\right).$$

A shared relation head $R$ combines the historical and current frame embeddings with their difference and element-wise product:

$$u_{i,t}=R\left([e_{\tau_i};e_t;e_t-e_{\tau_i};e_{\tau_i}\odot e_t]\right)\in\mathbb{R}^{2}.$$

The target is derived from the expert action sequence. Left and right turns contribute positive and negative nominal increments of 15 degrees; forward and stop contribute zero. The increments between the selected history index and the current time are summed and wrapped to remove angular discontinuity:

$$\Delta\psi_{i,t}=\operatorname{wrap}_{[-\pi,\pi)}\left(\sum_{s=\tau_i}^{t-1}\delta(a_s^{\star})\right).$$

The angle is represented as a unit vector:

$$r_{i,t}=[\sin\Delta\psi_{i,t},\ \cos\Delta\psi_{i,t}]^{\top}.$$

Prediction and target are compared with cosine distance:

$$\mathcal{L}_{\mathrm{heading}}=\mathbb{E}_{\mathcal{D}}\left[\frac{1}{|\mathcal{H}_{t}|}\sum_{i\in\mathcal{H}_{t}}\left(1-\cos(u_{i,t},r_{i,t})\right)\right].$$

The current expert action $a_t^\star$ is excluded from the accumulated target. The authors emphasize that this objective predicts accumulated relative rotation rather than the orientation represented by the next action. Together with the pre-action input for the progress head, this avoids giving the auxiliary heads access to the exact action token under teacher forcing.

The dependence on a fixed nominal turn increment is a meaningful approximation. Continuous control or simulator interpolation can produce rotations that do not correspond exactly to a sequence of 15-degree labels. The paper does not compare against targets computed from continuous camera poses, so the effect of this quantization remains untested.

6. Expert-Route Progress

Heading captures local rotation but not global route state. The progress target is defined as the fraction of expert forward actions completed before time $t$:

$$p_t=\frac{\sum_{s=0}^{t-1}\mathbf{1}[a_s^{\star}=\texttt{MOVE\_FORWARD}]}{\sum_{s=0}^{T-1}\mathbf{1}[a_s^{\star}=\texttt{MOVE\_FORWARD}]}.$$

Turns and stop do not enter either numerator or denominator. The target is bounded between zero and one and avoids direct dependence on metric distance or scene scale. It measures progress through the demonstration's action sequence, not semantic task completion in an arbitrary route.

Let $j_t$ be the position of the first supervised action token. The progress head $Q$ reads the hidden state immediately before it:

$$z_t=Q\left(H_{t,j_t-1}^{(\ell_p)}\right).$$

The soft-target binary cross-entropy is implemented in the numerically stable form

$$\mathcal{L}_{\mathrm{progress}}=\frac{1}{|\mathcal{B}_P|}\sum_{t\in\mathcal{B}_P}\left[\operatorname{softplus}(z_t)-p_tz_t\right].$$

This pre-action constraint matters. If $Q$ could read the action token, the progress task could become a trivial side channel that no longer forces the state before action generation to encode route progress. The head is excluded at inference and is not used as a stopping policy.

7. Joint Objective and Training Configuration

The four losses form one objective:

$$\mathcal{L}=\mathcal{L}_{\mathrm{nav}}+\lambda_s\mathcal{L}_{\mathrm{spatial}}+\lambda_h\mathcal{L}_{\mathrm{heading}}+\lambda_p\mathcal{L}_{\mathrm{progress}}.$$

The authors set $\lambda_s=0.3$ and $\lambda_h=\lambda_p=0.1$. Action supervision therefore remains the dominant term. The auxiliary losses alter the representation used to generate the action, but they do not replace the original action-generation target.

The navigator is Qwen3.5-4B and the geometry teacher is frozen VGGT-Ω. The visual encoder is also frozen. The language model and output head use a learning rate of $10^{-6}$; the visual merger, alignment projectors, and trajectory heads use $10^{-5}$. Training lasts one epoch with global batch size 64 on four H100 GPUs.

Navigation fine-tuning uses 10,819 R2R-CE episodes and 19,990 RxR-CE episodes. Evaluation uses validation-unseen splits of 1,839 and 3,669 trajectories. The zero external-data entry in the main table excludes the pretrained data of Qwen and VGGT-Ω; it means that no additional navigation training set is counted.

8. Main Navigation Results

On R2R-CE validation-unseen, StageVLN reaches 56.3% success rate, 51.4% SPL, and 64.7% oracle success with an NE of 5.17. On RxR-CE it reaches 54.3% SR and 42.8% SPL with an NE of 5.65. These results use a 4B navigator and no additional geometry encoder at inference.

Method	Parameters	R2R NE↓	R2R OS↑	R2R SR↑	R2R SPL↑	RxR NE↓	RxR SR↑	RxR SPL↑
ActiveVLN	3B	5.31	58.3	50.1	43.7	5.84	50.7	41.2
StreamVLN*	7B	6.05	53.8	45.5	41.6	-	-	-
NaVILA*	7B	5.22	62.5	54.0	49.0	6.77	49.3	44.0
InternVLA-N1	8B	4.89	60.6	55.4	52.1	6.41	49.5	41.8
JanusVLN	7B + 1B	5.17	58.0	52.8	49.2	6.46	51.4	44.3
StageVLN	4B	5.17	64.7	56.3	51.4	5.65	54.3	42.8

The table does not support a claim that StageVLN dominates every metric. It has the highest listed R2R OS and SR, but InternVLA-N1 has lower R2R NE and higher R2R SPL. StageVLN has the highest RxR SR and lowest NE, while NaVILA, JanusVLN, and ActiveVLN report higher RxR SPL. The pattern suggests more frequent goal completion with some sacrifice in path efficiency.

Against JanusVLN, which retains a 1B geometry encoder, StageVLN improves R2R SR and SPL by 3.5 and 2.2 points, and RxR SR by 2.9 points, but loses 1.5 points of RxR SPL. The defensible interpretation is that training-only geometric supervision improves success under the evaluated configuration without requiring the extra inference module. It is not a universal efficiency or accuracy win.

The comparison also mixes model scales, pretraining data, and observation interfaces. The authors control external navigation data through explicit labels, but do not run a full factorial comparison over backbone size and pretraining recipe. The 4B result relative to larger models is therefore informative, but it does not isolate every source of advantage.

9. Inference Cost and Real-World Demonstration

Efficiency is measured on one RTX 5090 and one RTX 6000 Pro. StageVLN allocates 10.24 GiB of peak memory and takes 304.5 or 229.8 milliseconds per step. It substantially reduces memory relative to geometry-fusion baselines, but it is not the fastest policy in the comparison.

Hardware	Method	Peak Allocated VRAM GiB↓	Mean Step Time ms↓
RTX 5090	JanusVLN	OOM	-
RTX 5090	NaVILA	17.47	365.2
RTX 5090	StreamVLN	17.48	233.4
RTX 5090	StageVLN	10.24	304.5
RTX 6000 Pro	JanusVLN	35.73	481.3
RTX 6000 Pro	NaVILA	17.47	383.1
RTX 6000 Pro	StreamVLN	17.48	209.8
RTX 6000 Pro	StageVLN	10.24	229.8

Relative to NaVILA, StageVLN lowers peak allocated memory by 41.4%, reduces step time by 16.6% on the RTX 5090, and reduces it by 40.0% on the RTX 6000 Pro. StreamVLN remains faster at both measured latencies. The main advantage of StageVLN is therefore architectural: it avoids carrying a separate geometry encoder, which is different from claiming the lowest absolute latency among all VLN systems.

The robot experiment uses a Unitree G1, a ZED X Mini camera, and a Jetson AGX Orin. The Jetson handles image preprocessing and communication, while policy inference runs on a remote RTX 5090. Figure 3 shows two office routes with corridor traversal, doorway transitions, plant avoidance, room entry, and object-referenced stopping. Both demonstrations succeed without additional real-world fine-tuning.

Figure 3: The G1 platform, a simulated comparison with JanusVLN, and two real-world deployment sequences.

This is evidence of feasibility, not broad real-world validation. There is no reported success rate, collision rate, localization drift, dynamic-obstacle test, lighting study, or network-latency distribution. The remote workstation architecture also means the policy is not executing fully on the robot's onboard computer.

10. What the Ablations Show

The component study uses VGGT rather than the stronger VGGT-Ω teacher used in the headline result. Supervised fine-tuning obtains 46.49% SR and 42.32% SPL. Single-level geometric alignment reaches 48.50/41.86, while multi-level alignment reaches 52.26/46.72. The hierarchy is therefore more useful than supervising only the final pair in this setup.

Action	Geometry	Heading	Progress	NE↓	OS↑	SR↑	SPL↑
Yes	No	No	No	5.19	56.01	46.49	42.32
Yes	Single	No	No	6.33	61.83	48.50	41.86
Yes	Multi	No	No	6.12	61.01	52.26	46.72
Yes	Multi	Yes	No	6.17	63.30	49.97	43.80
Yes	Multi	No	Yes	5.66	63.89	53.18	48.67
Yes	No	Yes	Yes	5.86	64.06	54.43	48.58
Yes	Multi	Yes	Yes	5.58	67.37	55.36	48.44

The most important observation is that the auxiliary terms do not combine monotonically. Multi-level geometry alone reaches 52.26/46.72, but adding heading falls to 49.97/43.80. Multi-level geometry plus progress reaches 53.18/48.67. Using heading and progress without geometry reaches 54.43/48.58, and the full combination raises OS and SR while reducing SPL slightly relative to some intermediate variants.

Against supervised fine-tuning, the complete VGGT version improves SR and SPL by 8.87 and 6.12 points, yet its NE of 5.58 is worse than the 5.19 of SFT. The model reaches goals more often without making every trajectory closer to the shortest path. The authors explicitly acknowledge that the auxiliary objectives do not improve all metrics uniformly.

Changing the teacher to VGGT-Ω improves NE from 5.58 to 5.17, SR from 55.36 to 56.28, and SPL from 48.44 to 51.37, while OS drops from 67.37 to 64.71. The pattern indicates that teacher representation quality affects downstream navigation, but the reduction in oracle success remains unexplained.

11. Limitations

The first author-stated limitation is the scale of the real-world evidence. The authors emphasize that no extra real-world navigation data or adaptation is used, and they show two successful instructions. This establishes transfer feasibility, not a statistically supported claim about robustness. The remote RTX 5090 also means the onboard Jetson is not executing the full 4B policy.

The second limitation follows from the removal of explicit geometry at inference. StageVLN gains deployment simplicity, but the deployed policy cannot inspect a depth map, a point cloud, or a geometric confidence signal. The paper does not evaluate transparent surfaces, reflections, dynamic crowds, lighting changes, or camera-calibration drift, all of which can invalidate an implicit geometry prior.

The third limitation is label resolution. Relative heading is derived from discrete left and right actions with a nominal 15-degree increment, while the actual continuous rotation can be finer. Progress counts forward actions from the expert route and does not represent metric distance or semantic completion. Both targets provide useful coarse structure, but neither is a full continuous state estimate.

The fourth limitation is experimental completeness. The paper labels external navigation data and runtime geometry use clearly, which is important, but it does not provide a full factorial study of backbone scale, pretraining corpus, observation length, and longer ordinary multitask training. Such controls would be necessary before attributing every improvement to the auxiliary formulation alone.

12. Conclusion

StageVLN offers a concrete engineering pattern for RGB-language navigation. A frozen geometry teacher and two trajectory heads act as temporary training scaffolds. Hierarchical feature alignment shapes the navigator's scene representation, relative-heading supervision shapes cross-frame rotation, and progress supervision shapes the route-level state. At inference, only the original action policy remains.

The results support the pattern while also exposing its limits. StageVLN improves success on two continuous navigation benchmarks and avoids a separate geometry encoder, but RxR SPL is lower than several baselines, the auxiliary effects are non-monotonic, training is studied for one epoch in simulation, and real-world evidence consists of two successful qualitative demonstrations.

The next step is not simply another auxiliary loss. Stronger evidence would compare hierarchical representation distillation with explicit teacher-output distillation, match ordinary multitask supervision and training budgets, report statistically meaningful physical-robot trials, and test whether implicit geometric state survives camera, lighting, and dynamic-scene shifts.

Closing thought: The useful geometric intelligence for deployment may not be another model kept beside the robot; it may be spatial structure that the navigation policy has already internalized before it leaves training.