RIWANav: Recursive World-Action Models with Self-Improvement for Urban Navigation

RIWANav: Recursive World–Action Models with Self-Improvement for Urban Navigation
Jing Xie, Shouwei Ruan, Yubin Wang, Yuxiang Zhang, Haitao Yang, Songchang Jin, Dianxi Shi (MoE Key Lab of Artificial Intelligence, Institute of AI, Shanghai Jiao Tong University / Beihang University / HKUST / Tsinghua University / UT Austin / Intelligent Game and Decision Lab, Beijing)
arXiv:2610.08640v1 [cs.RO], 2026-10-06 · arXiv:2610.08640
Code status: no public code released. The arXiv full text carries no repository link, and searching GitHub by title and authors returns no matching implementation. The three upstream projects it builds on are public: the UrbanNav dataset (CASIA-IVA-Lab/UrbanNav, AAAI 2026), Navigation World Models (facebookresearch/nwm), and NVIDIA Cosmos.

One-sentence summary

A frozen world model becomes a bottleneck once the policy it evaluates has moved on: its predictions no longer match the state distribution the policy actually visits. RIWANav treats the world model and the navigation policy as a single task-specific recursive self-improvement (RSI) system, alternating two updates per cycle. The world model imagines the visual consequences of four candidate actions and feeds group-relative advantages to GRPO; the improved policy then proposes action-video pairs that pass an expert-consistency filter and get prioritised by behavioural novelty and prediction error, which in turn adapts the world model. On UrbanNav test-unseen, success rate rises from 80.31% for the frozen-world-model baseline to 88.25%, SPL from 76.09% to 82.59%, and navigation error falls from 3.3314 to 2.0429.

1. Background: errors compound in urban navigation

The task shape is concrete. The agent sees only street-level observations and a short motion history, and must translate a language-specified destination into a sequence of local decisions, including deciding when to stop. The paper decomposes the failure modes of existing approaches into three layers.

First, imitation learning never sees failure. CityWalker and UrbanNav supervise on expert trajectories, which gives a strong initialisation, but the supervision only covers the actions an expert got right. The states a deployed policy wanders into have no labels at all. This is unusually damaging in urban navigation because one inaccurate local motion changes the next observation, and the error compounds through every subsequent decision: drift half a step off the sidewalk and the visible traversable corridor, the obstacles, and the pedestrians you must yield to all change.

Second, physical trial and error is expensive. Reinforcement learning can learn from those failures, but long-horizon trial-and-error interaction has to happen in a real city, where a failed episode means clipping a bollard, stepping into a traffic lane, or idling at a junction until timeout. The paper therefore states the requirement precisely: post-training needs outcome feedback without physically executing every candidate action.

Third, and this is the real target: a frozen world model goes stale. Action-conditioned world models such as NWM and Fast-WAM act as learned simulators, predicting the visual consequence of a candidate action so the policy can compare local decisions before committing. That solves the "feedback without execution" problem. But the paper argues that a world model used for policy optimisation should not stay frozen. As the policy improves, the state distribution it visits and the action distribution it proposes both drift, and the frozen model does not follow. Its prediction error then distorts subsequent policy updates, making it the bottleneck of model-based post-training.

Figure 1: from imitation learning to recursive world-action self-improvement, in three steps. (a) Imitation learning trains on expert trajectories and offers almost no supervision for policy-induced failures. (b) A frozen world model lets the policy learn from imagined outcomes, but its predictions drift out of alignment with the evolving policy. (c) RIWANav closes the training loop: imagined outcomes improve the policy, and grounded policy-induced samples refine the world model used for the next round of policy learning.

The proposed remedy is framed in terms of recursive self-improvement: systems in which each improvement stage changes the conditions for the next, of which POET's co-evolving environment-agent pairs are the canonical example. RIWANav brings that principle to navigation post-training in a specific form, coupling policy optimisation with world-model adaptation. The improved policy decides which demonstrations the world model should learn next; the refined world model provides updated feedback for subsequent policy learning. Self-improvement emerges from the recursive loop between the two components rather than from any single stronger component.

2. Related work, and where RIWANav sits

Real-world navigation. This line advances through data, supervision, and policy design. CityWalker scales urban navigation learning from large collections of walking and driving video. UrbanNav aligns city-scale human trajectories with landmark-grounded language. OmniVLA learns one policy shared across image, language, and mixed goal modalities. NoMaD uses a goal-masked diffusion policy, and VENTURA generates instruction-conditioned visual paths for downstream control. The paper positions itself against all of them the same way: they improve training, this work improves post-training.

World models for policy learning. PlaNet and Dreamer established latent imagination for policy learning; later work scaled it to broader domains and high-dimensional visual observation, and more recently to navigation and embodied control. NWM ranks navigation trajectories with controllable video prediction, while Fast-WAM jointly models video and actions to improve policy learning without explicit future generation. Where these treat the world model as a fixed source of imagined feedback, this paper studies how the world model and the policy can continually adapt to one another as policy behaviour evolves.

Recursive self-improvement. At the level of learned dynamics the problem has been approached several times: uncertainty-aware prediction and short rollouts reduce exploitation of model error, conservative objectives penalise uncertain transitions, PDML tilts model learning toward the current policy distribution, and a separate line prioritises experience by learning progress, novelty, or curiosity. RIWANav's contribution is to connect these into one closed loop: the world model improves the policy through imagined feedback, and the evolving policy guides the next world-model update.

3. Preliminaries: formulation and the world-model interface

The offline dataset is UrbanNav, written $\mathcal{D}=\{(s_t, a_t^{\mathcal{D}}, o_{t+1:t+T}^{\mathcal{D}})\}_t$, where $a_t^{\mathcal{D}}$ is the recorded expert action and $o_{t+1:t+T}^{\mathcal{D}}$ holds its subsequent $T$ RGB frames. A state packs the recent observation window, the corresponding poses, and the language goal:

$$s_t = (o_{t-C+1:t},\ q_{t-C+1:t},\ \ell)$$

The policy $\pi_\theta(a\mid s)$ defines a distribution over an $H$-waypoint action $a=[p_1,\ldots,p_H]\in\mathbb{R}^{H\times 2}$ in the current camera frame, where each $p_j\in\mathbb{R}^2$ is a planar waypoint; a separate head predicts the stop probability. In the experiments $H=8$, covering eight waypoints over eight seconds.

The action-conditioned Cosmos world model predicts the visual consequence of a proposed action as

$$\hat{o}_{t+1:t+T} = f_\phi\big(o_t,\ \mathcal{A}(a),\ \ell\big) \tag{1}$$

where $\mathcal{A}$ converts the waypoint chunk into the relative-pose sequence that Cosmos consumes; concretely, 16 relative-pose actions drive a 17-frame prediction at 2 fps. The supervised policy $\pi_{\theta_0}$ and the pretrained world model $f_{\phi_0}$ initialise the recursion. At recursion stage $k$, $f_{\phi_k}$ is held frozen while the policy improves by comparing imagined outcomes for actions proposed from the same state; the improved policy then supplies expert-consistent action-video pairs, and the grounded self-curriculum selects from that pool to update $f_{\phi_k}$ into $f_{\phi_{k+1}}$. Repeating the recursion yields the final world model $f_{\phi_K}$, with $K=19$ rounds in the reported runs.

Two fixed references run through the whole method and deserve to be called out separately. The recorded expert action $a_t^{\mathcal{D}}$ anchors future supervision: the future frames paired with any candidate always come from the dataset. The initial policy $\pi_{\theta_0}$ provides a fixed baseline for behavioural novelty: it defines how far a candidate has moved from the cold-start behaviour. Neither is updated during recursion, and that is what keeps the scheme grounded rather than free-running.

4. Method

4.1 The two beats of one recursive cycle

Figure 2: the recursive self-improvement loop in RIWANav. (1) The current world model imagines action consequences that improve the policy. (2) The improved policy constructs a grounded self-curriculum that improves the next world model. Repeating these two updates changes both components and the feedback exchanged between them. In the original figure a flame marks the component being optimised in each phase.

Each cycle strictly alternates two updates. Beat one, policy improvement: the current world model predicts the visual outcomes of several candidate actions, and their relative quality is the basis for comparing actions and improving the policy. Beat two, world-model improvement: the improved policy generates candidate actions, those consistent with expert motion are paired with recorded future frames and enter the policy-induced replay pool, and the grounded self-curriculum prioritises those pairs by behavioural novelty and current prediction error for world-model adaptation. The refined world model then supplies imagined feedback for the next policy update.

The load-bearing detail is that each beat freezes the other: the world model is fixed during the policy update, and the policy does not move during the world-model update. This avoids the instability of letting the evaluator and the evaluated change simultaneously, and it makes attribution clean, which is exactly how the ablation in Section 5.3 is sliced.

4.2 Policy cold start: behaviour-clone a strong initialisation

The recursion needs a strong starting point. The paper obtains it by supervised fine-tuning on UrbanNav expert trajectories, implemented as behaviour cloning: frozen DINOv2 visual features, CLIP text features, and pose embeddings are fused by a transformer waypoint head. The SFT objective $\mathcal{L}_{\mathrm{SFT}}$ combines waypoint $L_1$ error, directional error, and stop-prediction binary cross-entropy on the expert data.

The resulting $\pi_{\theta_0}$ plays two roles during post-training: it is the initialisation for policy optimisation, and it is the fixed behavioural reference throughout, serving both as the reference policy for the KL penalty and as the baseline for behavioural novelty. The paper is explicit that on its own it does not compare alternative actions proposed by the learned policy, which is precisely the capability post-training has to add.

4.3 Learning from imagination: GRPO with three constraints

At state $s$ the current policy outputs a mean waypoint chunk $\mu_\theta(s)$, from which $M$ candidate actions are sampled:

$$a_i \sim \mathcal{N}\big(\mu_\theta(s),\ \sigma^2 I\big),\qquad i\in\{1,\ldots,M\}$$

where $\sigma$ controls the sampling scale and $I$ is the identity matrix; the implementation uses $M=4$. The world model is held fixed during this phase and predicts each candidate's visual outcome $\hat{o}_i^{+}$ through Eq. (1). A navigation scoring function assigns each candidate a score

$$r_i = R_{\mathrm{nav}}(s,\ a_i,\ \hat{o}_i^{+})$$

measuring the quality of its imagined consequence. The paper stresses a methodological choice here: comparing candidates from the same state reduces sensitivity to the absolute reward scale and directly identifies which local decision produces the better imagined outcome. That is also the argument for GRPO over PPO, since group-relative comparison is inherently immune to reward calibration.

The group-relative advantage is

$$A_i = \frac{r_i - \bar{r}}{\operatorname{std}(r_{1:M}) + \epsilon_A} \tag{2}$$

with $\bar{r}$ and $\operatorname{std}(r_{1:M})$ the mean and standard deviation of the $M$ rewards and $\epsilon_A>0$ for numerical stability. Let $\pi_{\mathrm{old}}$ be the policy that sampled the group and $\rho_i(\theta)=\pi_\theta(a_i\mid s)/\pi_{\mathrm{old}}(a_i\mid s)$ its likelihood ratio. The waypoint-action update optimises

$$\mathcal{L}_{\mathrm{Policy}} = -\frac{1}{M}\sum_{i=1}^{M}\min\Big(\rho_i A_i,\ \operatorname{clip}(\rho_i, 1-\eta, 1+\eta)A_i\Big) + \beta D_{\mathrm{KL}}\Big(\pi_\theta(\cdot\mid s)\ \big\|\ \pi_{\theta_0}(\cdot\mid s)\Big) + \lambda_{\mathrm{exp}}\mathcal{L}_{\mathrm{exp}} \tag{3}$$

The three terms divide labour in a way worth reading carefully. The clipped term bounds the per-step update magnitude by the range $\eta$. The KL term constrains the action distribution relative to the frozen initial policy $\pi_{\theta_0}$. The expert anchor $\mathcal{L}_{\mathrm{exp}}$ retains the waypoint and direction components of the supervised objective and compares the current prediction against recorded expert waypoints. So the structure of Eq. (3) is: the group-relative term learns from imagined action consequences, while clipping, the KL penalty, and the expert anchor jointly prevent unstable drift away from the supervised policy.

4.4 Policy-induced replay: first make sure the supervision is valid

The central difficulty in beat two is that an action proposed by the improved policy does not necessarily form a reliable correspondence with the recorded future $o_{t+1:t+T}^{\mathcal{D}}$. For a candidate that deviates substantially from $a_t^{\mathcal{D}}$, that recorded clip is simply not its consequence, and training on it teaches the world model a false causal link. The paper's answer is to retain only policy-proposed actions that remain locally consistent with recorded expert motion.

Consistency is measured with the four UrbanNav metrics: average distance error (ADE), maximum average distance error (MADE), average orientation error (AOE), and maximum average orientation error (MAOE). Indexing them by $\mathcal{M}=\{\mathrm{ADE},\mathrm{MADE},\mathrm{AOE},\mathrm{MAOE}\}$, the expert-consistency score is

$$Q_i = \frac{\sum_{m\in\mathcal{M}} w_m \exp\!\left[-d_m(a_i, a_t^{\mathcal{D}})/\tau_m\right]}{\sum_{m\in\mathcal{M}} w_m} \tag{4}$$

where $d_m(a_i, a_t^{\mathcal{D}})$ is the discrepancy from the recorded expert action under metric $m$, $w_m$ is its non-negative weight, and $\tau_m>0$ is a metric-specific scale. The exponential kernel maps unbounded distance and orientation errors into $Q_i\in[0,1]$, and the weighted average across four metrics means a candidate that is close in position but reversed in heading cannot hide behind a single indicator.

Admission requires three conditions at once: a valid rollout, $Q_i \geq q_{\min}$, and the individual maximum-error tolerances, where $q_{\min}$ is the expert-consistency threshold, set to 0.35. Each retained action and its recorded future form an expert-consistent action-video pair, and each admitted triple $(s, a_i, o^{+\mathcal{D}})$ enters the policy-induced replay pool $\mathcal{R}_k$.

One design point is easy to read past: candidates that fail the filter are not discarded. They still take part in policy optimisation; they are only excluded from world-model training. The filter therefore decouples two requirements, letting the policy phase explore broadly while the world-model phase sees only reliably supervised pairs.

4.5 The grounded self-curriculum: how priority is computed

Uniform replay treats all accepted samples equally, ignoring both behavioural novelty and prediction error. The self-curriculum computes three quantities per action-video pair.

The first is behavioural deviation from the initial policy:

$$D_i = \big\lVert a_i - \mu_{\theta_0}(s)\big\rVert_2 \big/ \sqrt{2H}$$

Dividing by $\sqrt{2H}$ normalises the Euclidean distance of an $H\times 2$ waypoint vector so that $D_i$ stays comparable across different $H$. The second is the current prediction error $e_i$: the mean pixel $L_1$ error between temporally aligned frames of the predicted and recorded futures. Both are mapped to bounded scores:

$$N_i = 1-\exp(-D_i/\tau_N),\qquad E_i = 1-\exp(-\tau_E e_i) \tag{5}$$

with $\tau_N>0$ and $\tau_E>0$ controlling the respective scales. The replay priority uses the consistency score as a multiplicative gate and novelty and error as additive terms:

$$p_i = Q_i\left(\epsilon_p + \lambda_N N_i + \lambda_E E_i\right) \tag{6}$$

where $\lambda_N$ and $\lambda_E$ balance novelty against error, both set to 0.50, and $\epsilon_p>0$ preserves a non-zero priority. Placing $Q_i$ in the multiplicative slot is deliberate: a poorly consistent sample is damped globally no matter how novel or badly predicted it is, while $\epsilon_p$ keeps a baseline sampling probability for pairs that are consistent but neither novel nor hard.

At each world-model update, newly admitted and historical pairs compete in the same replay pool and are sampled without replacement in proportion to $p_i^\alpha$, with $\alpha\geq 0$ controlling the strength of prioritisation. The combined effect is to focus adaptation on pairs that are expert-consistent and either differ from the initial policy's behaviour or are still poorly predicted by the current world model.

4.6 The world-model update: Cosmos' native objective

The sampled batch $\mathcal{B}_p\subset\mathcal{R}_k$ defines the world-model objective

$$\mathcal{L}_{\mathrm{WM}}(\phi) = \mathbb{E}_{(s,a_i,o^{+\mathcal{D}})\sim\mathcal{B}_p}\Big[\mathcal{L}_{\mathrm{video}}\big(f_\phi(o_t, \mathcal{A}(a_i), \ell),\ o^{+\mathcal{D}}\big)\Big] \tag{7}$$

where $o^{+\mathcal{D}}$ is the recorded future paired with $a_i$ and $\mathcal{L}_{\mathrm{video}}$ is the native conditional video-model objective of Cosmos. This matters engineering-wise: the paper does not invent a new loss for world-model adaptation. What changes is which data is fed in and with what priority; the video reconstruction objective itself is inherited from the pretrained model. After adaptation, $f_{\phi_{k+1}}$ evaluates subsequent policy-proposed actions and supplies updated imagined feedback, closing the recursive self-improvement loop.

4.7 Implementation numbers and the full data flow of one round

The reported implementation details are complete. The policy predicts eight waypoints over eight seconds and compares four candidates at each state; Cosmos predicts 17 frames at 2 fps from 16 relative-pose actions. Policy training uses a learning rate of $10^{-5}$ and a clipping range $\eta=0.20$. The self-curriculum uses $q_{\min}=0.35$ and equal weights $\lambda_N=\lambda_E=0.50$. The world model is updated every 500 policy steps using at most 500 replay examples. Training runs on four NVIDIA H200 GPUs. Real-world deployment uses a custom-built mobile robotic platform with a forward-facing camera providing RGB observations.

flowchart TD
    D[UrbanNav offline dataset
state + expert action + recorded future] --> SFT[policy cold start SFT
DINOv2 + CLIP + pose embeddings
transformer waypoint head]
    SFT --> P0[initial policy theta0
also the KL reference and novelty baseline]
    P0 --> SAMP[sample M=4 candidate actions
Gaussian perturbation of the mean chunk]
    WM[current world model phik frozen] --> IMAG[imagine each candidate outcome
Cosmos 17 frames at 2fps]
    SAMP --> IMAG
    IMAG --> SC[navigation score R_nav
group scores r_i]
    SC --> GRPO[GRPO group-relative advantage
clip + KL to theta0 + expert anchor]
    GRPO --> PK[improved policy thetak+1]
    PK --> FILT[expert-consistency filter
ADE MADE AOE MAOE weighted Q_i
threshold q_min = 0.35]
    FILT --> POOL[policy-induced replay pool R_k
action-video pairs]
    POOL --> CURR[grounded self-curriculum
novelty N_i + prediction error E_i
priority p_i]
    CURR --> WMUP[world-model update
every 500 policy steps max 500 samples
native Cosmos video objective]
    WMUP --> WM2[new world model phik+1]
    WM2 -. imagined feedback for the next round .-> WM
    PK -. rejected candidates still used for policy optimisation .-> GRPO

Putting the numbers back into the flow exposes the cost structure. One world-model update corresponds to 500 policy steps, and each policy step runs a Cosmos video prediction for each of four candidates. With $K=19$ recursive rounds the world model is updated 19 times, on at most 500 samples each. World-model adaptation is therefore the small line item; imagined feedback during the policy phase is the expensive part. That is the practical content of the authors' limitation about memory and compute for adapting a large video world model.

5. Experiments

5.1 Setup: a genuinely controlled comparison

Evaluation uses the official UrbanNav train, test-seen, and test-unseen splits. Every training framework receives the same recent frames, pose history, and language goals, and predicts eight local waypoints plus a stop probability, sharing the same data split, episode list, trajectory mapping, and 3-m success radius. Unless stated otherwise, results are means and standard deviations over three random seeds.

Six metrics are reported: Success Rate (SR, final goal attainment), Oracle Success Rate (OSR, whether any visited pose reached the goal region), Navigation Error (NE, final distance to the goal), SPL (success weighted by path length, an efficiency measure), and nDTW with its success-weighted variant SDTW (normalised dynamic time warping, a trajectory-fidelity measure). All are better when higher except NE.

The baseline construction is the most instructive part of the experimental design: the controlled comparison fixes the policy architecture, initialisation, data, interface, and evaluation protocol, and changes only the post-training framework. Imitation learning is the policy shared by all frameworks. PPO and GRPO are policy-only baselines optimising the same navigation objective without action-conditioned visual feedback. PPO-WM and GRPO-WM add a frozen Cosmos on top of the corresponding optimiser. RIWANav uses GRPO-based policy improvement and additionally adapts Cosmos with the proposed grounded self-curriculum. In the comparison with prior approaches (Table 2), NoMaD receives text conditioning and a stop head, OmniVLA is fine-tuned through LoRA and its continuous action head, NWM reranks four policy candidates by visual-language goal alignment, and UrbanNav and Fast-WAM keep their trained checkpoints and native action interfaces.

5.2 Main results: the jump produced by closing the loop

Training framework (test-unseen)	SR (%) ↑	SPL (%) ↑	NE ↓	nDTW (%) ↑	SDTW (%) ↑	OSR (%) ↑
Imitation learning	67.88	66.03	4.4537	80.85	64.99	68.16
PPO	67.94 ± 0.09	65.65 ± 0.16	4.3768 ± 0.0434	80.76 ± 0.05	65.04 ± 0.08	68.38 ± 0.07
GRPO	70.63 ± 1.10	68.49 ± 1.06	4.5795 ± 0.0894	82.34 ± 0.54	67.82 ± 1.13	70.91 ± 1.07
PPO-WM (frozen world model)	75.05 ± 0.21	72.44 ± 0.19	3.5912 ± 0.0462	84.96 ± 0.15	72.27 ± 0.22	75.56 ± 0.21
GRPO-WM (frozen world model)	80.31 ± 0.94	76.09 ± 1.05	3.3314 ± 0.1014	87.77 ± 0.45	77.62 ± 0.92	81.01 ± 0.82
RIWANav (ours)	88.25 ± 0.43	82.59 ± 0.25	2.0429 ± 0.0815	91.95 ± 0.16	85.51 ± 0.39	89.37 ± 0.44

The ladder has the same shape on test-seen: imitation learning 70.91/70.55 (SR/SPL), PPO 72.04/71.40, GRPO 75.11/74.45, PPO-WM 78.90/78.25, GRPO-WM 83.35/81.27, RIWANav 89.92/86.87, with NE falling from 3.8427 all the way to 1.7842.

The table supports three separate conclusions. First, policy-only post-training buys very little. On test-unseen, PPO moves SR/SPL by +0.06%/-0.38% over imitation learning, and GRPO by +2.75%/+2.46%. Starting from a shared imitation initialisation, swapping the optimiser alone does not extract much. Second, imagined feedback from a frozen world model is the single largest jump. PPO-WM beats PPO by 7.11%/6.79%, and GRPO-WM beats GRPO by 9.68%/7.60%. Imagined visual consequences are already a useful training signal before any world-model adaptation happens. Third, closing the recursive loop adds another segment. Relative to GRPO-WM, RIWANav improves SR/SPL by 6.57%/5.60% on test-seen and 7.94%/6.50% on test-unseen, and is best on all six metrics on both splits. NE drops from 3.3314 to 2.0429, while nDTW, SDTW, and OSR rise from 87.77/77.62/81.01 to 91.95/85.51/89.37.

There is a detail in the third conclusion worth emphasising: success, path efficiency, route fidelity, and oracle success all move in the same direction. The paper uses this to argue that policy-guided world-model adaptation improves the complete rollout, not merely the stopping decision. If only the stop judgement had sharpened, SR and OSR would rise while nDTW and SDTW stayed flat.

Category	Method	test-seen SR / SPL (%)	test-unseen SR / SPL (%)	test-unseen NE ↓	test-unseen SDTW / OSR (%)
Imitation learning	UrbanNav	60.36 / 59.75	60.99 / 59.45	5.3680	57.93 / 61.18
Imitation learning	NoMaD	82.35 / 77.59	84.80 / 79.85	2.3520	80.37 / 85.09
Imitation learning	OmniVLA	77.49 / 77.25	81.11 / 80.80	2.9283	77.05 / 81.17
World-model based	Fast-WAM	58.48 / 39.57	55.25 / 35.00	9.2896	47.16 / 59.75
World-model based	NWM	71.26 / 70.67	67.99 / 65.93	4.4130	65.06 / 68.46
Recursive self-improvement	RIWANav (ours)	89.92 / 86.87	88.25 / 82.59	2.0429	85.51 / 89.37

Against prior approaches RIWANav is best on all six metrics on both splits. Relative to NoMaD, the strongest prior method on SR, it improves test-unseen SR/SPL by 3.45%/2.74%, reduces NE by 0.3091, and raises SDTW/OSR by 5.14%/4.28%. Relative to OmniVLA, the strongest prior method on SPL, it improves SR/SPL by 7.14%/1.79% and nDTW by 1.13%. Among predictive-model baselines it exceeds NWM by 20.26%/16.66% on SR/SPL and cuts NE from 4.4130 to 2.0429. The paper's reading of this row is that beating both learned policies and predictive-model reranking supports recursive self-improvement as a training effect rather than a consequence of a single baseline design.

The table also leaves a question the reader has to settle alone. Fast-WAM and NWM perform well below NoMaD, with test-unseen SR of only 55.25% and 67.99%, yet both are flagship world-model approaches. The paper does not separate how much of that gap is intrinsic to the methods on urban navigation and how much comes from interface adaptation; NWM in particular was repurposed to rerank four policy candidates by visual-language goal alignment. The strength of "20 points above the world-model baselines" therefore depends on whether that reranking adaptation was favourable to NWM.

5.3 Ablation: an honest negative result

WM update	Policy-induced replay	Grounded self-curriculum	SR (%) ↑	SPL (%) ↑
✗	✗	✗	80.31 ± 0.94	76.09 ± 1.05
✓	✗	✗	78.23 ± 1.91	74.72 ± 1.92
✓	✓	✗	84.83 ± 0.33	80.11 ± 0.32
✓	✓	✓	88.25 ± 0.43	82.59 ± 0.25

The ablation runs on test-unseen across four configurations, and the second row is the most informative line in the paper. Turning on world-model updates while replaying recorded expert action-video pairs (static expert replay) makes things worse: SR and SPL fall 2.08% and 1.37% below the frozen-world-model baseline, and the standard deviation grows from ±0.94 to ±1.91.

In other words, "updating the world model" is not itself the source of gain, and can even hurt. The gain comes from whose actions are used for the update. Replacing static expert replay with policy-induced replay under uniform sampling raises SR/SPL by 6.60%/5.39%. Adding the grounded self-curriculum on top, prioritising by novelty and prediction error instead of sampling uniformly, adds a further 3.42%/2.48%. Together they deliver 7.94%/6.50% over the frozen-world-model baseline.

This negative result sharpens the claim from "the world model should be updated" to "the world model must be updated on the behaviour distribution the policy itself produces". Continuing to fine-tune the world model on expert data only makes it better at predicting what an expert would see, whereas policy optimisation needs accurate prediction at the states the current policy actually reaches. The distribution gap between the two is exactly the 2.08% that goes missing, which lines up neatly with the motivation in Figure 1(b).

5.4 Evidence for the recursion: does the world model actually get more accurate

World model	Evaluation actions	$L_1$ ↓	LPIPS ↓
Initial $\phi_0$	Expert	0.1396 ± 0.0032	0.4532 ± 0.0065
	Policy-induced	0.1521 ± 0.0042	0.4539 ± 0.0059
Static expert adaptation	Expert	0.1352 ± 0.0034	0.4265 ± 0.0079
	Policy-induced	0.1505 ± 0.0044	0.4314 ± 0.0067
RIWANav $\phi_K$	Expert	0.1329 ± 0.0034	0.4203 ± 0.0079
	Policy-induced	0.1472 ± 0.0043	0.4217 ± 0.0070

This section answers the paper's third question: does policy-guided adaptation make the world model more accurate on the improved policy's own actions. There are two fixed evaluation sets of 128 state-action pairs each, all excluded from every world-model update: one contains expert actions, the other contains actions generated by the improved policy near the end of training. Over the same eight-second horizon the paper reports frame-wise $L_1$ error and LPIPS, both lower-is-better.

The final RIWANav world model is best on both distributions. Relative to the initial model it reduces $L_1$/LPIPS by 4.8%/7.3% on expert actions and 3.2%/7.1% on policy-induced actions. Static expert adaptation improves both distributions as well; policy-guided adaptation improves on it by a further 1.7%/1.5% on expert actions and 2.2%/2.2% on policy-induced actions. The paper's Fig. 3 traces both metrics across the 19 recursive rounds (round 0 uses the initial world model, the final round $K=19$ the RIWANav model): despite small fluctuations between individual updates, both errors decrease overall on both action distributions, and most of the improvement is retained through the final round.

Read together with the ablation in 5.3, this gives the cleanest causal chain in the paper: policy-guided adaptation improves prediction on the policy's own actions and on expert actions simultaneously, so it is not trading away the original distribution to chase the new one. The magnitudes deserve equal honesty, though. $L_1$ drops only 3.2% (0.1521 to 0.1472) while SR rises 7.94%. The paper offers no mechanism for that leverage ratio between a small fidelity gain and a large performance gain.

5.5 Qualitative results: three world models side by side

Figure 3 (paper Fig. 4): qualitative evidence for recursive world-model improvement on actions generated by the improved policy. The initial world model, static expert adaptation, and RIWANav all receive the same observation, language goal, and action, with the temporally aligned recorded future shown as reference. The rows depict night traffic, a bicycle-lined sidewalk, and a rainy sidewalk. In these cases the RIWANav world model after recursive updates better preserves traversable corridors and scene layout, with fewer long-horizon artifacts.

The paper's account is that the initial world model frequently loses scene structure over the eight-second rollout, static expert adaptation does not consistently resolve distortions under the improved policy distribution, and the final RIWANav world model produces more coherent sidewalk boundaries, facade geometry, and arrangements of vehicles and pedestrians. These examples are consistent with the $L_1$/LPIPS improvements in Table 4. The caveat is that these are three hand-picked cases with no accompanying quantitative statistic, such as an artifact rate across the evaluation set.

5.6 Real-world deployment

Figure 4 (paper Fig. 5): real-world urban navigation with RIWANav. Orange dashed arrows indicate schematic paths and green boxes highlight the targets. The three language goals are a blue circular sign by a white utility box, a black trash bin by green seating, and a blue-tiled passage leading to yellow tables, covering different route layouts and lighting conditions.

The deployment form is simple: given a natural-language instruction and onboard visual observations, the policy predicts waypoints that guide the robot toward the specified goal. The paper shows navigation sequences across different route layouts and lighting conditions and concludes that these qualitative trials demonstrate the learned policy can guide a physical robot toward language-specified landmarks, supporting practical feasibility beyond offline evaluation. This section reports no success rate, no trial count, and no real-world baseline comparison. It is a feasibility demonstration, not a real-robot benchmark.

6. Limitations

Four from the authors. First, the current implementation is task-specific and relies on a fixed offline dataset. Second, and most fundamental: recorded futures can supervise only policy actions close to expert motion, which limits counterfactual exploration without new interaction data. The consistency filter of Eq. (4) is what guarantees valid supervision, and the price is that world-model adaptation is pinned to the neighbourhood of the expert trajectories. Third, adapting a large video world model increases memory and computation. Fourth, although real-world trials demonstrate practical feasibility, broader evaluation across cities, environments, and robot embodiments is still needed.

Five of my own.

First, $R_{\mathrm{nav}}$ is never defined anywhere in the paper. Equations (1) through (7) are written out in detail, but the entire signal source for policy improvement, $r_i = R_{\mathrm{nav}}(s, a_i, \hat{o}_i^{+})$, gets one sentence about measuring the quality of an imagined consequence. Whether it reads predicted pixels at all, or relies only on waypoint geometry and alignment with the language goal, and whether it uses the stop head, determines both whether this can be reproduced and how literally the phrase "imagined feedback" should be taken. This is the largest reproducibility gap in the paper.

Second, the magnitudes of the world-model gain and the policy gain do not match. $L_1$ on policy-induced actions improves by 3.2% and LPIPS by 7.1%, while SR improves by 7.94% and NE by 38.7%. Either GRPO is extremely sensitive to small biases in the scoring function, which would raise questions about stability, or $R_{\mathrm{nav}}$ depends mainly on some local feature of the prediction, such as the boundary of the traversable corridor, that gets averaged away by $L_1$ and LPIPS. The paper does not attempt this attribution.

Third, the static-replay negative result correctly credits the data distribution, but it also weakens the need for the "recursive" framing. If the key ingredient is updating the world model on the policy's own actions, then a non-recursive alternative that simply samples from the current policy, filters, and updates, alternating but without a priority curriculum, already reaches 84.83/80.11, which is most of the final 88.25/82.59. The 3.42%/2.48% contributed by the self-curriculum is real, but the framework name carries more conceptual weight than the share of the gain it explains.

Fourth, none of the key hyperparameters are ablated. $q_{\min}=0.35$ sets the admission gate for the replay pool and directly controls the trade-off between supervision validity and coverage of the policy distribution. $\lambda_N$ and $\lambda_E$ are simply set equal at 0.50. No values or sensitivity analyses are given for $\alpha$ (prioritisation strength), $\tau_N$, $\tau_E$, $\tau_m$, or $w_m$. The threshold matters most: raising it pushes the world model back toward the expert distribution, reproducing the negative result in row two of the ablation, while lowering it admits mismatched action-video pairs. Where the optimum sits and how wide it is remains unanswered.

Fifth, the novelty reference $\mu_{\theta_0}$ is fixed. $D_i$ measures distance from the cold-start policy, which is equivalent to "novel" early in training, but by round 19 the policy has moved far enough that most samples saturate at high $D_i$. The novelty term then decays into a constant offset and the curriculum is effectively driven by $E_i$ alone. Measuring deviation from the current policy $\mu_{\theta_k}$, or density relative to the replay pool history, would fit the intended meaning of a curriculum better. The paper does not discuss this degradation across rounds.

7. Conclusion and outlook

RIWANav's contribution compresses to one sentence: a world model used for policy optimisation cannot stay frozen, but it also cannot be updated on expert data. It has to be updated on the part of the policy's own behaviour that recorded futures can still legitimately supervise. Concretely, the paper formalises the coupled adaptation of a world model and a navigation policy as task-specific recursive self-improvement, alternating two updates per round with each freezing the other. On the policy side, GRPO group-relative advantages are combined with clipping, a KL penalty to $\pi_{\theta_0}$, and an expert anchor. On the world-model side, a four-metric exponential-kernel consistency score $Q_i$ gates admission, behavioural novelty $N_i$ and prediction error $E_i$ add priority, and the sampled batch is trained with Cosmos' native video objective.

The experimental design is where the paper is strongest: a controlled comparison holding the policy architecture, initialisation, data, interface, and evaluation protocol fixed while varying only the post-training framework; three seeds with means and standard deviations; six metrics moving together; and two fixed world-model evaluation sets excluded from all updates. The negative result about static expert replay is reported honestly and used to support the argument, which is not common in the world-model literature.

Looking forward, the authors' own limitations are effectively the roadmap, and the first item is relaxing the fixed offline dataset. The most awkward tension in the current design is that the policy is supposed to explore new behaviour while supervision can only come from recorded futures, so the consistency filter has to trim exploration back into the expert neighbourhood. Allowing even a small amount of real interaction, perhaps only at the states where the world model is least confident, would let the $Q_i$ gate loosen, open up counterfactual exploration, and raise the substance behind the word "recursive". A second direction is moving this coupled update from task-specific navigation to multi-task or cross-embodiment settings; today it depends on the strictly aligned pair of expert action and recorded future that UrbanNav provides, and a different dataset would require redefining the consistency measure.

Three things are directly transferable for anyone doing robot post-training. One, imagined feedback is a strong signal even from a frozen world model: the GRPO to GRPO-WM step of +9.68% SR is the largest single jump in the paper, so a world model is worth having even without online adaptation. Two, continuing to fine-tune the world model on expert data can hurt, so check first whether the update distribution matches the states the policy now visits. Three, put validity in the multiplicative slot and value in the additive slot, as in $Q_i(\epsilon_p + \lambda_N N_i + \lambda_E E_i)$ from Eq. (6); that structure guarantees no amount of novelty can push an invalidly supervised sample into training, and it transfers to any prioritised replay setting.

Golden line

The world model is not the policy's teacher, it is the policy's mirror. When the policy walks away, the mirror has to be reground, and grinding it with old photographs of the expert only leaves it unable to show where the policy is standing now.