LoopWAM: Closed-Loop Generalist Policy Improvement from Deployment Rollouts

Paper Metadata

Title: LoopWAM: Closed-Loop Generalist Policy Improvement from Deployment Rollouts

Authors: Wenxuan Guo, Yingping Liang, Zhaoguo Wang (core contributors), Ziyuan Li, Angyuan Ma, Zhenyu Wu, Chengkun Wang, Bingyao Yu, Ze Chen, Borui Zhang, Xiuwei Xu, Meng Zhang, Erjin Zhou, Jiwen Lu (corresponding), et al. (Tsinghua University, Dexmal, Loopeva)

Links: arXiv:2610.18873 (v2, 2026-10-15) · Project page · Code · HuggingFace dataset

Code status: Fully released. The repository ships training and evaluation scripts, Hydra configs, per-embodiment real-robot checkpoints plus a RoboTwin 2.0 checkpoint, and the mixed-quality rollout archives are published as a dataset. Training was done on Alibaba Cloud PPU-810E accelerators (non-NVIDIA).

Background and Motivation

Scaling human-collected demonstrations has made generalist manipulation policies markedly more capable, but demonstration scaling alone does not close the loop between deployment and continual improvement. Deployment exposes states, interactions, and failure modes that curated demonstrations never contain, and that experience is exactly what an embodied data flywheel needs. Real-world reinforcement learning can improve a policy during deployment, yet it tends to narrow the model's predictive distribution and struggles to produce multi-task generalists; distilling a generalist from several RL specialists fixes the multi-task issue but still supervises the generalist with curated successful rollouts rather than the full interaction history. The authors therefore attack the harder formulation: improve a generalist directly from the mixture of successful, suboptimal, and failed deployment rollouts.

The difficulty is how to use that mixture productively. Imitating every recorded action absorbs undesirable behavior; keeping only high-quality trajectories throws away the evidence of what physically happens when things go wrong - precisely the evidence a world model needs. Existing routes each solve half of the problem. On the VLA side, pi-0.6 uses advantage-conditioned learning to incorporate autonomous experience and interventions, and pi-0.7 uses quality, speed, and mistake metadata to absorb mixed-quality data and distill specialists into a generalist; both steer behavior distributions during training and select a desirable mode at inference, but they carry no explicit objective for the physical dynamics reflected in action-video transitions. On the world-model side, WAMs couple predictive visual representations with action generation and action-conditioned world models (ACWMs) predict future observations given actions; tau0-WM trains a separate WAM and ACWM and routes failure and recovery data to the ACWM, using it as a selector for policy improvement, while FACT implements both roles in one shared model but keeps sequential action-then-consequence generation, so failure consequences influence the policy only implicitly through shared representations.

The question LoopWAM answers is whether a single WAM can learn directly from mixed-quality deployment rollouts, with every quality supervising the model and learned consequences participating directly in action generation. The answer is built from three mechanisms. Source-dependent noise sampling unifies WAM behavior learning and ACWM consequence learning in one model and one generation stage: video and action noise levels are drawn independently, so low-noise rollout actions can guide prediction of their observed consequences. Quality steering encodes a numeric trajectory quality as a suffix on the instruction and conditions on the expert score at deployment. Quality-weighted supervision modulates how strongly each trajectory's actions are imitated, masking failed action targets while retaining video supervision at every quality. Video and action tokens interact bidirectionally throughout a single parallel denoising process, so learned future representations inform action formation without any separate consequence-evaluation stage.

Figure 1: LoopWAM learns from deployment. The outer ring is the deploy-collect-improve loop: diverse real tasks yield mixed-quality rollouts that return to training through quality-aware learning. Center: three deployment-and-retraining rounds on RoboChallenge Table30-V2 raise overall success from 23.67% to 43.33%. Bottom: benchmark comparisons on RoboTwin 2.0, Table30-V2, and RoboCasa365.

Prerequisites

World-action models and action-conditioned world models. A WAM generates actions while predicting future visual observations, forcing representations to capture scene dynamics; an ACWM predicts future observations conditioned on actions, i.e. it learns the consequences of actions. LoopWAM's central claim is that these two roles need not be two models or two sequential stages: with the right noise sampling, one joint model can serve both.

Flow matching. For a target $X$ and noise $\epsilon\sim\mathcal{N}(0,I)$, a noise level $\sigma\in[0,1]$ defines the linear interpolation $X^{\sigma}=(1-\sigma)X+\sigma\epsilon$, and the network learns a velocity field $v_\theta$ matching $u=\epsilon-X$; inference integrates from pure noise back to clean samples along a schedule $\sigma_k$. Because $\sigma=0$ means clean data and $\sigma=1$ means pure noise, this scalar doubles as a per-modality corruption knob.

Mixture-of-Transformers (MoT). The video and action streams keep separate token projections, transformer parameters, and prediction heads, exchanging features through joint attention at every layer. LoopWAM initializes its video backbone from Wan2.2-TI2V-5B and uses a hidden-1024 ActionDiT (linearly interpolated from the Wan DiT) for the action stream.

Method

Joint world-action modeling: one parallel denoising for video and actions

The setting is language-conditioned manipulation with visual observation $o_t$, robot state $s_t$, and instruction $\ell$; the prediction targets are an action chunk $A_t=(a_t,\ldots,a_{t+H-1})$ and the corresponding future observations $V_t$ over the same horizon. A pretrained video VAE maps $(o_t,V_t)$ to an observation latent $z_o$ and future latents $Z_t$. LoopWAM models the joint distribution

$$p_\theta(A_t,Z_t\mid o_t,s_t,\ell,q), \tag{1}$$

where $q$ is a numeric trajectory-quality score. The model is a dual-stream diffusion transformer: separate token projections, transformer parameters, and heads per stream, with joint attention connecting the streams at every layer. Training uses flow matching with independent perturbation of each target $X_m$ (with $X_v=Z_t$ and $X_a=A_t$):

$$X_m^{\sigma_m}=(1-\sigma_m)X_m+\sigma_m\epsilon_m,\qquad \epsilon_m\sim\mathcal{N}(0,I),\quad m\in\{v,a\}. \tag{2}$$

The joint network predicts velocity fields $(v_\theta^v,v_\theta^a)$ against targets $u_m=\epsilon_m-X_m$, and the observation latent $z_o$ stays clean. A block mask couples the two predictions throughout denoising: writing observation, future-video, and action tokens as $O,V,A$ with queries indexing rows and keys indexing columns,

$$M_{(O,V,A)}=\begin{pmatrix}1&0&0\\ 1&1&1\\ 1&1&1\end{pmatrix}, \tag{3}$$

so observation tokens attend only within the initial frame while video and action tokens see each other and both cross-attend the quality-conditioned instruction (robot-state tokens additionally condition the action stream). Two consequences follow: the world-prediction loss trains features used for action generation, and learning how tasks can be completed is coupled with learning how particular actions affect outcomes, encouraging consistency between video and action generation.

Figure 2: Method overview. Left: mixed-quality data (expert demonstrations plus successful, suboptimal, and failed rollouts). Middle: the MoT dual stream (video branch + action branch) with joint attention; robot-state and text tokens are cross-attended. Top right: quality steering - training conditions on "instruction + Quality q", deployment uses the expert score q*. Bottom: (a) source-dependent noise sampling, expert segments draw video noise from Beta(7,1) and action noise from the base distribution P0 while rollout segments do the opposite; (b) quality-weighted supervision, world weight always 1 and action weight lambda(q), zero for failures; (c) rollout annotation, episode-level (human or VLM) or batch-level (success rate).

Source-dependent noise sampling: recorded actions teach their consequences

Jointly predicting video and actions does not by itself guarantee that predicted futures reflect the consequences of generated actions. To strengthen action-conditioned world modeling inside the same unified model and the same training stage, the authors make recorded rollout actions accessible when predicting their observed consequences: expert and rollout segments use different noise distributions, with video and action noise levels drawn independently within each segment,

$$d=E:\quad \sigma_v\sim\mathrm{Beta}(7,1),\quad \sigma_a\sim\mathcal{P}_0;\qquad d=R:\quad \sigma_v\sim\mathcal{P}_0,\quad \sigma_a\sim\mathrm{Beta}(1,7). \tag{4}$$

where $\mathcal{P}_0$ is the base noise distribution obtained by sampling $u\sim\mathcal{U}(0,1)$ and setting $\sigma=\phi_\kappa(u)=\kappa u/[1+(\kappa-1)u]$ with $\kappa=5$. For expert data, high video noise emphasizes recovering expert futures from strongly corrupted inputs alongside action generation over the base distribution; for rollouts, low action noise preserves much of the recorded motion while the video stream predicts its visual consequences. The world model therefore sees a much broader action-outcome distribution, including transitions where the task remains incomplete or an interaction fails, and joint attention connects this consequence-learning signal to the representations used for action generation. In the released code this mechanism is beta_sigma_timesteps: timesteps equal $k\times T$ with $k\sim\mathrm{Beta}(a,b)$ clamped to $[10^{-4},1-10^{-4}]$, and training_loss selects per sample via torch.where(expert_m, tv_expert, tv_weak) for video and torch.where(expert_m, ta_expert, ta_weak) for actions, with the first-frame latent replaced by the clean latent and excluded from the video loss.

Quality steering and quality-weighted supervision: how much each trajectory teaches

Every trajectory receives a numeric quality score $q$ inherited by all training segments extracted from it; quality is encoded as an instruction suffix, $\ell_q=[\ell;\texttt{Quality: } q]$, whose text embeddings condition both streams. Two annotation granularities are supported: episode-level (a human or VLM scores each trajectory) and coarse-level (one human-provided score per policy checkpoint based on overall success rate and motion smoothness, with failed rollouts taking the minimum score, so only one annotation per checkpoint). Expert demonstrations receive an expert score $q^\star$ above the rollout range, and deployment conditions on $q=q^\star$ to steer sampling toward expert-quality behavior.

On the supervision side, video prediction is retained at all qualities while action supervision is modulated by a nondecreasing weight $\lambda(q)\in[0,1]$: experts get $\lambda(q^\star)=1$ and failed trajectories $\lambda(q)=0$. For sample $i$ and modality $m$ the denoising loss is

$$\ell_{m,i}=\omega(\sigma_{m,i})\left\|v_{\theta,i}^m-(\epsilon_{m,i}-X_{m,i})\right\|_{\mathrm{valid}}^2, \tag{5}$$

with $\omega$ the shared timestep weighting function, the norm taken over valid target elements only, and the clean observation latent plus padding excluded. For a minibatch of $B$ segments the complete objective is

$$\mathcal{L}_{\mathrm{LoopWAM}}=\underbrace{\frac{1}{B}\sum_{i=1}^{B}\ell_{v,i}}_{\text{world supervision}}+\underbrace{\frac{\sum_{i=1}^{B}\lambda(q_i)\,\ell_{a,i}}{\max\!\left(1,\sum_{i=1}^{B}\lambda(q_i)\right)}}_{\text{quality-weighted action supervision}}. \tag{6}$$

The source indicator determines corruption; quality determines the behavior condition and the action weight. Setting $\lambda(q)=0$ removes direct imitation of a trajectory's actions while world-model gradients still flow through the jointly attended video and action representations - the formal version of "failures are useful but not imitated". The released config makes the weight table concrete: quality 1 and 2 (failures) get weight 0.0, quality 3 gets 0.3, quality 4 gets 0.6, and the expert score 5 gets 1.0; in code the action loss is exactly (per-sample loss * timestep weight * mask).sum() / mask.sum().clamp(min=1), matching the denominator of Equation 6 verbatim, with an optional linear score-to-weight mapping (action_loss_quality_score_linear).

Closed-loop self-improvement: deploy, score, retrain, redeploy

The deployed policy uses the high-quality action marginal of the joint model,

$$\pi_\theta(A_t\mid o_t,s_t,\ell;q^\star)=\int p_\theta(A_t,Z_t\mid o_t,s_t,\ell,q^\star)\,\mathrm{d}Z_t, \tag{7}$$

sampled in practice by jointly denoising video and actions from Gaussian noise while holding the current observation latent fixed. Starting from an expert-trained model, deployment rollouts are collected and scored, the model is updated on expert demonstrations plus all accumulated rollouts with Equation 6, and the updated model is redeployed at $q^\star$ to continue the loop. More broadly, the update accepts rollouts from arbitrary deployed policies, even poorly performing ones, because the joint architecture and quality-aware objective allocate each trajectory's contribution according to its observed transitions and behavioral quality; RoboTwin 2.0 and RoboCasa365 validate exactly this learn-from-others setting.

flowchart TD
  DEMO["expert demos D_E
quality q*"] --> MIX["training corpus
D_E + accumulated D_R"]
  POL["deploy pi_theta at q*
joint video+action denoising
sample 64 run 32 then replan"] --> ROLL["rollouts D_R
success / suboptimal / failure"]
  ROLL --> ANN["quality annotation
episode-level 1-12 or coarse 1-5
failed always q=1"]
  ANN --> MIX
  MIX --> NS["source-dependent noise
d=E: sigma_v Beta(7,1), sigma_a P0
d=R: sigma_v P0, sigma_a Beta(1,7)"]
  NS --> MOT["MoT joint denoising
video branch + action branch
bidirectional attention every layer
clean observation latent fixed"]
  MOT --> LV["world loss l_v
kept on every quality"]
  MOT --> LA["action loss l_a x lambda(q)
lambda(1)=0 lambda(5)=1"]
  LV --> OBJ["L_LoopWAM Eq.6
mean l_v + lambda-normalized l_a"]
  LA --> OBJ
  OBJ --> POL

Figure 2 supplement: data flow of the closed loop as implemented. Deployment rollouts enter the training corpus after quality annotation; source-dependent noise sampling decides which modality each segment teaches; quality-weighted supervision concentrates action imitation on high-quality behavior while world supervision covers every quality; the updated model is redeployed at the expert score.

Experiments

Benchmarks and implementation details

Three benchmarks cover real robots and simulation. RoboChallenge Table30-V2 comprises 30 tabletop tasks across four embodiments (ALOHA, DOS-W1, ARX5, UR5) and tests closed-loop self-improvement: one expert-trained multi-task model per embodiment undergoes three successive rounds, each deploying every model on its assigned tasks and collecting about 100 rollout trajectories per task, which are quality-annotated and combined with expert demonstrations and all previous rounds' rollouts for retraining. RoboTwin 2.0 provides 50 bimanual tasks in clean and domain-randomized scenes; following the LingBot-VA multi-task setup the authors use 2,500 clean and 25,000 heavily randomized expert demonstrations, augmented with rollouts from expert-trained LoopWAM, X-VLA, and pi-0.5 (20 clean and 130 randomized rollouts per task per policy, 22,500 trajectories in total, roughly the expert corpus duration), with different random seeds for collection and evaluation. RoboCasa365 adds rollouts from two expert-trained LoopWAM policies (64-step and 32-step action chunks) and from RLDX-1 on top of expert demonstrations.

Implementation: the video backbone, text encoder, and video VAE are initialized from Wan2.2-TI2V-5B, with an action DiT of hidden dimension 1024; each training segment pairs a 64-step action chunk with nine RGB frames sampled at intervals of eight control steps including the initial observation; multi-camera views are stitched into a single image and actions/states are normalized with expert-data statistics; Table30-V2 uses relative joint targets and absolute gripper commands. The repository config adds reproducible detail: batch size 16, learning rate 1e-4 with cosine schedule, 5 epochs, weight decay 1e-2, save_every 2500; a 30-layer video DiT (hidden 3072) and a 30-layer action DiT (hidden 1024); umT5 text embeddings precomputed and cached (load_text_encoder: false); RoboTwin evaluation on 8x A100 with 100 episodes per phase, sampling 64 actions and executing 32 before replanning, prompt_quality_score=5, and gripper binarization. The authors also report engineering gains: preprocessing videos and decoding only sampled frames cut training time from over 70 hours to 40 hours.

Table30-V2: three deploy-and-train rounds nearly double success

Method	ALOHA SR	DOS-W1 SR	ARX5 SR	UR5 SR	Score	Overall SR
Baseline Model	6.00	6.00	32.86	26.67	31.48	14.33
mc	7.00	0.00	47.14	20.00	23.63	15.33
my16	13.00	27.00	54.29	46.67	41.27	30.67
VLA-DM0.5	18.00	42.00	58.57	70.00	54.42	40.67
LoopWAM (expert-trained)	8.00	38.00	24.29	26.67	36.10	23.67
LoopWAM (after 3 rounds)	26.00	42.00	55.71	76.67	56.80	43.33

Table 1: RoboChallenge Table30-V2 (success rate %, selected from the 12 official leaderboard entries). After three deployment-and-training rounds LoopWAM reaches a score of 56.80 and 43.33% overall success, beating runner-up VLA-DM0.5 by 2.38 score points and 2.67 percentage points; against its own expert-trained starting point (23.67% / 36.10) the gain is 19.7 percentage points. Note the ARX5 column, where LoopWAM's 55.71 remains below VLA-DM0.5's 58.57.

The per-round numbers are the most persuasive part of the paper: overall success moves 23.67% to 28.00%, 37.67%, and 43.33%, while the score rises 36.10 to 42.63, 51.65, and 56.80, with both aggregate metrics improving at every iteration. In the single-task post-training regime, shirt folding - a deformable-object task with casually tossed shirts whose corners may be hidden - starts from a DOS-W1 multi-task model post-trained on newly collected expert demonstrations, and two deployment-and-training rounds raise success from 13.20% to 32.54% and 55.56%, with scores 14.44, 38.89, and 67.78. Deployment feedback therefore remains effective after adapting a multi-task model to a challenging individual task.

Figure 3: Closed-loop improvement from real-robot deployment. Top: success-rate and score curves over three deployment-and-training iterations on Table30-V2 (23.67 to 43.33, 36.10 to 56.80) with task snapshots for ALOHA, DOS-W1, ARX5, and UR5. Bottom: two iterations of single-task post-training on shirt folding (13.20 to 55.56, 14.44 to 67.78) with four example initial shirt configurations.

RoboTwin 2.0 and RoboCasa365: best overall without embodied pretraining

Method	Embodied PT.	Clean	Randomized	Average
pi-0	yes	65.9	58.4	62.2
X-VLA	yes	72.9	72.8	72.9
pi-0.5	yes	82.7	76.8	79.8
Motus (from Wan2.2)	no	77.6	77.0	77.3
LingBot-VA	yes	92.9	91.5	92.2
Fast-WAM	no	91.9	91.8	91.8
AHA-WAM	no	93.4	92.2	92.8
LingBot-VA 2.0	yes	93.8	93.4	93.6
LoopWAM	no	94.6	94.4	94.5

Table 2: RoboTwin 2.0 success rates (%, selected). Without additional embodied pretraining, LoopWAM averages 94.5%, improving over AHA-WAM by 1.7 points and over the pretrained LingBot-VA 2.0 by 0.9 points; clean and randomized scores are nearly identical (94.6 / 94.4), indicating that the broader state-action-outcome coverage from mixed rollouts converts into robustness under domain randomization.

Method	Embodied PT.	Atomic Seen	Composite Seen	Composite Unseen	Overall
Diffusion Policy	no	15.7	0.2	1.3	6.1
pi-0.5	yes	39.6	7.1	1.2	16.9
GR00T N1.5	yes	50.7	14.8	2.7	23.9
RLDX-1	yes	67.6	27.9	8.5	36.0
PRTS	yes	66.3	30.3	18.8	39.6
ABot-M0.6	yes	79.4	48.3	7.9	46.6
LoopWAM	no	79.7	50.3	14.9	49.5

Table 3: RoboCasa365 success rates (%, selected). LoopWAM reaches 49.5% overall, 2.9 points above ABot-M0.6 without embodied pretraining, and the improvement extends to held-out composite tasks (14.9%); PRTS remains higher on Composite Unseen at 18.8%, one of the few cells where LoopWAM trails.

Ablations: steering is the key, weighting is the bonus

Quality steering	Quality-weighted action supervision	Clean	Rand.	Average
None	no	73.5	71.1	72.3
Extra-coarse	no	92.2	93.3	92.8
Extra-coarse	yes	93.9	93.3	93.6
Coarse	no	93.2	92.2	92.7
Coarse (ours)	yes	94.6	94.4	94.5

Table 4: RoboTwin 2.0 ablation. Extra-coarse distinguishes rollouts only by success or failure (failed = low, successful = medium, expert = high); coarse assigns one shared numeric score per policy's successful rollouts. Three findings: quality steering is essential for learning from mixed-quality rollouts (72.3 without it); even extra-coarse labels suffice, so fine-grained annotation is not required; and quality-weighted rollout action supervision adds a further gain under both annotation schemes.

Failure recovery, future prediction, and subtask switching

The appendices provide three qualitative strands of evidence. First, iterative execution and failure recovery: after an initial attempt fails due to execution uncertainty, incomplete interaction, or object displacement, the model observes the resulting environment state and generates corrective behavior that completes the task (tidying a makeup table, lint-roller cleaning, sweeping trash). This is where the world model pays off relative to a pure action policy. Second, future observation prediction: for the task "put the pens on the desk into the pen holder", predicted frames are compared against ground truth on head and left-wrist cameras; predictions preserve the temporal structure of the interaction while differing from ground truth in fine visual detail. Third, subtask switching: four multi-stage Table30-V2 tasks (press the button, water the flowers, shred paper, stamp positioning) are segmented with Qwen2.5-VL-7B, switching stage-specific language instructions during online execution while the remaining tasks keep their full-task instruction.

Figure 4: Failure recovery and iterative execution (paper Appendix C, Figure 9). Rows show tidying the makeup table, lint-roller cleaning, and sweeping trash: the first attempt drops the grasped object, cleans incompletely, or lets debris roll back out; the model then launches a second or third attempt conditioned on the observed state and succeeds.

Figure 5: Future observation prediction (paper Appendix D, Figure 10). For "put the pens on the desk into the pen holder", the top two rows are ground truth and prediction from the head camera and the bottom two rows from the left-wrist camera; predictions track arm motion and object displacement over time while fine texture differs from ground truth.

Figure 6: Subtask switching examples (paper Appendix A.4). Left: stage-1 stitched three-camera input for the watering task (pick up the can and water the plant); right: the stitched observation when the active instruction switches to stage 2 ("the watering is finished, put the watering can down"). Multiple camera views are stitched into one image, and subtask instructions come from Qwen2.5-VL-7B segmentation.

Limitations and Open Questions

First, predicted futures preserve structure but not detail (author-stated). Appendix D explicitly concedes that generated observations may differ from ground-truth images in fine visual details while preserving the temporal structure of the interaction. The world representation is therefore control-oriented: it cannot yet serve as a high-fidelity simulator for human inspection or downstream visual planning, and the benefit of consequence learning is currently readable only indirectly through action-level scores.

Second, quality annotation is a human/VLM-dependent pipeline (by the authors' own design). Episode-level annotation requires a human or VLM to score each trajectory (Table30-V2 reuses the platform operators' 1-10 ratings, with successful trajectories typically at least 8.5); the coarse scheme compresses annotation to one label per checkpoint but still needs that checkpoint's overall success rate plus manual inspection of about 0.7% of its trajectories for smoothness. A single scalar conflates completion and smoothness, and "failed trajectories get q=1" is a hard convention; if the annotation rubric drifts, both lambda(q) and the steering condition drift with it.

Third, the cost and saturation of the closed loop. Each improvement round requires roughly 100 rollouts per task on real hardware plus a full retraining of the joint model (the RoboTwin recipe uses 16 GPUs per node for tens of hours), and three Table30-V2 rounds mean three physical deployment campaigns per embodiment across four embodiments. The paper reports no saturation curve over rounds and no round-level control against cheaper data policies such as success-only rollouts or uniform replay, so how much of the 19.7-point gain comes from failure transitions versus sheer data volume rests on the indirect evidence of Table 4.

Fourth, compositional generalization and per-embodiment gaps remain. Composite-Unseen on RoboCasa365 is only 14.9% (PRTS reaches 18.8%), and on Table30-V2 the ARX5 column (55.71%) stays below VLA-DM0.5's 58.57%. Mixed-quality rollouts broaden the state-action-outcome distribution, but transfer to task compositions absent from both expert and rollout data is still limited.

Conclusion and Outlook

LoopWAM's contribution is turning deployment experience from something to be filtered into a complete supervision source. Source-dependent noise sampling makes one parallel denoising process serve both behavior learning and consequence learning; quality steering and quality-weighted supervision turn "what does this trajectory teach" into a configurable scalar rule - failures teach the world but not the actions, successes teach actions in proportion to quality, and the expert score selects behavior at deployment. Best-overall results on three benchmarks, monotone improvement across three real-robot rounds, and an ablation showing that even extra-coarse labels work make the mechanism look like a practical starting point for an embodied data flywheel rather than a one-off trick; the public code, checkpoints, and rollout datasets (including a non-NVIDIA training stack and a 70-to-40-hour dataloader optimization) lower the bar for verification. The natural next questions are whether quality annotation can be replaced by automated consequence-consistency signals, where improvement saturates over rounds, and whether the latent world representation can grow high-fidelity futures usable for planning and inspection.