Social-WM: Safety-Aware Latent World Models for Robot Social Navigation

Paper Metadata

Title: Social-WM: Safety-Aware Latent World Models for Robot Social Navigation

Authors: Zhihao Zheng, Mooi Choo Chuah (Department of Computer Science and Engineering, P.C. Rossin College of Engineering and Applied Science, Lehigh University, Bethlehem, PA, USA)

Links: arXiv:2609.40177 (v2, cs.RO, 01 Oct 2026) · arXiv HTML full text · License CC BY-NC-SA 4.0 · 9 pages, 5 figures, 6 tables

Code status: No public code. Neither the full text (including footnotes and conclusion) nor a title/author search on GitHub turns up an official repository. Reproduction therefore rests on three public components: the LeWM latent world-model backbone (arXiv 2603.19312, github.com/lucas-maes/le-wm, project page le-wm.github.io), whose planning configuration the MPC baseline in Table III reuses; the Social-HM3D / Social-MP3D splits and evaluation protocol released with Falcon (arXiv 2409.13244, github.com/Zeying-Gong/Falcon); and the frozen DINOv2-small visual encoder (facebookresearch/dinov2). Three implementation details are undisclosed and would have to be tuned from scratch: the inverse-dynamics weight $\lambda_{\mathrm{id}}$ and the remaining loss weights, the depth of the CVAE proposal head with its KL weight $\beta$, and the numeric value of the safety threshold $\tau$ together with any sensitivity analysis.

Background and Motivation

A robot operating around people has to satisfy two requirements of very different character: reach the goal, and neither collide with nor crowd the pedestrians nearby. The authors frame the difficulty as coupled dynamics — robot motion affects the people around it, while pedestrians and scene geometry constrain which robot actions can be executed safely at all. The observation model makes this harder still: the robot carries a single forward-facing RGB camera, so occlusion and a limited field of view put it in a partially observable regime where neither the map nor the human states are directly available. Following NavThinker, the paper denotes the underlying state as

$$s_{t}\triangleq\big(x_{t},\;\mathcal{M},\;\{h_{t}^{(i)}\}_{i=1}^{N}\big) \tag{1}$$

where $x_{t}$ is the robot pose, $\mathcal{M}$ the static environment, and $h_{t}^{(i)}$ the state of the $i$-th human; neither $\mathcal{M}$ nor any $h_{t}^{(i)}$ is observed. At each step the robot receives an RGB observation $I_{t}$, its own pose $x_{t}$, and a goal specification $g$, and it emits continuous body-frame actions $a_{t}=(a_{t}^{\mathrm{fwd}},a_{t}^{\mathrm{lat}},a_{t}^{\mathrm{yaw}})\in\mathbb{R}^{3}$, planning over $H$-step chunks $\mathbf{a}_{t:t+H-1}\in\mathbb{R}^{H\times 3}$.

World models are the natural mechanism for acting with foresight, yet the authors observe that existing social-navigation work stops at predicting the environment. Falcon predicts future pedestrian trajectories for socially aware policy learning; NavThinker couples an action-conditioned world model with a DD-PPO policy by predicting future scene and interaction representations. Both demonstrate the value of future prediction but leave an important question underexplored: can a world model also reason about whether a nominal action is actually realizable under the prevailing physical and social constraints? That question is this paper's entry point.

The paper therefore separates two action concepts. The nominal action $a_{t}$ is the issued command. The realizable action $\tilde{a}_{t}$ is the motion aligned with the pose change between $x_{t}$ and $x_{t+1}$, and it absorbs the physical and social constraints of the environment. In free space the two agree, $a_{t}\approx\tilde{a}_{t}$; when a command conflicts with an obstacle or a nearby pedestrian, the resulting motion is constrained or stopped outright, so $a_{t}\neq\tilde{a}_{t}$. The authors' key claim is that this discrepancy is a safety-relevant signal rather than execution noise. Data collection supports it directly: training trajectories are gathered in Habitat on Falcon's Social-HM3D training split, and a keep-out gate rejects nominal actions that would enter a pedestrian, so unsafe commands produce stopped or constrained realized motion. Such blocked events account for 18.1% of transitions, meaning the world model sees "command overruled by reality" abundantly during offline training — without ever being handed human states or the keep-out signal itself.

Fig. 1: Motivation. A nominal action that appears feasible from the current observation may become unsafe when executed. Social-WM predicts its future consequence and the corresponding realizable action before execution, letting the planner reject unsafe candidates and select a safer alternative.

A second design choice is to neither reconstruct nor model explicitly. Social-WM predicts entirely in latent space: no RGB reconstruction, no depth estimation, no explicit pedestrian trajectories or human states. This buys a small model with fast per-step planning (51 ms on an A5000), and it makes the learned dynamics and realizability model goal-independent, so one world model serves both position-goal and image-goal navigation. The goal enters only through the task cost $c_{\mathrm{task}}$ at selection time.

Preliminaries

Joint-embedding predictive world models. Social-WM builds on the compact latent world-model design of LeWM: encode observations into low-dimensional latent states and predict action-conditioned futures there, instead of generating images. LeWM training combines a one-step teacher-forced objective with a representation regularizer; Social-WM adds a multi-step open-loop objective because the planner must roll out $H$ steps, and one-step-only training lets error accumulate until the imagined later steps stop being informative. The choice of backbone is itself an experimental finding here: the representation ablation in Table VI shows that LeWM's own CLS and patch representations have negative decodability for forward motion, whereas frozen DINOv2 spatial tokens are uniformly better. Social-WM therefore keeps the LeWM predictor structure but swaps in a frozen DINOv2-small encoder.

Standard inverse dynamics and this paper's change. An inverse-dynamics (ID) model predicts the action relating two consecutive states and has long been used to learn control-relevant representations (INTACT is the reference the authors follow). The standard formulation associates an observed transition with the recorded action. In social navigation that is the wrong target, because the issued command is not necessarily the motion that occurred. This paper supervises ID with the odometry-recovered realizable action $\tilde{a}_{t}$ and, at inference, applies the same model to imagined transitions. The gap between the commanded action and the predicted realizable action then becomes the safety score of a candidate. This is the central conceptual move of the paper.

Multimodal action generation. Planning needs candidates. Rather than optimization search in action space (MPC) or iterative denoising (diffusion policy), the authors sample diverse local behaviors — proceed, turn, wait, bypass — in parallel from a lightweight conditional variational autoencoder over recent latent and action history. Table III quantifies the accuracy/latency trade-off among the three proposers, and the CVAE wins overall.

Method

Step 1: visual representation. A frozen DINOv2-small encodes each RGB observation $I_{t}$ at $224\times 224$ into spatial patch tokens. The class token is discarded, the patch grid is adaptively pooled to $M$ tokens, and a lightweight projector yields $z_{t}\in\mathbb{R}^{M\times D}$; the implementation uses $M=16$ and $D=384$. Dropping the CLS token is a deliberate, measured choice: in Table VI, DINOv2-CLS attains $R^{2}_{x}=-0.10$ and the worst goal-ranking error (Err@Min 0.756 m), while DINOv2-Patch is the only representation with positive $R^{2}$ on all pose dimensions and cuts Err@Min to 0.320 m. Social navigation needs a spatial representation that retains geometry, not a single token diluted by global semantics.

Step 2: action-conditioned future prediction. Given the latent history and the corresponding nominal actions, a six-layer transformer predictor $F_{\phi}$ produces the next latent feature:

$$\hat{z}_{t+1}=F_{\phi}\!\left(z_{t-T+1:t},a_{t-T+1:t}\right) \tag{2}$$

The history length is $T=8$. The easy-to-miss property of this equation is its asymmetry: the conditioning term is the nominal action, while the target $z_{t+1}=E(I_{t+1})$ is the future that was actually observed. A refused command is therefore paired with the real stopped or constrained future, not with a hypothetical future in which the command had been fully executed. Safety-relevant consequences are written into the dynamics with no collision labels and no human states. The prediction objective is a weighted sum of three terms:

$$\mathcal{L}_{\mathrm{pred}}=\lambda_{\mathrm{tf}}\mathcal{L}_{\mathrm{tf}}+\lambda_{\mathrm{ol}}\mathcal{L}_{\mathrm{ol}}+\lambda_{\mathrm{sig}}\mathcal{L}_{\mathrm{sig}} \tag{3}$$

namely one-step teacher forcing, multi-step open-loop rollout, and a representation regularizer. The authors single out the multi-step term as important for reducing accumulated prediction error during the planner's rollouts.

Fig. 2: Overview. Candidate action chunks are proposed and imagined through the latent world model; realizable-action inverse dynamics estimates the motion associated with each imagined transition, enabling candidate selection based on both goal progress and action realizability.

Step 3: realizable inverse dynamics. The prediction objective already exposes the world model to the true consequences of nominal actions, including blocked and refused transitions, but latent prediction alone does not state how much of a nominal action was realizable. The authors add a realizable inverse-dynamics objective. As in INTACT, a local intent $m_{t}^{\ell}=z_{t+1}-z_{t}$ is combined with the current latent state to predict the realizable action

$$\hat{\tilde{a}}_{t}=I_{\psi}\!\left(z_{t},\,m_{t}^{\ell}\right) \tag{4}$$

and the supervision is the realizable action recovered from consecutive robot poses rather than the demonstrated command:

$$\mathcal{L}_{\mathrm{id}}=-\log p_{\psi}\!\left(\tilde{a}_{t}\mid z_{t},\,m_{t}^{\ell}\right) \tag{5}$$

The distinction matters because the nominal action $a_{t}$ alone does not reveal whether it was fully executed, constrained, or refused. Predicting $\tilde{a}_{t}$ instead forces the latent transition to retain the scene information associated with safety. Combined with the prediction terms, the full training objective is

$$\mathcal{L}_{\mathrm{WM}}=\lambda_{\mathrm{tf}}\mathcal{L}_{\mathrm{tf}}+\lambda_{\mathrm{ol}}\mathcal{L}_{\mathrm{ol}}+\lambda_{\mathrm{sig}}\mathcal{L}_{\mathrm{sig}}+\lambda_{\mathrm{id}}\mathcal{L}_{\mathrm{id}} \tag{6}$$

One set of navigation experience is thus squeezed for two complementary signals: the action-conditioned world model predicts the actual future resulting from a nominal action, while inverse dynamics binds the resulting latent transition to the realizable action. As an implementation note, the ID head is randomly initialized on a pretrained trunk and trained at ten times the trunk learning rate — one of the few explicit engineering choices the paper states.

Step 4: trajectory proposal. The world model predicts consequences but does not generate actions. A lightweight generative model $G_{\theta}$, conditioned on recent latent and action history, produces $K$ plausible $H$-step action chunks:

$$\mathbf{a}^{(k)}_{t:t+H-1}=G_{\theta}\left(z_{t-T+1:t},a_{t-T+1:t},\epsilon^{(k)}\right),\quad k=1,\ldots,K \tag{7}$$

with $\epsilon^{(k)}$ supplying stochasticity; the implementation uses $K=8$ and $H=8$. The default proposer is a CVAE, chosen because its compact stochastic latent generates diverse local behaviors in parallel. The proposer is also goal-independent: its only job is to offer plausible alternatives, while goal progress and realizability are evaluated after each candidate has been imagined. The CVAE is trained with standard reconstruction and KL objectives:

$$\mathcal{L}_{\mathrm{CVAE}}=\mathbb{E}_{q(u\mid\mathbf{a},c_{t})}\left[\|\mathbf{a}-D_{\theta}(u,c_{t})\|_{1}\right]+\beta D_{\mathrm{KL}}\left(q(u\mid\mathbf{a},c_{t})\|p(u)\right) \tag{8}$$

Step 5: safety evaluation and selection. At deployment every candidate chunk is rolled through the frozen world model, giving an imagined latent trajectory $\hat{z}^{(k)}_{t+1:t+H}$. The inverse-dynamics model is then applied to each imagined transition to estimate its realizable action before execution. The discrepancy between nominal and predicted realizable motion is measured as

$$c_{\mathrm{safe}}^{(k)}=\frac{1}{H}\sum_{\tau=t}^{t+H-1}\left\|a_{\tau}^{(k)}-I_{\psi}\left(\hat{z}_{\tau}^{(k)},\hat{z}_{\tau+1}^{(k)}-\hat{z}_{\tau}^{(k)}\right)\right\| \tag{9}$$

A small residual means the imagined future is consistent with the nominal actions; a large residual means the candidate is likely to be constrained. The paper's example is concrete: if a candidate commands forward motion toward a pedestrian while the world model predicts a stopped future, inverse dynamics estimates that same stopped action and $c_{\mathrm{safe}}^{(k)}$ rises. Realizability is treated as a hard feasibility requirement rather than a soft cost, keeping only candidates whose residual falls below a threshold $\tau$:

$$\mathcal{C}^{\mathrm{safe}}_{t}=\left\{\mathbf{a}^{(k)}\mid c_{\mathrm{safe}}^{(k)} < \tau\right\} \tag{10}$$

from which the controller picks the lowest goal-conditioned cost:

$$\mathbf{a}^{\star}=\arg\min_{\mathbf{a}^{(k)}\in\mathcal{C}^{\mathrm{safe}}_{t}}c_{\mathrm{task}}^{(k)}(g) \tag{11}$$

Safety evaluation is independent of the goal representation; the goal enters only through $c_{\mathrm{task}}$. In the position-goal setting the benchmark supplies the final target position $p_{g}$, each candidate is integrated kinematically, and candidates are ranked by closest distance to the target:

$$c_{\mathrm{task}}^{(k)}=\min_{\tau}\left\|\hat{x}^{(k)}_{\tau}-p_{g}\right\| \tag{12}$$

The same formulation also accepts an intermediate position subgoal in place of the final target. The authors are explicit that this geometric cost does not account for humans or obstacles, so candidates with strong nominal progress may still be rejected by the realizability constraint — safety is carried entirely by the residual gate while the task cost only pushes toward the goal. In the image-goal setting, a goal image $I_{g}$ is encoded by the same frozen encoder and candidates are ranked by similarity between their imagined latents and the goal latent:

$$c_{\mathrm{task}}^{(k)}=-\max_{\tau\in\mathcal{T}_{k}}\cos\!\left(z_{\tau}^{(k)},z_{g}\right) \tag{13}$$

where $\mathcal{T}_{k}$ contains the rollout steps considered for comparison. Goal modality and granularity are treated as orthogonal to Social-WM itself and studied separately.

Step 6: closed-loop execution. Only the first action of the selected chunk $\mathbf{a}^{\star}$ is executed. The robot then observes the resulting RGB frame and repeats the full propose–imagine–evaluate–select cycle every step. Candidate generation and latent rollouts are evaluated in parallel, which preserves the efficiency of the compact backbone while providing multi-step foresight.

flowchart TD
  OBS["egocentric RGB I_t at 224x224"] --> ENC["frozen DINOv2-small encoder
drop CLS token, adaptive pool to M=16 tokens
lightweight projector to z_t in R^(16x384)"]
  ENC --> HIST["latent and nominal action history
z_(t-T+1..t) and a_(t-T+1..t), T=8"]
  HIST --> CVAE["CVAE proposer G_theta
K=8 action chunks, each H=8 steps
51 ms per planning step on A5000"]
  CVAE --> IMAG["frozen latent world model F_phi
6-layer transformer, multi-step open-loop rollout
imagined z_hat_(t+1..t+H) for every candidate"]
  IMAG --> ID["realizable inverse dynamics I_psi
local intent m = z_hat_(tau+1) - z_hat_tau
predicts the motion actually achievable"]
  ID --> RES["safety residual c_safe Eq.9
mean L2 norm of nominal minus realizable"]
  RES --> GATE{"c_safe below tau ?"}
  GATE -->|"yes"| SAFE["admissible set C_safe Eq.10"]
  GATE -->|"no"| REJ["candidate rejected before execution
likely blocked by pedestrian or obstacle"]
  SAFE --> COST["goal cost c_task
position Eq.12 min distance to p_g
image Eq.13 max cosine to z_g"]
  COST --> STAR["select a_star = argmin c_task Eq.11"]
  STAR --> EXEC["execute only the first action of a_star"]
  EXEC --> OBS

Fig. A: The closed-loop planning circuit of Social-WM, drawn from Sections III-C through III-F. The nominal action serves only as the world model's conditioning input and as one side of the residual comparison; what decides a candidate's fate is the realizable action that inverse dynamics infers from the imagined future. Only the first action is executed before the cycle restarts.

Fig. 3: Nominal versus realizable futures. For each example the top row shows ground-truth observations for the CVAE-proposed action chunk and the bottom row shows the future predicted by Social-WM. In the top-down view the solid purple trajectory is the proposed nominal motion and the dashed orange trajectory is the ID-predicted realizable motion; the small yellow dot marks the pedestrian, the large yellow circle its safety region, and the black circle the robot's start. When a nominal action conflicts with a pedestrian, the model predicts a constrained future and the inferred realizable motion deviates from the nominal one, exposing the unsafe candidate before execution. (The world model operates entirely in latent space, so each predicted latent is visualized by its nearest-neighbor observation from the dataset.)

Experiments

Setup. Training trajectories are collected in Habitat on Falcon's Social-HM3D training split, covering 200 scenes, 11,676 episodes, and 463,645 transitions, of which 18.1% contain a blocked nominal action. Evaluation follows NavThinker's single-robot protocol with Success Rate (SR), Success weighted by Path Length (SPL), Personal Space Compliance (PSC), and Human Collision rate (H-Coll), plus zero-shot transfer to Social-MP3D without retraining or adaptation. The model has 58M parameters: 18M for the LeWM backbone, 22M for the frozen DINOv2 encoder that replaces the trained ViT, 2.4M for the inverse-dynamics head, and 5M for the CVAE. All modules are trained with AdamW while the visual encoder stays frozen. Per-step planning on an NVIDIA A5000 costs 51 ms with the CVAE, against 126 ms for the diffusion policy and 139 ms for LeWM-style MPC.

Method	HM3D SR↑	HM3D SPL↑	HM3D PSC↑	HM3D H-Coll↓	MP3D SR↑	MP3D SPL↑	MP3D PSC↑	MP3D H-Coll↓
A* (rule-based)	44.81	43.99	90.38	54.80	45.67	44.69	91.97	54.00
ORCA (rule-based)	37.44	32.91	92.23	39.77	38.81	34.65	94.03	39.86
Habitat-official (RL)	38.99	33.53	90.37	55.48	37.00	31.76	92.03	52.33
Falcon	56.26	52.05	89.76	41.22	51.67	45.54	92.53	40.67
NavThinker	59.46	55.00	89.91	39.09	47.33	41.71	93.68	37.67
Social-WM (ours)	63.77	49.62	94.37	21.67	54.15	44.32	93.95	23.34

Table 1 (paper Table I): social navigation results on Social-HM3D and zero-shot transfer to Social-MP3D.

Reading the main result. On Social-HM3D, Social-WM attains the best SR (63.77%) and the best PSC (94.37%) while cutting H-Coll to 21.67% — a 44.6% relative reduction against NavThinker's 39.09%. That is the single most convincing number in the paper, because safety improves at the same time as task success rather than at its expense. Under zero-shot transfer to Social-MP3D, SR 54.15% and H-Coll 23.34% are again best, exceeding NavThinker's 47.33% by 6.82 points with no retraining or adaptation, which is evidence that the learned dynamics generalize across unseen environments. The visible cost is efficiency: SPL of 49.62% trails NavThinker's 55.00% and Falcon's 52.05%, and on MP3D 44.32% sits slightly below Falcon's 45.54%. This is consistent with the mechanism — the realizability gate rejects candidates that make fast nominal progress while closing in on pedestrians, so the robot slows or waits more often and path efficiency suffers. The paper does not discuss this trade-off head-on; it only notes in the limitations that the remaining failures are dominated by timeouts.

Fig. 4: Closed-loop behavior compared with Habitat-official and Falcon under pedestrian interactions. Red boxes highlight unsafe interactions with insufficient pedestrian clearance; both baselines continue goal-directed actions into pedestrian-occupied regions, whereas Social-WM slows, waits, or adjusts its action when the intended progress is not safely realizable.

How much does realizable-action learning contribute? Table IV isolates that question using the image-subgoal setting with the MPC planner for a fair comparison, since the CVAE and diffusion proposers would need retraining whenever the world-model latent space changes. The three rows form a clean causal chain. A plain latent world model gives SR 51.33% and H-Coll 44.97%. Adding the inverse-dynamics objective $\mathcal{L}_{\mathrm{id}}$ without using the residual during planning raises SR to 53.08%, lifts SPL from 26.49% to 30.16%, and lowers H-Coll to 42.12%. Using the nominal–realizable discrepancy explicitly for safety evaluation then pushes SR to 56.08% and H-Coll to 31.33%. Overall the full model reduces human collisions by 13.64 points against the latent-WM baseline. The decomposition is valuable because it shows that $\mathcal{L}_{\mathrm{id}}$ alone already improves the world model's representation of realizable futures (row one to row two), while gating on it delivers a separate and larger gain (row two to row three): the two benefits are complementary, not the same effect counted twice.

Variant	$\mathcal{L}_{\mathrm{id}}$	Safety eval.	SR↑	SPL↑	PSC↑	H-Coll↓
Latent WM	✗	✗	51.33	26.49	90.26	44.97
+ Realizable ID	✓	✗	53.08	30.16	90.34	42.12
Social-WM (full)	✓	✓	56.08	32.72	91.34	31.33

Table 2 (paper Table IV): effect of realizable-action learning on Social-HM3D, image-subgoal + MPC setting.

Does the residual really encode safety? The authors do not stop at end-to-end metrics; they run a transition-level diagnostic. For each observed transition $(z_{t},z_{t+1})$ the ID model predicts the realizable action $\hat{\tilde{a}}_{t}$, the residual $\|a_{t}-\hat{\tilde{a}}_{t}\|_{xy}$ is computed as in Eq. 9, and each transition is labeled blocked or free according to whether the nominal translation was prevented by safety constraints. The evaluation covers 22,512 transitions from 584 held-out episodes, plus an Easy subset with roughly 13% blocked interactions and a Hard subset with 36.8%, against 18.1% for the full dataset. Blocked transitions produce a mean residual of 0.97 versus 0.26 for free ones, an overall AUROC of 0.961, and similarly strong separation on Easy (0.988) and Hard (0.995). Equally important is the companion number: the ID model's MAE when predicting realizable actions on free transitions is only 0.10, so it has not degenerated into always predicting reduced motion. Without that property the residual gate would simply reject everything.

Subset	Blocked residual	Free residual	AUROC↑
All (18.1% blocked)	0.97	0.26	0.961
Easy (~13% blocked)	0.99	0.09	0.988
Hard (36.8% blocked)	0.99	0.10	0.995

Table 3 (paper Table V): nominal–realizable residual discrepancy on blocked versus free transitions (22,512 transitions, 584 held-out episodes).

Representation	$R^{2}_{x}$↑	$R^{2}_{y}$↑	$R^{2}_{\mathrm{yaw}}$↑	Err@Min↓ (m)	Regret↓ (m)
LeWM-CLS	-0.36	-0.00	0.37	0.659	0.422
LeWM-Patch	-0.41	0.15	0.58	0.740	0.503
DINOv2-CLS	-0.10	0.16	0.31	0.756	0.520
DINOv2-Patch (used)	0.12	0.29	0.42	0.320	0.083

Table 4 (paper Table VI): latent representation ablation. Pose decodability uses a lightweight regression probe that predicts the ground-truth relative motion $(\Delta x,\Delta y,\Delta\mathrm{yaw})$ from the latent features of two frames, reported as $R^{2}$ per component. Goal ranking selects one candidate frame per goal using the latent-space goal cost; Err@Min is its physical distance to the goal and Regret is the extra physical error relative to the geometrically closest candidate.

Table 4 is the easiest one to skip and the most useful for reproduction. Both LeWM-CLS and LeWM-Patch have negative $R^{2}_{x}$, meaning forward motion is essentially unreadable from those features. DINOv2-Patch is the only representation with positive $R^{2}$ on all three pose dimensions, and it improves goal ranking from 0.659 m to 0.320 m in Err@Min and from 0.422 m to 0.083 m in Regret. This explains why a mere encoder swap deserves its own ablation: the goal costs in Eq. 12 and Eq. 13 are defined directly on the latent space (one integrates kinematically, the other takes a cosine), so a representation without geometry makes the ranking select the wrong candidates, and no amount of accurate safety gating recovers from that. At the same time the absolute $R^{2}$ values remain low (0.29 at best), so this latent space is only relatively better suited to ranking; it is far from decoding pose.

Goal guidance	SR↑	SPL↑	PSC↑	H-Coll↓
Final image	42.91	27.07	92.11	26.26
Image subgoal	59.52	35.23	93.16	27.04
Final position	63.77	49.62	94.37	21.67
Position subgoal	68.65	58.72	93.28	22.09

Table 5 (paper Table II): ablation of goal settings on Social-HM3D with the world model, CVAE proposer, and safety evaluation held fixed.

Goal settings and proposer trade-offs. The gains from intermediate guidance in Table 5 are consistent: image goals improve SR from 42.91% to 59.52% (SPL 27.07% to 35.23%) and position goals from 63.77% to 68.65% (SPL 49.62% to 58.72%). Subgoals are produced by building a topological memory along the reference route, localizing the robot's current node at each step, and taking the closest node along the route as the immediate subgoal — its observation for image-subgoal guidance, its location for position-subgoal guidance. PSC and H-Coll stay nearly flat across all four rows, which the authors read as intermediate guidance improving long-horizon route progress while local safety remains the responsibility of the realizability mechanism. That orthogonality is itself strong support for the design: changing the goal modality requires no retraining of the world model. Position guidance consistently beats image guidance, with the largest gap when no subgoals are used (63.77% versus 42.91%), because matching a distant goal image from short-horizon latent rollouts is intrinsically hard.

Planner	SR↑	SPL↑	PSC↑	H-Coll↓	Latency (ms)↓
MPC (LeWM configuration)	56.08	32.72	91.34	31.33	139
CVAE (default)	59.52	35.23	93.16	27.04	51
Diffusion Policy	58.84	40.15	94.08	31.49	126

Table 6 (paper Table III): planner ablation on Social-HM3D under the image-subgoal setting, which offers moderate planning difficulty between the direct position goal and the harder final-image goal. Dynamics, safety evaluation, and goal settings are fixed.

The CVAE gives the best overall trade-off: highest SR (59.52%), lowest H-Coll (27.04%), and roughly one third of MPC's latency and two fifths of the diffusion policy's — the paper states the default planner is about $2.5\times$ faster than diffusion and $3\times$ faster than MPC. Diffusion Policy achieves better SPL (40.15%) and PSC (94.08%) at higher inference cost, while MPC is both slower and less effective. Because candidate generation and latent rollouts are cheap and parallel, a 58M-parameter model does not slow planning down, which supports the engineering route of a compact world model plus parallel multi-candidate evaluation.

Limitations and Open Problems

1. Remaining failures are dominated by timeouts, and long-horizon route selection is untouched (author-stated). The paper concedes that although human collisions drop substantially, overall success improves only modestly because many remaining failures are timeouts. The world model reasons mainly about short-horizon safety and action realizability, while long-horizon route selection remains unchanged. Fig. 5 is the representative case: a pedestrian persistently occupies the narrow passage along the nominal route, Social-WM successfully avoids unsafe motion near that pedestrian, but cannot determine whether the blockage will persist or search for an alternative route, so the episode times out. The authors note that similar failures occur whenever local decisions require a longer-term detour.

Fig. 5: Timeout failure under persistent human blockage. The gray region is navigable floor and white regions are non-navigable space; the green curve is the geodesic path to the red goal, the dark blue trajectory is the robot's path, and the colored trajectories are the full-episode motion of pedestrians 2–4. A pedestrian repeatedly occupies the narrow passage along the nominal route, so Social-WM remains in the initial room — it avoids unsafe motion near the pedestrian but lacks the long-horizon reasoning to decide whether to seek an alternative route.

2. Image goals are hard over long horizons (author-stated). A final goal image provides little guidance when its content is not yet visible, and route-sampled image subgoals become unreliable after large deviations from the nominal path. Table 5 corroborates this: the final-image setting reaches only 42.91% SR, 20.86 points below the position goal, with SPL as low as 27.07%. The authors' proposed direction is to combine the current local safety reasoning with route-level replanning, spatial memory, and adaptive subgoal selection.

3. Position goals rely on privileged information, and the task cost is blind to humans (reviewer judgment). The headline results in Table 1 assume the benchmark supplies the final target position $p_{g}$, a privileged quantity available only in simulation, and Eq. 12 is a purely geometric nearest-distance cost that the authors themselves state ignores humans and obstacles. In the position-goal setting, then, all safety is carried by the residual gate $c_{\mathrm{safe}} < \tau$, and the task side expresses no preference about social behavior at all. That division of labor explains how H-Coll gets so low, but it also shows that this method is distinct from socially aware path planning — proactively yielding a corridor or choosing a less crowded route. The "social" in the title is closer to constraint satisfaction than to social etiquette.

4. Key hyperparameters and the threshold are undisclosed, with no sensitivity analysis (reviewer judgment). Eq. 10 treats realizability as a hard constraint, so $\tau$ directly determines the size of the admissible set $\mathcal{C}^{\mathrm{safe}}_{t}$: too small and every candidate is rejected (the paper does not describe the fallback in that case), too large and the safety gate is effectively off. Table 3 puts the mean free residual at 0.26 and the mean blocked residual at 0.97, so $\tau$ clearly belongs somewhere in between, but neither the actual value nor a curve of SR/H-Coll versus $\tau$ is reported, and the loss weights including $\lambda_{\mathrm{id}}$ are likewise absent. For a paper whose central claim is that the residual is a safety signal, this is a notable evidentiary gap.

5. Simulation only, with monocular partial observability (reviewer judgment). Every result comes from Habitat's Social-HM3D and Social-MP3D; there are no real-robot experiments. Simulated pedestrians are goal-driven and follow the benchmark's behavior model, and the "command overruled" phenomenon is itself an artifact of the keep-out gate in the collection pipeline. On a real robot the sources of nominal–realizable discrepancy are far more varied — wheel slip, actuator latency, controller tracking error — and all of them land in the same residual, so they would not necessarily be interpretable as social constraints. The single forward-facing camera also means pedestrians behind the robot are absent from the observation entirely, so realizability judgments only cover constraints within the field of view.

Conclusion and Outlook

The contribution of Social-WM is not yet another larger world model but an output that world models had not been explicitly asked for: whether a command can actually be executed. Its three components interlock. Frozen DINOv2 patch tokens supply a compact latent space that retains geometry (Table 4). Action-conditioned prediction writes the consequences of overruled commands into the dynamics, with the 18.1% blocked transitions in the data acting as free supervision. Realizable inverse dynamics translates those consequences into a comparable scalar residual (Eq. 9), which at deployment becomes a pre-execution feasibility gate (Eq. 10 and Eq. 11). The Table IV ablation shows that using it during training and during planning yield independent benefits, while the 0.961 AUROC and the 0.10 MAE on free transitions in Table 3 show that the residual genuinely tracks blocked execution rather than a model that shortcuts to small actions.

The costs are equally clear: SPL trails NavThinker and Falcon, remaining failures are dominated by timeouts, and long-horizon route selection does not improve at all. That points to the directions the authors name — route-level replanning, spatial memory, adaptive subgoals — plus two more from this review: making $\tau$ adaptive to local crowd density instead of fixed, and, on real hardware, separating unrealizability caused by social constraints from unrealizability caused by control tracking error. For practitioners, the most transferable idea may be the data-collection trick: a keep-out gate makes unsafe commands produce constrained realized motion naturally in offline data, so "what is not executable" needs no manual annotation at all.