Skip to content
←Back to Papers

PAPER DEEP DIVE

Mobile ManipulationWorld-action modelsRobot navigation

UniWAM Technical Report: Unified Mobile Manipulation via Mixed-Stream World-Action Modeling and Manipulation Anchor Pose Supervision

UniWAM models navigation and manipulation as independently sampled control streams over shared video and action diffusion transformers. It introduces automatically generated Manipulation Anchor Pose supervision at the scale of 1.5 million episodes, improving manipulation-ready docking accuracy and mobile-manipulation success on MAP-Bench and 24 real-robot tasks.

Paper metadata: Wei Xue, Keliang Liu, Mingzhang Cui, Jinhua Xie, Jinjie Wei, Jianan Hou, Jingcheng Lu, Lintao Wang, Kaixiang Qiu, Yizhou Liu, Xinghai Ye, Jinghang Han, Mingcheng Li, Jie Gu, Shunli Wang, Lihua Zhang, and Dingkang Yang. Technical report submitted September 30, 2026. arXiv:2609.39388 | Project page | Code | MAP-Data | MAP-Bench. The repository is Apache-2.0 licensed; this analysis checked commit 0a83bdab.

One-Sentence Summary

UniWAM combines two independently sampled control streams over a shared world-action backbone and introduces an automatically generated Manipulation Anchor Pose that specifies both where the mobile base should stop and how it should face the object.

Why Mobile Manipulation Needs a Different Interface

A mobile manipulator does not simply alternate between two interchangeable control problems. The navigation stage needs a forward, wide-angle view of free space and distant objects. The manipulation stage needs a downward workspace view, wrist cameras, fine end-effector motion, and a stable base. These stages differ in sensors, action dimensions, temporal behavior, and the statistical shape of their labels. Packing them into one action vector creates a representation that has to decode meter-scale base motion and centimeter-scale arm motion through the same token interface.

Most unified policies either use one action expert for every degree of freedom or split the action module into behavior-specific experts. In both cases, a training sample drawn from a complete mobile-manipulation trajectory usually supervises navigation and manipulation together. This is wasteful because the arm is mostly idle while the base moves, and the base is mostly idle while the arm works. Joint supervision therefore fills training batches with counterfactual stand-still commands and prevents direct use of navigation-only or manipulation-only data.

A second issue concerns the destination. A navigation model can identify the semantic target and still stop beside it, too far away, or with the wrong heading. Earlier work uses reachability inversion, floor affordance maps, candidate-pose scoring, or preferences extracted from policy rollouts. Such representations may be robot-specific, policy-specific, or useful only at the final decision rather than along the entire approach. A mobile manipulator instead needs a terminal base pose that is tied to scene geometry, can be generated for many objects, and can supervise every intermediate observation.

UniWAM addresses both issues with one architectural choice and one data choice. The architecture treats navigation and manipulation as separate control streams that share video and action transformer weights. The data pipeline defines the Manipulation Anchor Pose, or MAP, as the target base position and heading, then synthesizes trajectories toward that target. The two ideas reinforce each other: independent streams can ingest large amounts of navigation-only data, while MAP provides those data with an explicit geometric endpoint rather than asking the model to infer stopping behavior from complete task demonstrations.

Challenges in mobile manipulation and the UniWAM approach
Figure 1: Existing unified policies either share one action representation or split only the action module. UniWAM separates the streams over shared transformers and supervises an explicit manipulation-ready base pose.

Defining the Manipulation Anchor Pose

For a target object $o_i$, the MAP is a planar base pose:

$$\mathbf{m}_i=(\mathbf{p}_i^{\mathrm{MAP}},\theta_i^{\mathrm{MAP}}),\qquad \mathbf{p}_i^{\mathrm{MAP}}\in\mathbb{R}^2,\quad \theta_i^{\mathrm{MAP}}\in\mathbb{S}^1.$$

The position specifies where the robot should stop, and the heading specifies how it should orient toward the target. The definition does not include a particular arm trajectory, policy, or robot morphology. It is a geometric interface between navigation and manipulation, which is why the authors can generate MAP labels automatically and use the same endpoint to supervise different approach starts.

The paper evaluates docking with mean position error and mean absolute wrapped heading error:

$$E_{\mathrm{pos}}=\frac{1}{N}\sum_{r=1}^{N}\left\|\widehat{\mathbf{p}}_r-\mathbf{p}_r^{\mathrm{MAP}}\right\|_2,\qquad E_{\mathrm{head}}=\frac{1}{N}\sum_{r=1}^{N}\left|\operatorname{wrap}\left(\widehat{\theta}_r-\theta_r^{\mathrm{MAP}}\right)\right|.$$

Errors are grouped into short, mid, and long initial distances below 1.5 m, between 1.5 and 3.5 m, and at least 3.5 m. This stratification exposes failures that an overall average can hide. A small heading error may be tolerable at close range, but it can create a large lateral offset during a long approach. The paper also standardizes stopping: an action chunk is near zero when at least 80% of its relative poses, including the last one, stay within 0.05 m and 0.05 rad. Three consecutive near-zero chunks terminate an episode, and the third chunk is not executed.

Generating MAP-Data from 3D Scenes

The data pipeline draws from HM3D, HM3D-Semantics, InteriorGS, and a large in-house scene corpus. Its first stage normalizes heterogeneous assets into target instances, navigable regions, and renderable observation poses. Mesh scenes pass geometry and navigability checks, while Gaussian-splatting scenes require occupancy validation and rendered-view checks. Semantic instance segmentation supplies candidate objects when instance labels are absent. A candidate survives only if its identity is semantically unique, its relevant surface is complete, collision-free base space exists near the manipulation-facing side, and feasible camera poses can observe it reliably.

The second stage uses a vision-language model to identify the front edge that faces manipulation. Let $\mathbf{p}_f$ be the ground-plane midpoint of this edge, $\mathbf{n}_f$ its unit normal toward free space, and $\mathcal{F}_{\mathrm{nav}}$ the clearance-aware navigable region. The MAP position and heading are obtained from a ray intersection with the region boundary:

$$\mathbf{p}^{\mathrm{MAP}}=\operatorname{Ray}(\mathbf{p}_f,\mathbf{n}_f)\cap\partial\mathcal{F}_{\mathrm{nav}},\qquad \theta^{\mathrm{MAP}}=\operatorname{atan2}\left((\mathbf{p}_f-\mathbf{p}^{\mathrm{MAP}})_y,(\mathbf{p}_f-\mathbf{p}^{\mathrm{MAP}})_x\right).$$

Observation poses are sampled independently across distance, bearing, and camera heading, subject to collision and visibility constraints. Because the endpoint stays fixed for a target while the start poses vary, the pipeline produces dense supervision over many approach configurations instead of a single canonical route.

The third stage connects each start pose to the MAP with a single or multi-segment cubic Bezier curve. One segment is parameterized as:

$$\mathbf{q}(s)=(1-s)^3\mathbf{c}_0+3(1-s)^2s\,\mathbf{c}_1+3(1-s)s^2\mathbf{c}_2+s^3\mathbf{c}_3,\qquad s\in[0,1].$$

If the direct curve intersects an obstacle, A* supplies a collision-free corridor and the system fits curves through that corridor. A generated trajectory is retained only if it stays inside the safe navigation mask, approaches from the annotated side, keeps the manipulation-facing boundary in the horizontal field of view, and satisfies curvature and length limits. The pipeline rejects paths with inaccurate terminal position or heading, excessive backtracking, or unsafe geometry.

Each frame stores the egocentric image, planar action, target box, and the MAP position projected into the camera:

$$\mathbf{u}_t^{\mathrm{MAP}}=\Pi\left(T_{c_t\leftarrow w}\begin{bmatrix}x_{\mathrm{MAP}}&y_{\mathrm{MAP}}&z_{\mathrm{MAP}}&1\end{bmatrix}^{\top}\right),$$

where $T_{c_t\leftarrow w}$ maps world coordinates to the current camera and $\Pi$ applies camera intrinsics and perspective division. The resulting dataset contains more than 1.5 million episodes and 7,500 hours. MAP-Bench holds out 300 targets in 109 scenes, with 35,454 training trajectories from 6,732 observation poses, 7,184,098 RGB frames at 15 Hz, and 15,000 evaluation trajectories. Initial poses are clustered in $SE(2)$, and an entire cluster stays in one split, so evaluation measures unseen starting configurations rather than near-duplicate views.

MAP-Data and MAP-Bench construction pipeline
Figure 2: The data pipeline curates targets, computes MAPs and feasible observation poses, synthesizes trajectories, and stores frame-aligned RGB, actions, boxes, and projected MAP points.

Mixed-Stream World-Action Modeling

Let $s\in\{\mathrm{nav},\mathrm{manip}\}$. At time $t$, stream $s$ receives its own observation $o_t^s$, the language instruction $\ell$, and the robot state $q_t$, then predicts an action chunk:

$$A_t^s=(a_{t,1}^s,\ldots,a_{t,H}^s)=f_{\phi}^s(o_t^s,\ell,q_t).$$

Each stream has a distinct action encoder, output head, visual interface, and set of learnable tokens. The video DiT is initialized from Wan2.2-TI2V-5B, and the action DiT is coupled to it through mixture-of-transformers attention. Crucially, both DiTs are shared across streams. The model therefore has one set of world and action parameters, while the control interfaces remain behavior-specific.

Navigation predicts relative planar pose increments $(\Delta x,\Delta y,\Delta\theta)$ in the frame at the start of the action chunk. The paper rejects base velocity $(v,\omega)$ as the target because recorded velocities cluster near zero or the platform limit, creating a strongly bimodal distribution. Accumulating velocity into pose increments produces smoother, continuous waypoint targets. The low-level controller can still convert these increments into velocity commands.

Each arm uses ten dimensions: three for position, six for rotation, and one for gripper state. A dual-arm action therefore has $(3+6+1)\times2=20$ dimensions. The six-dimensional rotation stores two columns of the rotation matrix and recovers the full matrix with Gram-Schmidt orthogonalization. Both positions and rotations are expressed in the current main camera frame, so visually similar motions have similar numerical targets across robots. A missing arm in a single-arm dataset is zero-padded and masked from the loss.

Encoded examples are packed along the batch dimension, not the sequence dimension:

$$X^{\mathrm{pack}}=\operatorname{Concat}_B\left(X^{\mathrm{manip}},X^{\mathrm{nav}}\right)\in\mathbb{R}^{(B_{\mathrm{manip}}+B_{\mathrm{nav}})\times L\times D}.$$

A navigation row can only attend to navigation tokens and a manipulation row can only attend to manipulation tokens. The two streams exchange information through shared parameter updates, not through cross-row attention. This is the central distinction from a sequence-concatenated model, where both behaviors occupy one row and can directly attend to each other.

flowchart LR
  A["Instruction and stream-specific observation"] --> R{"Active stream"}
  R -->|navigation| N["Navigation encoder"]
  R -->|manipulation| M["Manipulation encoder"]
  N --> V["Shared Video DiT"]
  M --> V
  N --> D["Shared Action DiT"]
  M --> D
  V --> F["Future video latent"]
  D --> NH["Navigation head"]
  D --> MH["Manipulation head"]
  NH --> NG["Pose increment plus target box and MAP point"]
  MH --> MG["Camera-frame end effectors plus 2D tracks"]
  NG --> P["Independent windows packed as separate rows"]
  MG --> P
  
UniWAM mixed-stream architecture and attention patterns
Figure 3: Stream-specific encoders, heads, visual inputs, and tokens share video and action DiTs. The right side shows causal action attention, clean committed prefixes, and the common camera-frame end-effector interface.

Causal Attention and Prefix-Conditioned Inference

Within one stream, an action token may attend to current visual tokens, stream tokens, itself, and earlier action tokens. Let $\mathcal{V}_t$ denote current visual tokens and $\mathcal{T}^s$ the stream tokens. The neighborhood of action token $a_{t,j}^s$ is:

$$\mathcal{N}\left(a_{t,j}^s\right)=\mathcal{V}_t\cup\mathcal{T}^s\cup\left\{a_{t,k}^s:k\leq j\right\}.$$

At deployment, the first $P$ actions of a chunk may already be committed and executing. Those actions form a clean prefix. The causal mask lets later actions condition on the prefix without allowing noisy later tokens to affect earlier ones. This aligns training with asynchronous inference and allows a new action chunk to continue an existing motion. The MAP-Bench ablation is substantial: removing the mask increases overall position and heading error from 12.5 cm and 3.24 degrees to 16.0 cm and 4.07 degrees, with the long-range position error rising from 21.0 to 29.0 cm.

Independent Sampling of Behavior Windows

Complete mobile-manipulation trajectories are divided into navigation and manipulation windows. Within each stream, a window is marked active or idle according to whether the controlled component moves. A navigation-only episode enters only the navigation stream, and a manipulation-only demonstration enters only the manipulation stream. Sampling is factorized across data source and window:

$$q(d,i)=\frac{\alpha_d}{\sum_{d'}\alpha_{d'}}\frac{w_{d,i}}{\sum_j w_{d,j}}.$$

The source weight $\alpha_d$ controls the total sampling mass assigned to a dataset, while $w_{d,i}$ controls the proportions of active, idle, navigation, and manipulation windows inside that dataset. Normalizing within a source prevents a very large dataset from dominating only because it contains more episodes. The batch builder also rejects duplicates and temporally adjacent windows, and windows from the same episode must be separated by at least $\delta$ frames. The selected navigation and manipulation examples can have different batch sizes and need not originate from the same trajectory.

Grounding Actions in the Image Plane

Each action token is extended with an auxiliary target and denoised jointly:

$$a_{t,i}^s=\left[a_{t,i}^{s,\mathrm{act}};a_{t,i}^{s,\mathrm{aux}}\right].$$

For navigation, the auxiliary target contains the target bounding box and projected MAP point. The box helps localize the object in the current view, while the point identifies the eventual stopping location. For manipulation, the auxiliary target contains the projected trajectories of both end effectors and their visibility states. These targets connect camera-frame actions to where the end effectors appear in the main image.

The manipulation labels are generated through calibration, projection, and appearance-based visibility. The geometry stage maps each arm from its native base frame into a common reference frame and then into the main camera. It projects both the sixth-joint flange and an offset gripper point. A point is geometrically valid when it is in front of the camera, inside the image, and within the manipulation interval. The visibility stage encodes frames with DINOv2, compares the projected patch with positive end-effector prototypes and negative background prototypes, and applies hysteresis plus temporal cleanup. Only visible points enter the auxiliary loss.

Ablation confirms that the two auxiliary targets mostly help their own streams. Removing navigation supervision changes real-robot MAP navigation from 15.1 cm and 4.91 degrees to 17.6 cm and 6.84 degrees and lowers mobile-manipulation success from 63.1% to 56.9%. Removing manipulation supervision leaves navigation close to the full model but reduces manipulation success from 44.3% to 39.3%. The image-plane labels therefore provide more than a visualization: they influence the learned action representation.

Flow-Matching Objective

Video latents, actions, and auxiliary targets are trained with flow matching. For a clean target $\mathbf{x}$, noise $\epsilon$, and noise level $\sigma$, the noisy sample and velocity target are:

$$\mathbf{x}_\sigma=(1-\sigma)\mathbf{x}+\sigma\epsilon,\qquad \mathbf{v}^*=\epsilon-\mathbf{x}.$$

Video and action noise levels are sampled independently. The current observation stays clean, only future video latents are supervised, and an action chunk and its auxiliary dimensions share one noise level. Committed prefix positions are replaced with clean actions and excluded from supervision. For each supervision group $g$, the per-example loss is:

$$\ell_b^g=\frac{\omega(\sigma_b)}{\left|\mathcal{I}_b^g\right|}\sum_{t\in\mathcal{I}_b^g}\frac{1}{\left|\mathcal{F}_{b,t}^g\right|}\sum_{j\in\mathcal{F}_{b,t}^g}\left(\widehat{v}_{b,t,j}-v^*_{b,t,j}\right)^2.$$

The full objective sums future-video, action, and auxiliary losses for both streams:

$$\mathcal{L}=\lambda_{\mathrm{vid}}\mathcal{L}_{\mathrm{vid}}+\sum_{s\in\{\mathrm{nav},\mathrm{manip}\}}\left(\lambda_{\mathrm{act}}^s\mathcal{L}_{\mathrm{act}}^s+\lambda_{\mathrm{aux}}^s\mathcal{L}_{\mathrm{aux}}^s\right).$$

A sample contributes only to the losses of its own stream because every group average uses valid positions and features. This is essential. Without it, zero-padded navigation samples would still alter manipulation gradients and the separation would be only nominal.

Model Scale and Inference Optimization

The video DiT uses Wan2.2-TI2V-5B. The action DiT has 30 layers and a hidden size of 1024, compared with 3072 for the video branch. Nine RGB frames are sampled every fourth frame from a 33-frame window and encoded into three temporal VAE latents. Inputs are arranged on a 384 by 320 canvas containing the main view and wrist panels. The VAE and text encoder remain frozen. Optimization uses AdamW with an initial learning rate of $10^{-5}$ and cosine decay.

UniWAM uses ten Euler denoising steps and no classifier-free guidance. The implementation compiles the mixed transformer during training, and separately compiles the VAE encoder, visual-cache prefill, and cached action denoiser during inference. Current-observation video keys and values are reused across denoising steps. With fixed shapes, reduce-overhead compilation, and CUDA graphs, the authors reduce two-stream latency on one H200 from roughly 220 ms to below 60 ms. Training throughput exceeds 600 samples per second on 16 H200 GPUs.

Repository Correspondence

The mixed-stream implementation confirms that packing occurs along the batch axis. In training/real_robot/training/src/uniwam/models/wan22/mixed_stream_action_dit.py, lines 107 to 166 enumerate active branches in manipulation-then-navigation order, concatenate their token tensors on dimension zero, and retain branch sizes for output reconstruction:

branch_inputs = (
    ("manip", manip_action, manip_timestep, manip_context, manip_context_mask),
    ("nav", nav_action, nav_timestep, nav_context, nav_context_mask),
)
for branch, action, timestep, context, context_mask in branch_inputs:
    chunk = self._branch_pre_dit(
        branch=branch,
        action=action,
        timestep=timestep,
        context=context,
        context_mask=context_mask,
    )
    chunks.append(chunk)
    branch_sizes[branch] = int(action.shape[0])

return {
    "tokens": torch.cat([chunk["tokens"] for chunk in chunks], dim=0),
    "meta": {
        "branch_sizes": branch_sizes,
        "branch_order": ("manip", "nav"),
    },
}

The camera-frame action convention is implemented in training/real_robot/deployment/common/uniwam_piper_common/camera_frame.py, lines 90 to 105. Position is the target minus the anchor, and rotation is the target orientation left-multiplied by the inverse anchor orientation:

def fixed_reference_relative_pose(
    anchor_xyz: np.ndarray,
    anchor_rot6d: np.ndarray,
    target_xyz: np.ndarray,
    target_rot6d: np.ndarray,
) -> tuple[np.ndarray, np.ndarray]:
    anchor_position = np.asarray(anchor_xyz, dtype=np.float32)
    target_position = np.asarray(target_xyz, dtype=np.float32)
    anchor_rotation = rotation_6d_to_matrix(anchor_rot6d)
    target_rotation = rotation_6d_to_matrix(target_rot6d)
    relative_xyz = target_position - anchor_position
    relative_rotation = np.matmul(
        target_rotation, np.swapaxes(anchor_rotation, -1, -2)
    )
    return relative_xyz.astype(np.float32), matrix_to_rotation_6d(relative_rotation)

Benchmark Results

All MAP-Bench methods are trained for 40,000 steps on the same 35,454 trajectories and 16 H200 GPUs. Every baseline observes only the forward RGB camera and keeps its native action interface. UniWAM-Base is the matched comparison because it uses no additional data. UniWAM-Scaled keeps the training budget but mixes in MAP-Data under a fixed ratio, isolating the contribution of explicit docking supervision.

Table 1: Closed-loop MAP-Bench docking errors
MethodShort position / headingMid position / headingLong position / headingOverall position / heading
GR00T N1.79.48 cm / 5.66°10.8 cm / 5.42°22.0 cm / 7.46°14.1 cm / 6.18°
π0.57.98 cm / 3.26°9.85 cm / 3.51°50.8 cm / 5.55°22.9 cm / 4.11°
FastWAM9.24 cm / 3.44°11.0 cm / 3.95°31.1 cm / 6.66°17.1 cm / 4.68°
UniWAM-Base7.71 cm / 2.37°8.85 cm / 3.06°21.0 cm / 4.29°12.5 cm / 3.24°
UniWAM-Scaled7.48 cm / 1.82°7.86 cm / 2.08°14.2 cm / 3.01°9.85 cm / 2.30°

UniWAM-Scaled improves over the strongest external baseline by 30.1% in position error and 44.0% in heading error. The shape of its distance curve is as important as the average. π0.5 increases from 7.98 cm at short range to 50.8 cm at long range, a factor of about 6.4, while UniWAM-Scaled increases from 7.48 to 14.2 cm, a factor of about 1.9. An explicit terminal pose counters the accumulation of heading and displacement errors over long approaches.

The fixed-budget data scaling experiment strengthens this interpretation. Using 0%, 25%, 50%, and 100% of the MAP-Data targets yields overall position errors of 12.5, 11.4, 10.3, and 9.85 cm. The long-range error drops from 21.0 to 14.2 cm, while the short-range error changes only from 7.71 to 7.48 cm. Giving the same full MAP-Data to π0.5 reduces its overall position error by only 5.7% and leaves the long-range error at 47.8 cm. The data are most useful when navigation is represented as its own stream and trained with image-plane MAP targets.

The architecture ablation separates stream layout from sampling. With joint sampling, unified action tokens reach 22.0 cm and 7.49 degrees, separate action experts reach 19.7 cm and 6.06 degrees, sequence concatenation reaches 22.1 cm and 6.78 degrees, and mixed-stream packing reaches 18.6 cm and 5.87 degrees. Mixed-stream therefore improves docking even with the same joint data. Enabling independent sampling within that architecture lowers the errors again to 17.1 cm and 5.30 degrees and raises mobile-manipulation success from 60.0% to 63.1%.

Table 2: Architecture and sampling ablation on four mobile-manipulation tasks
VariantPosition errorHeading errorSuccess rate
Unified action tokens22.0 cm7.49°46.9%
Separate action experts19.7 cm6.06°54.4%
Sequence-concatenated streams22.1 cm6.78°60.0%
Mixed-stream, joint sampling18.6 cm5.87°60.0%
Mixed-stream, independent sampling17.1 cm5.30°63.1%
Four unified mobile-manipulation architectures
Figure 5: Unified tokens, separate action experts, sequence concatenation, and the proposed mixed-stream layout. Only the last design keeps behavior-specific interfaces while allowing independent stream sampling.

Real-Robot Evaluation

The real-robot study covers 13 MAP Navigation tasks, four Mobile Manipulation tasks, and seven Manipulation tasks across AgileX Cobot Magic, AgiBot G2, dual-arm Franka Research 3, and an AgileX single-arm platform. Navigation trials start one to five meters from the reference MAP with a heading offset within 45 degrees. The same cameras, demonstrations, instructions, termination logic, and success predicates are used across methods. Each task is evaluated for 40 trials on every embodiment where it runs.

Mobile robot with separate navigation and manipulation camera views
Figure 4: A shallow wide-angle head view supports navigation, while a steep narrow view and wrist cameras support manipulation. During mobile manipulation, the navigation stream also receives wrist images.
Table 3: Category-level real-robot results over 24 tasks
MethodMAP navigation position / headingMobile manipulation position / heading / successManipulation success
π0.523.3 cm / 5.15°26.2 cm / 5.72° / 62.5%71.4%
GR00T N1.718.1 cm / 6.55°20.4 cm / 7.20° / 39.4%36.8%
UniWAM-Base15.1 cm / 4.91°17.1 cm / 5.30° / 63.1%44.3%
UniWAM-Scaled11.7 cm / 4.25°13.2 cm / 4.56° / 80.0%68.9%

UniWAM-Scaled reduces real-robot navigation position error by 35.4% and heading error by 17.5% relative to the strongest baseline. For mobile manipulation, it improves position error by 35.3%, heading error by 20.3%, and success by 17.5 percentage points. The 95% Wilson intervals for its 80.0% mobile-manipulation success and π0.5's 62.5% do not overlap. UniWAM-Base also docks more accurately than both baselines while using only the same task data, which supports an architectural contribution beyond extra data.

The standalone manipulation result is more mixed. π0.5 remains first at 71.4%, while UniWAM-Scaled reaches 68.9%. UniWAM leads on three sorting and placement tasks, but π0.5 leads on six-cup stacking, both tube-insertion tasks, and long-horizon coffee preparation by 2.5 to 12.5 points. The authors attribute this to π0.5's large-scale robot pretraining. This is a useful negative result: better docking and end-to-end mobile manipulation do not automatically imply better dexterous manipulation.

UniWAM-Sim adds a zero-shot transfer test. It is trained only on MAP-Data with no real robot, room, or target instance, then deployed to Cobot Magic and AgiBot G2. It averages 18.7 cm and 6.34 degrees, which beats π0.5's real-data position error of 23.3 cm but not its heading error of 5.15 degrees. MAP supervision therefore transfers a useful geometric stopping policy, while real-world actuation and camera dynamics still create a gap that synthetic labels alone do not close.

UniWAM executions across real robot embodiments
Figure 6: Representative navigation, mobile manipulation, and manipulation executions from one UniWAM-Scaled model across four robot embodiments.

Limitations and Open Risks

The first limitation is explicitly acknowledged by the authors. When two grasp candidates are close together, their action distributions form nearby modes. In the color-sorting failure, a pink cube and a blue cube stand side by side, and the deterministic ten-step sampler sends the gripper toward the gap between them. Adding the target object to the instruction may reduce ambiguity, but the current system has no dedicated mechanism that guarantees a committed mode. This kind of failure can be more dangerous than a random miss because it follows a coherent-looking approach trajectory.

The second author-observed limitation is residual base motion during manipulation. The navigation stream can predict near-zero but nonzero base chunks after docking, which may disturb precise placement. The evaluation ignores any predicted base chunk whose displacement stays below 0.05 m and 0.05 rad. That rule stabilizes execution, but it does not prove that the model has learned a hard stationary constraint. The threshold may need revision for lighter bases, higher-precision tasks, or different low-level controllers.

The evaluation also has boundary conditions. MAP-Bench targets a specific notion of manipulation-ready docking, and the 24 real tasks are defined by one research team with shared initial configurations. Some success-rate comparisons have overlapping intervals. The architecture and sampling conclusions are strongest when read together with docking errors, not from significance tests on every category. Independent reproduction across more teams, homes, and long-duration operations remains necessary.

MAP-Data inherits errors from scene reconstruction, instance segmentation, and the vision-language model that identifies the manipulation-facing edge. Automation is what makes 1.5 million episodes possible, but systematic mistakes can also propagate at scale. In addition, the optimized inference result depends on one H200, fixed input shapes, compiled functions, visual caching, and CUDA graphs. It should not be treated as a hardware-independent latency guarantee.

Finally, UniWAM-Scaled uses more data and compute than the matched baselines in the full system comparison. The Base variant isolates the architecture, the fixed-budget scaling experiment isolates MAP-Data, and the supervised ablations isolate auxiliary targets. None of those experiments removes the total engineering cost of large-scale mixed-data training. The practical question is not only whether the design works, but which component gives the best return under a deployment budget.

Conclusion

UniWAM's contribution lies in choosing a better boundary between sharing and separation. The video world model, action transformer, and parameter updates are shared. The visual inputs, action spaces, output heads, sampling distributions, and supervision groups remain separate. This arrangement lets navigation learn directly from large-scale MAP-Data without forcing the manipulation stream into the same sample, while still allowing one model to execute both behaviors in a packed batch.

MAP supplies the missing task definition. By turning the manipulation-ready base pose into an explicit geometric target, the pipeline can generate dense approach trajectories and image-plane labels at scale. The benchmark and real-robot results show that this supervision improves long-range docking, transfers from MAP-Data alone, and combines with independent sampling to raise end-to-end mobile-manipulation success. The remaining gaps in dexterous manipulation, target ambiguity, residual base motion, and cross-site reproducibility define the next technical frontier.

Core insight: Unified mobile manipulation should not collapse navigation and manipulation into one action space; it should let both control streams learn from one shared world model without sharing each other's sampling constraints.

Related Papers

Adaptive Whole-Body MPC with Optimal Posture Planning for Humanoid Robots under Unknown Payloads

Adaptive Whole-Body MPC with Optimal Posture Planning for Humanoid Robots under Unknown Payloads

When a humanoid lifts an unknown payload, abrupt changes in mass, center of mass, and inertia destabilize whole-body control. The paper couples centroidal MPC, adaptive fuzzy balance compensation driven by capture-point error, and triple-integrator posture-transition planning. MPC produces motion references under contact, friction, and joint constraints; fuzzy compensation allocates corrective torques to stance legs; and the planner generates smooth, feasible sit-to-stand and stand-to-sit trajectories. MuJoCo simulation and a 28-DoF humanoid validate stable lifting with 7 kg and 10 kg payloads and robust 7 kg load-carrying during walking.

Manh Cuong Nguyen, Duc Cuong Vu, Viet Anh Vu · 134 views · Oct 9, 2026
HumanoidWhole-Body Control
Oct 9, 2026
Action-Aligned Latent Planning for Versatile Humanoid Loco-Manipulation

Action-Aligned Latent Planning for Versatile Humanoid Loco-Manipulation

FIBER addresses a representation problem at the heart of versatile humanoid loco-manipulation: high-level planners need a compact abstraction, but kinematic or contact targets do not encode the state-dependent control responses that make contact-rich interaction succeed, while predicting low-level actions forces an offline planner to model high-frequency closed-loop feedback. FIBER resolves this by using the executable latent space of a frozen behavioral foundation model (BFM) as the shared planning representation. Its latent policy projection (LPP) maps each independently acquired task expert into that space by identifying latent commands whose BFM-induced actions match the expert's local control responses in physical joint-target space, regularized by a semantic term that anchors each command to the BFM backward representation of the same state. Because projection only requires the expert's actions, experts may differ in observations, objectives, and training pipelines, and are discarded once projected. Closed-loop latent rollouts from all projected skills are then aggregated to train a single perceptive flow-matching planner entirely offline, conditioned on egocentric depth, proprioception, and steering tokens derived from language or waypoints; at deployment it replans in a receding-horizon loop while the frozen BFM supplies 50 Hz state feedback. The paper further contributes AH-BFM, an adaptive-horizon BFM trained with multi-gamma forward-backward learning that keeps both short-horizon tracking fidelity and long-horizon behavioral coherence in one latent space, and successor-feature-guided behavior reuse, which composes an acquired interaction such as box carrying with native whole-body behaviors like crouching or running. In IsaacSim, across 128 variations per task, FIBER reaches 42.0% mean success versus 11.6% for projected latent replay and 3.5% for kinematic planning; on a real Unitree G1 it transfers zero-shot with 6/10 box pickup, 4/10 door opening, and 2/10 object grasping, running a 5 Hz TensorRT planner and the 50 Hz BFM on a single laptop GPU.

Yutang Lin, Guanya Shi, C. Karen Liu · 247 views · Oct 17, 2026
HumanoidMobile Manipulation
Oct 17, 2026
Systematically Exploring the Capabilities of GPT-6 Astra as Embodied Policies

Systematically Exploring the Capabilities of GPT-6 Astra as Embodied Policies

Evaluates GPT-6 Astra as an embodied policy across six domains: hybrid control reaches 48% on RoboDojo and 92% on RxR navigation, while direct in-hand control and dense locomotion references stay unreliable; token use and inference latency constrain practical control.

Galbot Team, Chen, Xuchuan, Cheng, Xiaoqian · 482 views · Sep 29, 2026
Embodied AIRobot Manipulation
Sep 29, 2026
ASENA: Self-evolving Agents for Embodied Navigation

ASENA: Self-evolving Agents for Embodied Navigation

We present ASENA, an embodied agent system that enables general-purpose coding agents to program humanoid navigation, manipulation, and interaction. Through a unified robot harness, agents read sensor data, write and execute programs, inspect recorded outcomes, repair failures, and reuse notes and executable skills while keeping their model weights fixed. We also introduce ASENA-VLN, a 4B monocular navigation policy that serves as an optional tool within this programmable system. ASENA-VLN predicts body-frame trajectories for both extended routes and short-horizon behaviors. Its shared vision-language decoder is trained on route instructions, visual question answering, and a newly curated dataset of geometry-derived atomic navigation tasks. As a standalone policy, ASENA-VLN achieves state-of-the-art success rates of 68.7% on R2R and 70.2% on RxR. When integrated with a coding agent, learned navigation improves ASENA's success rate by 11 percentage points on both agentic benchmarks and reduces execution time. Persistent workspace evolution uses simulator feedback to improve success over ten passes through recurring 100-task subsets. Success increases from 72% to 98% on R2R and from 65% to 89% on RxR. On embodied question answering, ASENA achieves state-of-the-art accuracy with fewer interaction steps. On a Unitree G1, we demonstrate search, visual inspection, spatial reasoning, synthesized gestures, and mobile manipulation combining object pickup, carrying, and handoff without a pre-built map. These demonstrations illustrate how online programming extends robot behavior beyond route following and predefined skills.

An-Chieh Cheng, Isabella Liu, Edmund Bu · 356 views · Sep 30, 2026
Vision-language navigationEmbodied AI
Sep 30, 2026

As an Amazon Associate, we earn from qualifying purchases.