LifelongCrossNav: Persistent 3D Semantic Memory for Cross-Floor Multi-Object Navigation

Paper: LifelongCrossNav: Persistent 3D Semantic Memory for Cross-Floor Multi-Object Navigation
Authors: Zehui Li*, Zihao Sun*, Jiawei Xu, Zheqi He†, Xiaoqiang Zhang, Jing-Shu Zheng, Lu Liu, Dahui Gao, Xiuwan Chen† (* equal contribution, † corresponding)
Affiliations: Peking University; Beijing Academy of Artificial Intelligence (BAAI)
Links: arXiv:2608.07079v1 · full HTML · project page · code (MIT, branch release-v2.0)
Code status: released. The repository ships weight downloads for SED, D-FINE, MobileSAM and SegFormer-B2, the HM3D-MFMON episode bundle, the full 927-episode evaluation driver and the 288-episode cross-floor subset analyzer; this close reading is based on the release-v2.0 source snapshot.

One-sentence summary

LifelongCrossNav merges two problems that the literature had kept apart, sequential multi-object ObjectNav inside a single episode and stair-aware navigation across floors, by storing geometry and goal-independent vision-language features in one support-aware sparse 3D voxel map that later goals can query directly; on 927 three-goal episodes it lifts sequence success from 16.83% for the planar persistent-map baseline OneMap to 29.13%, and on the 288 episodes whose full goal sequence provably requires a floor transition, where OneMap scores exactly zero, it reaches 7.99%.

Background and motivation

Object-goal navigation asks an embodied agent to explore an unseen environment and stop next to an instance of a requested object category. Most of the recent progress sits in perception and exploration: open-vocabulary detectors, vision-language frontier scoring and online 3D scene graphs made it possible to search for categories never seen in training. Nearly all of that work, however, assumes two simplifications at once: a single goal per episode, and a single-floor world. Real indoor service tasks look different. A robot may fetch a cup in the kitchen, carry it upstairs to a bedside table, and then return downstairs to switch off a lamp; goals arrive as an ordered sequence in which the next query is revealed only after the current one succeeds, and the building itself contains stairs and vertically overlapping rooms.

The literature attacks these two requirements separately. Cross-floor methods such as MFNP, ASCENT and TravExplorer model stairs, floor transitions or floor-level reasoning, but they consider a single goal and terminate once it is reached, so they never need to keep queryable semantic observations for a goal that has not been announced yet. Multi-object methods such as MultiON, GOAT-Bench and OneMap retain scene information across a goal sequence, but their representations are planar or bird's-eye-view maps: height is collapsed, overlapping corridors and stairwells overwrite each other, and the staircase, the only physical connection between floors, is either absent from the map or not traversable in it. The result is a gap. Planar persistent memory does not know how to move vertically, and cross-floor methods do not preserve semantics for future goals.

The authors use the word lifelong for the within-episode regime: the environment and everything accumulated so far persist across consecutive goal subtasks, and only the goal text changes. This is weaker than lifelong learning across episodes, but it is exactly the constraint that multi-object navigation imposes. When the third goal is revealed, the robot is standing wherever the second goal happened to be, and every sofa, bed and toilet it has already seen is worth remembering rather than re-exploring. The design problem therefore has two halves. Memory must be goal-independent, because future goals are unknown, yet re-queryable by any new goal text. The map must keep height, otherwise a second-floor bedroom is indistinguishable from a first-floor living room, and it must expose executable stair connections, otherwise even perfect memory cannot produce a walkable plan.

LifelongCrossNav answers by placing geometry and semantics in the same sparse 3D voxel coordinate system. The geometric branch uses support-aware ray casting to separate floors, stairs and unsupported voids; the semantic branch lifts dense SED vision-language features onto observed surface voxels and fuses them with cumulative confidence. Because both branches share coordinates, querying historical semantics and planning a traversable path happen on one map. A unified policy then switches between four candidate classes (basic frontiers, stair frontiers, history POIs, live POIs) and four navigation modes (Basic Explore, Stair Explore, POI Navigation, Target Approach), all served by a single type-aware 3D A* planner.

The second contribution is HM3D-MFMON, a benchmark built from HM3D v0.2: 36 scenes with valid multi-floor structure and stair connectivity, 927 three-goal episodes over six object categories, and a verified Cross-Floor-Required subset of 288 episodes whose complete goal sequence cannot be finished without at least one floor transition; the first mandatory transition falls in the first, second or third subtask for 102, 101 and 85 episodes respectively. Because the start of a later subtask depends on which instance the agent itself chose for the previous goal, the authors also introduce a post-hoc stage-wise shortest-path protocol: each stage's shortest-path denominator is recomputed from that stage's actual start and never uses future goals, which keeps the evaluation causal instead of rewarding an oracle route that sacrifices the current goal to get closer to a future one.

Figure 1: the paper's conceptual illustration. The agent moves from Start to Goal 1 (TV), Goal 2 (bed) and Goal 3 (toilet) in order; colored trajectories are consecutive object-goal subtasks, including a cross-floor segment through the staircase.

Task formulation and preliminaries

Each episode is an ordered sequence of $K$ object goals $(g_{1},\ldots,g_{K})$. At any time the agent sees only the current goal $g_{k}$; $g_{k+1}$ is revealed and activated only after subtask $k$ succeeds. Per step the agent receives an egocentric RGB image $I_{t}$, a depth image $D_{t}$, the 6-DoF camera pose $T_{t}$ and the current goal text. The action space is discrete: MoveForward advances 0.25 m, TurnLeft, TurnRight, LookUp and LookDown rotate or pitch by $30^{\circ}$, and Stop ends the subtask, which counts as successful only if issued within the success threshold of a valid target instance. In HM3D-MFMON, $K=3$ and the category set is chair, bed, toilet, plant, sofa and tv_monitor; every valid instance of the current category is a legal target.

The memory lifecycle is the paper's central design decision. Within an episode, the 3D geometric map, the stair states and the goal-independent vision-language voxel features persist across goals. When a new goal arrives, only its text embedding and the query-conditioned similarity field are recomputed, while the POIs, the active path and the verification states bound to the previous goal are reinitialized; between independent episodes everything is reset. What has been seen is kept; what is being chased is cleared. That boundary is precisely what separates goal-independent memory from goal-conditioned retrieval.

Method

Support-aware 3D voxel mapping

The map is a sparse voxel set rather than a planar grid. Valid depth pixels are back-projected into the camera frame, transformed into the world frame by the 6-DoF pose and quantized into voxels. Depth-ray endpoints provide observed-surface evidence, intermediate ray samples provide free-space evidence, and only observed voxels plus locally inferred navigation states are stored, so the map grows incrementally with exploration. Unlike a planar projection, voxels keep height and therefore distinguish vertically overlapping rooms, corridors, landings and stairwells. The frozen configuration uses 0.10 m voxels, a 60 by 60 m indexed XY extent, roughly plus or minus 30 m in Z, a 0.20 m inflation radius and a 0.40 m support-search depth.

For navigation the observed space is abstracted into four functional voxel types. Occupied voxels are observed surfaces, including walls, furniture, floors and stair candidates that are not confirmed yet. Traversable voxels are ray-observed free space with reliable support below. Stair voxels are stair surfaces confirmed jointly by semantic and geometric evidence. Unsupported voxels are free space for which the downward support search fails, and they are the geometric evidence for potential downward transitions. For each ray-observed free-space voxel the system searches downward within 0.40 m for the first valid support: ordinary support yields Traversable, confirmed stair support yields stair-traversable air, and no support yields Unsupported. The released code expresses this as two support sets plus a two-phase update, SUPPORT_FLOOR = {OCCUPIED, FAKE_STAIR} and SUPPORT_STAIR = {STAIR, STAIR_EXPANDED}, computing all proposed types from one map snapshot before committing them so iteration order cannot change the result. The code also carries a seventh state, FAKE_STAIR, for rejected or reversible stair evidence: it remains available to A* at high cost but can never serve as a navigation endpoint or a frontier source.

Stair confirmation is dual-channel. SegFormer-B2 produces a pixel-level stair mask (pixel threshold 0.25, minimum component area 80 px), which is lifted into 3D with depth and pose and verified through local height variation, step trends, spatial continuity and support relations (descent confirmation 0.48, geometry threshold 0.65). More importantly, promotion is delayed: during ordinary exploration, ascent or descent evidence does not rewrite Occupied voxels into Stair. Only after the corresponding stair entrance has been reached and verified, when the controller enters Stair Explore, are accepted surface voxels promoted to STAIR, adjacent occupied cells may become STAIR_EXPANDED to close gaps in sparse stair observations, and rejected evidence is retained as FAKE_STAIR. The delay prevents uncertain stair observations collected during ordinary exploration from polluting the cross-floor planning graph.

Persistent 3D semantic memory: goal-independent features, goal-conditioned retrieval

The semantic branch uses a SED dense vision-language encoder. Each RGB frame yields a spatial feature tensor $\mathbf{F}_{t}^{\mathrm{dense}}\in\mathbb{R}^{24\times 24\times 768}$ aligned with the CLIP text-embedding space; the release code comments state the mechanism directly, ConvNeXt-L with output stride 32 turns a 768 by 768 input into 24 by 24 spatial locations, each projected to a 768-D CLIP embedding. The feature map is bilinearly upsampled to the depth resolution, and each valid depth pixel carries a local-contextual visual feature that is written only to the surface voxel its depth ray hits. Features never diffuse through free space, which avoids smearing "saw a sofa" into the entire air column above it.

When several pixels of one frame quantize into the same voxel $\mathbf{v}$, their features are aggregated with observation-quality weights:

$$\bar{\mathbf{f}}_{\mathbf{v}}^{(t)}=\frac{\sum_{p\in\mathcal{P}_{\mathbf{v}}}q_{p}\mathbf{f}_{p}}{\sum_{p\in\mathcal{P}_{\mathbf{v}}}q_{p}+\epsilon} \tag{1}$$

where $\mathcal{P}_{\mathbf{v}}$ is the pixel set assigned to the voxel, $\mathbf{f}_{p}$ the pixel feature and $q_{p}$ the observation quality. The quality itself couples depth gradients with a preferred-range prior:

$$q_{t}(u,v)=\left[1-\tanh\!\left(\gamma\left\|\nabla D_{t}(u,v)\right\|\right)\right]\cdot\exp\!\left[-\frac{(d_{0}-D_{t}(u,v))^{2}}{3\sigma_{d}^{2}}\right] \tag{2}$$

so depth discontinuities at object edges and observations far from the preferred distance $d_{0}$ receive smaller weight; the frozen values are $\gamma=0.20$, $d_{0}=1.0$ m and $\sigma_{d}=1.5$ m. Multi-view fusion across frames is a cumulative weighted average rather than an exponential moving average: with stored feature $\mathbf{F}_{\mathbf{v}}^{(t-1)}$, accumulated confidence $C_{\mathbf{v}}^{(t-1)}$ and current observation confidence $c_{\mathbf{v}}^{(t)}$,

$$\mathbf{F}_{\mathbf{v}}^{(t)}=\frac{C_{\mathbf{v}}^{(t-1)}\mathbf{F}_{\mathbf{v}}^{(t-1)}+c_{\mathbf{v}}^{(t)}\bar{\mathbf{f}}_{\mathbf{v}}^{(t)}}{C_{\mathbf{v}}^{(t-1)}+c_{\mathbf{v}}^{(t)}},\qquad C_{\mathbf{v}}^{(t)}=C_{\mathbf{v}}^{(t-1)}+c_{\mathbf{v}}^{(t)} \tag{3}$$

Fused features and confidences persist across goal subtasks within the episode. When a new goal text $q$ is activated, its normalized embedding $\mathbf{t}_{q}$ is compared with every feature-bearing voxel by cosine similarity, producing a query-conditioned 3D similarity field:

$$s_{q}(v)=\frac{\mathbf{f}_{v}^{\top}\mathbf{t}_{q}}{\|\mathbf{f}_{v}\|_{2}\,\|\mathbf{t}_{q}\|_{2}} \tag{4}$$

The similarity field depends on the query and is not part of the persistent memory; the stored voxel features do not change by a single byte when the goal switches. Spatially consistent high-response regions are grouped by 3D neighborhood clustering and the four top-ranked clusters become History POIs, each assigned a nearby reachable navigation position. Historical semantic retrieval therefore needs no fixed detector label set: any object describable in text can be looked up in memory, which is exactly the complement of the YOLO-style live detections.

Unified navigation policy: four candidate classes, four modes

The candidate set is $\mathcal{C}_{t}=\mathcal{P}_{t}^{\mathrm{live/history}}\cup\mathcal{F}_{t}^{\mathrm{trav}}\cup\mathcal{F}_{t}^{\mathrm{desc}}\cup\mathcal{F}_{t}^{\mathrm{asc}}$ with default priority POIs above basic frontiers, basic frontiers above descend frontiers, and descend above ascent. Basic Frontiers are the three planar candidate types used during ordinary exploration. A Traversable Frontier lies on the boundary between supported traversable voxels and genuine unknown space. A Descend Frontier lies near the boundary between traversable space and Unsupported or FAKE_STAIR geometry and flags a potential downward transition. An Ascent Frontier is generated from stair-semantic evidence, RGB-D step geometry and multi-frame consistency, with its navigation position kept on the current exploration plane near a potential ascending entrance. The controller first explores reachable Traversable Frontiers on the current floor and considers Descend and Ascent only when no reachable ordinary frontier remains; descent is attempted before ascent by default, although the order adapts to the most recent successful floor-transition direction.

Stair Frontiers are direction-aware advancing boundaries extracted from the currently confirmed stair-voxel cluster; their heights track the observed staircase so the agent can keep climbing or descending inside the stair structure. Live POIs come from a configurable detection interface: YOLOv7 at confidence 0.70 for the six HM3D categories covered by MS-COCO, YOLO-World outside that set, with MobileSAM masks and depth plus pose recovering the 3D location. History POIs are the retrieval results of the previous section. A selected candidate is committed for a short grace period so that incremental map updates do not cause target thrashing; commitment is a stability mechanism, not a fifth candidate class.

The four modes differ in which voxel types and which graph connectivity the planner may use. Basic Explore runs a strict 26-neighbor graph over the current traversable region and keeps the selected target on the current exploration level; it treats Descend and Ascent targets as entrance proposals and does not actively traverse unconfirmed stair bodies. POI Navigation uses the regular 26-neighbor graph and reuses previously established stair connectivity when the chosen POI lies on another floor. Stair Explore, implemented as the StairSem mode, admits confirmed stair states and stair-supported air, expands the endpoint neighborhood to all pathable endpoints inside a 1.0 m physical sphere to bridge sparse stair observations, and constrains vertical progress by the requested ascent or descent direction so unrelated platforms and neighboring stair flights are suppressed; an active stair session retains control and is not interrupted by ordinary frontiers or unrelated semantic candidates. Target Approach picks a safe observation position near a Live POI and performs local path planning with discrete view correction, refining camera pose, target mask and the stopping decision without initiating new cross-floor exploration.

All modes share one type-aware 3D A* planner. The heuristic is the 3D Euclidean distance from the current voxel to the goal, $h(n)=\sqrt{(i_{x}-g_{x})^{2}+(i_{y}-g_{y})^{2}+(i_{z}-g_{z})^{2}}$, and the transition cost is $c(n,n^{\prime})=\|n-n^{\prime}\|_{2}+p(\tau_{n^{\prime}})$ with type penalties $p$ of 0 for TRAVERSABLE and TRAVERSABLE_STAIR, 4 for STAIR, 5 for STAIR_EXPANDED and 10 for FAKE_STAIR. The penalties favor stable supported air while preserving access to confirmed stair geometry; FAKE_STAIR remains a costly fallback and is never selected as a navigation endpoint. Ordinary planning prefers non-inflated voxels, multiplies the transition cost by two when a mode explicitly permits an inflated endpoint, and may temporarily retain inflated states to let the agent escape an inflated region before resuming strict planning. The StairSem graph is deliberately permissive: it verifies the stored endpoints of each edge but, as the authors state explicitly, does not claim continuous swept-volume collision certification along every intermediate sample of a long edge. That is a pragmatic compromise with sparse stair observations, and it is one of the paper's acknowledged boundaries.

The final approach and verification closes the loop. Once stable target evidence exists, the detected mask is projected into 3D and a nearby safe observation position is selected, with an internal target-approach range of 1.30 m. During the approach the system keeps verifying the target using detection confidence, mask quality, visibility, viewing direction and 3D distance. Arriving at a History POI or a Live POI does not by itself complete the subtask: the current target observation must pass verification. On success the next goal is activated while the geometric map, the stair structure and the goal-independent semantic memory are retained; on failure the system returns to candidate selection and keeps exploring. In the released configuration the boolean enable_history_poi is exactly where the w/o H-POI ablation lives: disabling it removes only History-POI generation and selection while the persistent SED feature map, cumulative fusion, live POIs, support-aware mapping, stair perception and cross-floor planning all remain.

flowchart TD
  A[RGB-D frames + 6-DoF pose + current goal text] --> B[Support-aware 3D voxel map
Occupied Traversable Stair Unsupported]
  A --> C[SED dense encoding 24x24x768
quality-weighted aggregation Eq1 Eq2]
  C --> D[Cumulative confidence fusion Eq3
goal-independent semantic voxel memory]
  B --> E{Candidate priority
POI then traversable then descend then ascent}
  D -->|cosine retrieval with goal text Eq4
top-4 clusters as History POIs| E
  A -->|YOLOv7 or YOLO-World + MobileSAM
Live POIs| E
  E -->|reachable POI exists| F[POI Navigation
26-neighbor A* reusing stair connectivity]
  E -->|ordinary frontiers only| G[Basic Explore
traversable first then descend then ascent]
  E -->|stair entrance verified| H[Stair Explore StairSem
direction-aware Stair Frontier + 1.0 m sphere]
  F --> I[Target Approach
confidence mask visibility direction distance]
  G --> E
  H -->|multi-frame landing confirmation| E
  I -->|verification fails, blacklist POI| E
  I -->|verification succeeds, next goal| J[keep geometry and semantic memory
reset POIs path and verification states]
  J --> E

Method flowchart drawn from Section 3 and Appendices E and F. The geometric and semantic branches share one voxel coordinate system; the policy switches between four modes, and a failed verification returns to candidate selection instead of terminating.

The paper's Figure 2, system overview. Inputs on the left (RGB, depth, pose, goal text); 3D voxel mapping on the top middle row (depth plus pose, ray casting, stair perception, support-aware voxel map); persistent semantic memory in the middle row (dense feature encoding, feature lifting and fusion, 3D similarity field); the unified navigation policy on the right (candidate set, navigation modes, goal approaching).

The paper's Figure 3, basic frontier types with default priority: 1 Traversable, 2 Descend, 3 Ascent. The dashed staircase denotes unexplored cross-floor structure, and the legend shows Occupied, Traversable and explored or unexplored Stair voxels.

Benchmark and evaluation protocol

HM3D-MFMON is built on HM3D v0.2 and its semantic annotations: 36 scenes with valid multi-floor structure and stair connectivity, 927 three-goal episodes, six target categories, and a Cross-Floor-Required subset of 288 episodes whose complete goal sequence cannot be completed without at least one floor transition. The frozen evaluation settings are RGB and depth at 640 by 640, horizontal field of view 90 degrees, camera height 0.88 m, an episode action budget of 1200 steps, a success radius of 1.50 m, navmesh agent height and radius 1.50 and 0.10 m, maximum climb 0.30 m at 60 degrees; all experiments run on a single NVIDIA RTX 5090 with 32 GB under Habitat-Sim and Habitat-Lab 0.2.4.

The hard part of sequential multi-object evaluation is the denominator. In single-object ObjectNav the episode start and the target category are fixed, so the shortest geodesic distance to the nearest valid instance can be computed before navigation. In the sequential setting the start of stage $k$ depends on which instance the agent chose for goal $k-1$ and where it finished. A globally optimized route over the whole sequence would remove the ambiguity, but it uses future goals that have not been issued yet and may deliberately avoid the nearest instance of the current category simply because another instance sits closer to a future goal, violating the causal information constraint of the task. The authors therefore evaluate post-hoc: with $\mathbf{x}^{\mathrm{start}}_{i,k}$ the actual agent position when goal $k$ of episode $i$ is activated and $\mathcal{X}_{i}(g_{i,k})$ the valid navigable viewpoints of all instances of the current category,

$$d^{*}_{i,k}=\min_{\mathbf{x}\in\mathcal{X}_{i}(g_{i,k})}d_{\mathrm{geo}}\left(\mathbf{x}^{\mathrm{start}}_{i,k},\mathbf{x}\right) \tag{5}$$

where $d_{\mathrm{geo}}$ is geodesic distance on the navigation mesh. The reference depends only on the actual stage start, the current goal and its valid instances, never on future goals, so different agents are compared with shortest paths consistent with the instances their own earlier subtasks selected. Per-goal path efficiency is then

$$\mathrm{SPL}_{i,k}=\mathrm{SR}_{i,k}\frac{d^{*}_{i,k}}{\max\left(d^{*}_{i,k},\,l_{i,k}\right)} \tag{6}$$

Sequence-level metrics follow MultiON: SR, SPL, Progress Rate and Progress weighted by Path Length, where PR is the fraction of completed goals and PPL weights the successful prefix by its path efficiency. Stage-wise, the paper reports conditional and global views. Under sequential termination, subtask $k$ executes only if all preceding goals succeeded, so with $\mathrm{SR}_{i,0}=1$ the number of episodes reaching stage $k$ is $\sum_{i}\mathrm{SR}_{i,k-1}$, giving conditional success $\mathrm{SR}^{\mathrm{cond}}_{k}=\sum_{i}\mathrm{SR}_{i,k}\,/\,\sum_{i}\mathrm{SR}_{i,k-1}$ and conditional SPL $\mathrm{SPL}^{\mathrm{cond}}_{k}=\sum_{i}\mathrm{SPL}_{i,k}\,/\,\sum_{i}\mathrm{SR}_{i,k-1}$. Global metrics average over all $N$ episodes with unexecuted subtasks contributing zero and satisfy the cascade $\mathrm{SR}^{\mathrm{global}}_{k}=\mathrm{SR}^{\mathrm{global}}_{k-1}\cdot\mathrm{SR}^{\mathrm{cond}}_{k}$; for three-goal episodes $\mathrm{SR}^{\mathrm{global}}_{3}$ equals the episode-level SR. In the released code the protocol appears in eval/habitat_multi_evaluator.py as object_nav_gen.get_geodesic(pos, sim, obj, correct_start=True) followed by spl = min(1.0, best_dist / max(path_length, best_dist)), and scene loading calls recompute_cross_floor_navmesh(sim) because the bundled HM3D navmesh is too coarse for several stair and landing connections, so runtime motion and post-hoc geodesics must share one cross-floor geometry.

The paper's Figure 4, post-hoc stage-wise shortest-path evaluation. A globally optimized oracle route depends on future goals that the agent cannot know, while the post-hoc reference is recomputed from each stage's actual start.

Experiments

Multi-object navigation: planar persistent memory fails at the staircase

The primary baseline is OneMap, which maintains a persistent open-vocabulary 2D semantic map reusable across successive goals and is therefore the representative planar persistent-memory method. Over all 927 episodes LifelongCrossNav reaches sequence SR, SPL, PR, PPL of 29.13, 9.64, 48.40 and 16.70 against 16.83, 6.95, 33.69 and 14.03 for OneMap: sequence completion nearly doubles, showing that the 3D support-aware representation supports more subtasks involving vertical-space exploration. The w/o H-POI variant scores slightly higher SR and PR, 29.77 and 48.54, but clearly lower SPL and PPL, 8.46 and 14.91 versus 9.64 and 16.70, which locates the benefit of historical retrieval in path efficiency rather than in raw success.

Method	SR ↑	SPL ↑	PR ↑	PPL ↑
OneMap (planar persistent semantic map)	16.83	6.95	33.69	14.03
w/o H-POI (history retrieval off)	29.77	8.46	48.54	14.91
LifelongCrossNav	29.13	9.64	48.40	16.70

Table 1: the paper's Table 1, sequential multi-object results on all 927 HM3D-MFMON episodes, in percent.

The Cross-Floor-Required subset is where the comparison becomes decisive. OneMap obtains sequence SR and SPL of exactly 0.00 (PR 18.52, PPL 7.97): it completes some preceding subtasks but never a full CFR sequence. LifelongCrossNav reaches 7.99, 2.35, 29.17 and 9.75, and w/o H-POI 7.29, 1.91, 28.94 and 9.16. The authors' explanation is direct. In CFR episodes that reach later stages, the earlier same-floor subtasks are already done, so the remaining target most likely carries the unresolved floor transition; without an explicit stair representation and cross-floor connectivity, a planar semantic map cannot offer an executable solution. The low absolute number, 7.99%, is itself the point: the subset turns cross-floor traversal from an option into a requirement.

Method	SR ↑	SPL ↑	PR ↑	PPL ↑
OneMap (planar persistent semantic map)	0.00	0.00	18.52	7.97
w/o H-POI (history retrieval off)	7.29	1.91	28.94	9.16
LifelongCrossNav	7.99	2.35	29.17	9.75

Table 2: the paper's Table 2, sequential multi-object results on the 288 Cross-Floor-Required episodes, in percent.

Stage-wise conditional metrics: later efficiency comes from semantic reuse

The conditional metrics quantify whether memory actually saves walking. On all episodes, conditional SR at stage 1 is 54.58, 70.33 and 70.33 for OneMap, w/o H-POI and LifelongCrossNav, and at stage 3 it is 56.73, 65.40 and 63.68. Conditional SPL separates the methods more cleanly: 24.81, 25.68 and 32.28 at stage 2, and 30.81, 25.56 and 34.58 at stage 3. Both persistent-memory methods become more path-efficient on later goals while w/o H-POI stays nearly flat, which attributes the late-stage efficiency gain to reuse of historical semantic observations rather than to geometric exploration alone. On the CFR subset conditional SPL is no longer monotone, because later goals may still require unseen-floor exploration and extra stair traversal, yet the complete framework stays ahead at stage 3 with 20.54 against 12.62 for w/o H-POI and 0.00 for OneMap, showing that historical retrieval remains useful after floor transitions.

Method	Cond SR k=1	k=2	k=3	Cond SPL k=1	k=2	k=3
OneMap (all 927)	54.58	54.35	56.73	23.46	24.81	30.81
w/o H-POI (all 927)	70.33	64.72	65.40	25.26	25.68	25.56
LifelongCrossNav (all 927)	70.33	65.03	63.68	25.26	32.28	34.58
OneMap (CFR 288)	42.36	31.15	0.00	17.82	16.26	0.00
w/o H-POI (CFR 288)	55.56	43.13	30.43	19.00	19.43	12.62
LifelongCrossNav (CFR 288)	55.56	43.13	33.33	19.00	23.38	20.54

Table 3: stage-wise conditional SR and conditional SPL in percent, read from the paper's Figures 5 to 8. The global counterparts in Tables 11 and 12 satisfy the cascade relation; for example on all episodes LifelongCrossNav's global SPL at stages 2 and 3 is 22.71 and 15.82 versus 18.06 and 11.64 for w/o H-POI.

The paper's Figure 5, stage-wise conditional SR on all 927 episodes. LifelongCrossNav stays above OneMap throughout; at stage 3 w/o H-POI is marginally higher, 65.40 versus 63.68, a reversal the failure analysis explains.

The paper's Figure 7, stage-wise conditional SR on the 288 CFR episodes. OneMap collapses to zero at stage 3: any CFR episode that reaches the third goal has a remaining target carrying the unresolved floor transition.

Single-object auxiliary evaluation and failure analysis

As an auxiliary check the authors evaluate on the HM3D ObjectNav validation split with 2000 single-goal episodes against twelve representative methods. LifelongCrossNav scores SR 59.7 and SPL 28.0 without any online LLM reasoning, on par with BeliefMapNav at 61.4 and 30.6 and ApexNav at 59.6 and 33.0, below ASCENT at 65.4 and 33.5 and XGX at 72.9 and 35.7, and above OneMap's 55.8 success but with lower SPL. The authors read the SR-over-SPL tradeoff as the cost of a multi-floor policy that prioritizes current-floor exploration before activating cross-floor candidates. The table's value is that the cross-floor machinery does not destroy single-object competence; it is equally fair to say the method is not a single-object state-of-the-art.

Method	Zero-shot	Multi-floor	LLM reasoning	SR ↑	SPL ↑
XGX	no	no	no	72.9	35.7
ASCENT	yes	yes	yes	65.4	33.5
BeliefMapNav	yes	no	yes	61.4	30.6
SGM	no	no	no	60.2	30.8
LifelongCrossNav	yes	yes	no	59.7	28.0
ApexNav	yes	no	yes	59.6	33.0
MFNP	yes	yes	yes	58.3	26.7
InstructNav	yes	no	yes	58.0	20.9
OneMap	yes	no	no	55.8	37.4
VLFM	yes	no	no	52.5	30.4

Table 4: excerpt of the paper's Table 3, single-object ObjectNav results on the HM3D validation split, in percent.

The failure analysis explains why w/o H-POI shows slightly higher SR and PR in Table 1. Paired episode analysis attributes the difference to a limited number of outcome reversals after History-POI retrieval changes the selected target instance or the approach viewpoint. The category-wise counts in the paper's Tables 4 and 5 show that History POIs change the composition of failures: they reduce failures from local planning, incomplete exploration and unreachable candidates, for instance plant stuck failures 59 versus 58 and fewer not-reached sofa cases, but they increase false target detections. Bed misdetection failures rise from 133 to 163, because historical POIs direct the agent toward alternative instances and approach viewpoints from which YOLOv7 occasionally confuses beds with visually similar sofas, triggering unnecessary approaches and failed verifications. The authors conclude that target verification is the main remaining source of error, and that the consistent gains in SPL, PPL and later-stage conditional SPL are the cleaner evidence for historical retrieval.

Code correspondence: where the paper's method lives in release-v2.0

The repository is a 3D restructuring on top of the OneMap codebase, and the layout is legible: mapping/voxel_map.py is the support-aware voxel map, mapping/feature_map.py handles SED feature projection and fusion, mapping/navigation3d/ holds frontier management, POI management, stair proposals and stair traversal, planning_cpp/ is the C++ planning kernel, eval/ contains the Habitat evaluators and the post-hoc metrics, and config/mon/ carries the frozen configuration. Five places map onto the paper's equations line by line.

First, the observation-quality weight of Equation 2 is two lines of tensor arithmetic in mapping/feature_map.py, and the parameter names match Table 7 of the paper exactly (gradient_factor: 0.2, optimal_object_distance: 1.0, optimal_object_factor: 1.5, which is $\sigma_{d}$):

scores = ((1 - torch.tanh(gradient_magnitude * self.gradient_factor)) *
          torch.exp(-((self.optimal_object_distance - depth) / self.optimal_object_factor) ** 2 / 3.0))

Second, cumulative confidence fusion (Equation 3) is implemented in fuse_maps in the same file as "old plus new confidence in the denominator, two weighted averages, confidences added", exactly the cumulative weighted average the paper insists on rather than an EMA (the branch is still named EMA in the code, a leftover from the OneMap naming):

confidence_denominator = confs_new + confs_old
weight_1 = torch.nan_to_num(confs_old / confidence_denominator)
weight_2 = torch.nan_to_num(confs_new / confidence_denominator)
self.feature_map[idx] = self.feature_map[idx] * weight_1.unsqueeze(-1) + values * weight_2.unsqueeze(-1)
self.confidence_map[idx] = confidence_denominator

Third, the support classification of Appendix D, Equation 13, is spelled out at the top of mapping/voxel_map.py as frozenset constants, and the single line TARGETABLE_TYPES = PATHABLE_TYPES - {FAKE_STAIR} is precisely the paper's rule that a FAKE_STAIR voxel is traversable but may not serve as an episode endpoint; the support update uses a two-phase propose-then-commit scheme so that voxel iteration order cannot change the result. Fourth, the w/o H-POI ablation is one boolean in config/mon/planning_conf.yaml, enable_history_poi: True, which agrees with the ablation boundary described in Appendix G (only History-POI generation and selection are disabled); the same file reproduces every frozen value of Table 7: objectnav_found_distance: 1.3, history_poi_objectnav_match_xy_m: 1.0, stair_sem_session_radius_m: 2.5, segformer_stair_pixel_threshold: 0.25, segformer_stair_min_area: 80, descend_pitched_conf_threshold: 0.48, descend_geometry_threshold: 0.65, object_found_min_box_area_ratio: 0.015625 (that is 1/64 of the image area) and object_found_min_mask_voxels: 10. Fifth, the post-hoc protocol appears in eval/habitat_multi_evaluator.py as get_geodesic(pos, sim, obj, correct_start=True) together with spl = min(1.0, best_dist / max(path_length, best_dist)), and scene loading calls recompute_cross_floor_navmesh(sim) with a comment explaining that the stock HM3D navmesh is too coarse at several stair and landing connections, so runtime locomotion and post-hoc geodesic measurement must share one cross-floor geometry.

One divergence is worth recording. The main text describes YOLOv7 (six categories) plus YOLO-World (open set), while the release-v2.0 configuration comments mark yolov7_confidence and using_ov as legacy fields: the current evaluation actor uses D-FINE-X (dfine_confidence: 0.8) plus SegFormer for object and stair perception, with an additional SED-based found confirmation (sed_found_threshold: 0.15). When reading the code, trust the configuration comments to separate the paper snapshot from the current release; every number in this article comes from the paper's main text and supplementary material.

Limitations

The authors state three boundaries themselves. The first is in the conclusion: real-world deployment and more robust cross-floor navigation are left to future work, so every claim here comes from Habitat simulation, stair perception leans on the ideal quality of simulated depth and pose, and sensor noise, dynamic objects and lighting change are untested. The second is in the failure analysis: target verification is the main remaining source of error, and the alternative viewpoints that History POIs steer the agent toward make YOLOv7 confuse beds with visually similar sofas (bed misdetection failures rise from 133 to 163), which shows that the loop between retrieval and detection is still not solid. The third is in Appendix F: for long StairSem edges only the stored endpoints are verified, and the paper explicitly does not claim collision certification over a continuously swept volume, so planning under sparse stair observation retains a theoretical gap.

Three further judgments are mine. First, "lifelong" here means persistent within an episode: Appendix B states that all state is reset between independent episodes, so the method accumulates no memory across episodes, and the naming is one notch weaker than cross-episode lifelong learning, which readers should not extrapolate. Second, the post-hoc protocol is causally correct but its denominators depend on each agent's own earlier choices, so cross-method comparison measures efficiency under each method's own trajectory rather than against a shared reference route; the conditional metrics mitigate this and the global metrics keep the cascade, yet neither fully removes selection bias, and an absolute 7.99% success rate on the CFR subset means the statistical power of the conclusion is bounded by 288 samples. Third, the multi-floor conservative ordering costs something on single-object tasks (SPL 28.0, below OneMap's 37.4 and ASCENT's 33.5), and the setting of six categories, three goals and one HM3D scene family is still far from open-world long-horizon tasks; the paper reports no data on how voxel memory footprint and retrieval latency scale when the goal sequence grows past ten objects.

Summary and outlook

The contribution of LifelongCrossNav compresses into one sentence: put "remembering what was seen" and "knowing how to get there" into the same support-aware 3D voxel map. The former relies on goal-independent SED/CLIP voxel features with cumulative confidence fusion, so any newly issued goal can be retrieved by cosine similarity; the latter relies on seven voxel states, delayed stair promotion and type-aware 3D A*, which turns a staircase from a semantic label into a plannable edge. HM3D-MFMON together with the post-hoc stage-wise protocol then gives the community a causally correct evaluation convention for sequential multi-object navigation. Empirically, the advantage over planar persistent-memory baselines is sharpest on the CFR subset (sequence SR 7.99 versus 0.00), and the ablation attributes the later-stage efficiency gains specifically to historical semantic retrieval. For engineering teams building embodied navigation, the reusable assets are concrete: the two-phase support update, the directional constraints of the stair session, and the practice of turning an ablation into a configuration switch are all written in the open-source code.

The outlook follows from the same list: move support-aware mapping and stair sessions onto a real robot (the authors' stated first future work), replace the fixed detector with a stronger open-vocabulary verifier to suppress bed/sofa misclassification, and study genuinely lifelong memory across episodes, because when the same building is served repeatedly the n-th task ought to be faster than the first, while the current framework rebuilds its map from scratch every time.

The value of persistent memory is not remembering more, but letting the k-th goal avoid re-walking the ground the previous k-1 goals already covered; and when that goal sits on another floor, the memory must also remember where the stairs are.