P-BFM: Perceptive Behavioral Foundation Models for Humanoid Control via Unsupervised Reinforcement Learning

Paper: P-BFM: Perceptive Behavioral Foundation Models for Humanoid Control via Unsupervised Reinforcement Learning (arXiv:2610.15833v1, cs.RO, 2026-10-11)
Authors: Xue Wang, Yitang Li, Yiming Zhang (equal contribution), Siqin Wang, Junhan Sun, Zhihao Liu, Yaojie Shen, Jingbo Yuan, Hao Zhao, Qingdu Li (corresponding)
Affiliations: University of Shanghai for Science and Technology, RoboPartyLab, Tsinghua University, Zhejiang University, Harbin Institute of Technology
Code: github.com/Roboparty/UFO (released) · Project page: roboparty.github.io/P-BFM

One-Sentence Summary

P-BFM splits "what to do" from "how to do it on this terrain": unsupervised RL encodes motion references, target states, and reward functions into a terrain-free intent space, and a single perceptive policy realizes each intent differently per terrain from depth alone — the same "walk forward" intent becomes high, discrete footholds on stairs and smooth continuous contact adjustment on slopes, with no terrain-paired demonstrations at all.

Background and Motivation

Humanoids are built for environments designed for humans: living rooms, kitchens, stairwells, construction sites. Delivering on that premise requires a reusable skill repertoire plus one controller that can execute a broad range of skills. Behavioral Foundation Models (BFMs) have advanced quickly on this front — BeyondMimic combines robust motion tracking with guided diffusion to synthesize and compose motion primitives; NVIDIA's SONIC scales motion tracking with large-scale human motion data into a unified whole-body controller; and the family of methods built on forward-backward (FB) representations (Meta Motivo, BFM-Zero, UFO) let reward functions directly specify latent behaviors, so a single controller exhibits diverse whole-body behaviors, perturbation recovery, and flexible task specification.

But real-world generality is not only "many skills" — it is also perceiving and adapting to the surroundings. A long-avoided data bottleneck sits exactly here: most tracking-based BFMs depend on demonstrations, and extending them to perceptive control explodes the data space from "motions" into combinations of "terrain geometry × motion". Covering that combinatorial space with paired demonstrations is infeasible, so generalization to unseen combinations is out of reach. Prior work relieves this with terrain-aware motion augmentation, but the underlying optimization can yield suboptimal references for contact-rich behaviors; PMT reduces demonstration dependence through offline reference optimization and teacher-student training, yet separating reference synthesis from policy learning introduces discrepancies and complicates extension to contact-rich behaviors such as getting up.

P-BFM takes a different route: instead of imitating "terrain-matched correct answers", the robot interacts with complex terrains on its own and learns adaptation from interaction. The route rests on FB representations — a reward-free framework that jointly learns behavioral representations and a latent-conditioned policy, mapping task-level reward functions onto latent variables for execution. Rewards map robot states to scalars and specify intents such as "raise an arm" or "lie down", and the exploration-driven framework lets the robot acquire diverse behaviors without demonstrations for every terrain-motion combination.

Within this framework one invariant stands out: the reward-specified intent should stay consistent across terrains, while its execution adapts to terrain geometry. The paper therefore separates intent encoding from perception-driven execution — the backward representation encodes body-state features without terrain inputs, so the same reward specification yields the same latent intent everywhere, and the policy uses depth observations to realize that intent on any terrain.

Purely latent control has a complementary flaw, though: unsupervised RL supervises root motion only weakly, so latents preserve local motion patterns while losing global progress — stepping in place before an obstacle, or drifting in heading. The remedy is ERI (Explicit Root Instructions): factor global transport (heading, planar velocity) out of the behavioral intent and feed it as an explicit instruction, leaving local whole-body adaptation to the perceptive policy. The decomposition yields a capability conventional BFMs lack: one fixed intent induces different terrain-appropriate behaviors — step height, stride, posture, foothold placement, and contacts all change — without terrain-specific demonstrations.

Preliminaries

The paper formulates perceptive humanoid control as a partially observable Markov decision process $\mathcal{M}=(\mathcal{S},\mathcal{O},\mathcal{A},P,\Omega,\gamma)$, where $\mathcal{S}$, $\mathcal{O}$, $\mathcal{A}$ are state, observation, and action spaces, $P$ is transition dynamics, $\Omega$ the observation model, and $\gamma\in[0,1)$ the discount factor. The policy's observations comprise proprioception and its history $o_{t}^{p}$, visual features $h_{t}^{v}\in\mathbb{R}^{256}$ temporally aggregated from 8 depth frames, estimated root velocity $\hat{\mathbf{v}}_{t}^{\mathrm{yaw}}$, a latent $z_{t}$, and global heading and planar-velocity instructions $c_{h,t}$, $c_{v,t}$. The action $a_{t}\in\mathbb{R}^{29}$ specifies joint targets executed by a low-level PD controller:

$$a_{t}\sim\pi_{\theta}\left(\cdot\mid o_{t}^{p},h_{t}^{v},\hat{\mathbf{v}}_{t}^{\mathrm{yaw}},z_{t},c_{h,t},c_{v,t}\right)$$

The latent specifies the desired behavior, visual features guide terrain adaptation, and global instructions specify heading and velocity requirements. Training may use privileged information (true velocity, terrain information); deployment uses only onboard observations, the latent, and the instructions.

For FB representations, let $\rho$ be a reference state distribution; a forward representation $F:\mathcal{S}\times\mathcal{A}\times\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ and a backward representation $B:\mathcal{S}\rightarrow\mathbb{R}^{d}$ are learned, with each latent $z\in\mathbb{R}^{d}$ indexing a policy $\pi_{z}$. The discounted successor-state measure of $\pi_{z}$ is approximated by the low-rank factorization:

$$M^{\pi_{z}}(ds^{\prime}\mid s,a)\approx F(s,a,z)^{\top}B(s^{\prime})\,\rho(ds^{\prime})$$

For a downstream state reward $\tilde{r}$, the corresponding latent is $z_{\tilde{r}}=\mathbb{E}_{s\sim\rho}[B(s)\tilde{r}(s)]$, giving $Q_{\tilde{r}}^{\pi_{z_{\tilde{r}}}}(s,a)\approx F(s,a,z_{\tilde{r}})^{\top}z_{\tilde{r}}$, so the policy favors actions with high inner-product values. P-BFM's key usage defines rewards on body-state features: $\tilde{r}(s)=r(\chi(s))$. The reward-to-latent mapping therefore specifies behavioral intent only, while terrain-dependent execution is left entirely to the perception-conditioned policy.

Method

P-BFM's architecture has two pieces: a terrain-agnostic intent space that answers "what to perform", and a terrain-aware perceptive actor that answers "how to perform it". Pretraining learns both jointly — encoding depth into visual features for terrain perception, learning the intent space with FB representations, and training the actor with a discriminator and auxiliary critics.

Figure 1: Unified intent space. Motion references, teleoperation inputs, target states, and reward functions are mapped into one terrain-agnostic intent space; one intent is realized as different behaviors on different terrains.

Perception integration. The actor is conditioned as $a_{t}\sim\pi_{\theta}\!\left(\cdot\mid o_{t}^{p},h_{t}^{v},\operatorname{sg}(\hat{\mathbf{v}}_{t}^{\mathrm{yaw}}),z_{t},c_{h,t},c_{v,t}\right)$, where $\operatorname{sg}$ stops gradients. The velocity estimator $\hat{\mathbf{v}}_{t}^{\mathrm{yaw}}$ predicts three-dimensional root linear velocity in a yaw-aligned frame. It receives no intent latent, global instructions, or true velocity — only deployable onboard observations — so it works as-is at inference. It is trained with its own estimation loss against simulator velocity (updating the estimator and depth encoder), while policy gradients are stopped through the estimated velocity: estimation error is never laundered into policy dependence.

ERI: Explicit Root Instructions. For tracking intent, $z_{t}$ encodes behavior from a short reference window while ERI extracts explicit target root heading and planar velocity from the same reference. Let $\psi_{t}$, $\psi_{t}^{\star}$ denote current and target headings and $\mathbf{v}_{t,xy}^{\star}$ the target world-frame planar velocity. The instructions are encoded as:

$$c_{h,t}=m_{h,t}\begin{bmatrix}1-\cos(\Delta\psi_{t})\\\sin(\Delta\psi_{t})\end{bmatrix},\quad\Delta\psi_{t}=\psi_{t}^{\star}-\psi_{t}$$

$$c_{v,t}=\begin{bmatrix}m_{v,t}R(-\psi_{t})\mathbf{v}_{t,xy}^{\star}\\m_{v,t}\end{bmatrix}$$

$R(-\psi_{t})$ rotates the velocity instruction into the current heading frame, and $m_{h,t},m_{v,t}\in\{0,1\}$ mark valid targets. Zero velocity is treated as a valid stopping instruction; random or goal-conditioned latents do not automatically receive reference-velocity targets. For training, critics $Q_{H}$ and $Q_{V}$ learn from:

$$r_{H,t}=-m_{h,t}\left[1-\cos(\psi_{t+1}^{\star}-\psi_{t+1})\right]$$

$$r_{V,t}=-m_{v,t}\,\ell_{\mathrm{Huber}}\!\left(\frac{\|\mathbf{v}_{t+1,xy}^{\mathrm{sim}}-\mathbf{v}_{t,xy}^{\star}\|_{2}}{\sigma_{v}}\right)$$

The heading reward uses the next reference heading; the velocity reward compares post-action planar velocity against the current instruction. One easily missed but crucial detail: $Q_{V}$ constrains only planar velocity, so vertical motion remains free to adapt to terrain — precisely why ERI does not destroy terrain adaptation. Here $\ell_{\mathrm{Huber}}(u)=u^{2}/2$ for $0\leq u\leq1$ and $u-1/2$ otherwise, with $\sigma_{v}$ setting the velocity-error scale.

Pretraining. The backward representation is implemented as $B(s)=B_{\phi}(x)$ with $x=\chi(s)$ containing proprioceptive and privileged body-state information but no depth, terrain maps, or instructions; the forward representation $F_{\psi}(\xi,a,z)$ receives history, terrain information, and instructions through $\xi$. The FB score $Q_{\mathrm{FB}}(\xi,a,z)=F_{\psi}(\xi,a,z)^{\top}z$ ensures the same target body state or reward is encoded into the same latent regardless of terrain. Forty flat-ground LaFAN sequences provide behavior prompts and positives for the conditional discriminator $D_{\omega}(x,z)$, with no terrain-specific motion synthesis. $F$ and $B$ are trained with the FB Bellman objective plus orthonormality regularization. The actor maximizes FB value and auxiliary critics:

$$\mathcal{L}_{\pi}=-\overline{Q}_{\mathrm{FB}}-\alpha\sum_{k\in\{D,A,H,V\}}\lambda_{k}\overline{Q}_{k},\qquad\alpha=\operatorname{sg}\!\left(\mathbb{E}[|Q_{\mathrm{FB}}|]\right)$$

$\alpha$ takes the stop-gradient expectation of the FB value magnitude, auto-balancing the main objective against auxiliary terms: FB value scales drift across training, and fixed weights would either swamp the auxiliary rewards or be swamped; the normalization keeps the four components ($Q_{D}$, $Q_{A}$, $Q_{H}$, $Q_{V}$) in constant proportion at any scale.

Zero-shot inference. With all parameters frozen, P-BFM supports three intent types: tracking intent (from motion references), goal-reaching intent (from target body states), and reward intent (from reward functions). All are encoded through the frozen backward representation:

$$z_{t}=B_{\phi}\!\left(\chi(s_{t}^{\mathrm{align}})\right)\ \text{(tracking)},\quad z_{t}=B_{\phi}\!\left(\chi(s_{t}^{\mathrm{goal}})\right)\ \text{(goal)},\quad z_{t}=\frac{1}{N}\sum_{i=1}^{N}B_{\phi}\!\left(\chi(s_{i}^{\mathrm{replay}})\right)\tilde{r}(s_{i})\ \text{(reward)}$$

Reference sequences of length $T$ are encoded over a window; reward specifications normalize nonzero vectors to norm 1, matching the latent scale used in training. Teleoperation reuses the tracking interface with online retargeted body-state targets. At deployment the policy executes its mean action and uses ERI instructions whenever valid.

The diagram below traces the full data flow: during pretraining, FB representations learn the intent space in a terrain-free body-state space while the depth encoder and actor interact with simulated terrains; at inference, three task specifications are encoded into latents that converge on the same actor.

graph LR
  subgraph PRETRAIN["Pretraining (then frozen)"]
    A1["Motion refs LaFAN×40"] --> B1["Backward repr Bφ
body state only · no terrain"]
    R1["Reward fn r(χ(s))"] --> B1
    G1["Target body state"] --> B1
    B1 --> Z["Terrain-agnostic intent space z"]
    Z --> ACT["Perceptive actor πθ"]
    D1["Depth frames ×8"] --> H1["Visual features h^v (256d)"]
    H1 --> ACT
    E1["ERI instructions c_h, c_v"] --> ACT
    ACT --> FB["FB value Q_FB"]
    ACT --> QD["Discriminator reward Q_D"]
    ACT --> QA["Physical auxiliary Q_A"]
    ACT --> QH["ERI critics Q_H, Q_V"]
    FB --> LOSS["L_π = -Q̄_FB - α·Σλ_k Q̄_k"]
    QD --> LOSS
    QA --> LOSS
    QH --> LOSS
  end
  subgraph INFER["Zero-shot inference (all frozen)"]
    M1["Motion reference"] --> Z2["Encode as z"]
    G2["Target state"] --> Z2
    R2["Reward function"] --> Z2
    Z2 --> ACT2["Same perceptive actor"]
    DEP["Onboard depth D435i"] --> ACT2
    ACT2 --> PD["PD controller"]
    PD --> J["29-dim joint targets"]
  end

Experiments

Setup. P-BFM is trained with a Unitree G1 model in MJLab for 192 million environment steps on NVIDIA H200×8 GPUs (about 10 hours), with all parameters frozen at inference. Metrics include joint-angle MAE $E_{\mathrm{jpos}}$, root horizontal-velocity error $E_{\mathrm{vel}}^{\mathrm{root}}$, root-heading error $E_{\mathrm{head}}^{\mathrm{root}}$, foot sliding FS (mean displacement in cm at 50 Hz), and tracking recovery rate RR. Recovery requires heading and horizontal-velocity errors within tolerance within 5 s after perturbation removal, held continuously for at least 1 s. Evaluation covers 7 terrain types (flat, rough, ascending stairs, descending stairs, platforms, inverted pyramid slopes, box obstacles) at 10 difficulty levels each.

Comparison with general-purpose humanoid controllers. Two representative baselines are used: UFO (an FB-based unsupervised RL framework) and PMT (offline reference optimization plus terrain-aware tracking). The paper is candid that PMT's full training set is not public, so it is evaluated on its only released reference trajectory while UFO and P-BFM use a broader set of 40 LaFAN sequences. Even so, P-BFM achieves lower joint-position and heading errors, less foot sliding, and dramatically higher recovery rates:

Method	Seen motions (40 LaFAN)					Unseen motions (994 SOMA)
	$E_{\mathrm{jpos}}$ (rad)↓	$E_{\mathrm{vel}}^{\mathrm{root}}$ (m/s)↓	$E_{\mathrm{head}}^{\mathrm{root}}$ (°)↓	FS (cm)↓	RR (%)↑	$E_{\mathrm{jpos}}$ (rad)↓	$E_{\mathrm{vel}}^{\mathrm{root}}$ (m/s)↓	$E_{\mathrm{head}}^{\mathrm{root}}$ (°)↓	FS (cm)↓	RR (%)↑
UFO	0.15±0.06	0.65±0.46	49.6±29.3	0.092±0.050	3.4	0.16±0.06	0.37±0.27	28.9±23.9	0.066±0.047	16.6
PMT	0.21±0.04	0.20±0.08	8.8±7.7	0.078±0.029	2.4	0.46±0.09	0.86±0.27	84.4±22.5	0.205±0.103	0.5
P-BFM	0.11±0.04	0.25±0.12	6.9±5.4	0.043±0.026	41.2	0.13±0.06	0.21±0.12	6.7±4.7	0.026±0.016	61.4

Table 1: Comparison with perception-based general-purpose humanoid controllers. PMT is evaluated on its single released trajectory, so the comparison is directional rather than strictly matched.

The starkest numbers are recovery rates: 3.4% (UFO) and 2.4% (PMT) versus 41.2% seen and 61.4% unseen for P-BFM. The authors attribute the gains to end-to-end learning through autonomous exploration — adaptation comes from direct interaction with diverse terrains rather than offline reference optimization. It is worth noting PMT's 0.20 m/s velocity error on seen motions does beat P-BFM's 0.25, a product of optimizing on its single trajectory, which underscores the evaluation-set caveat.

Terrain-adaptive behavior from a single intent. A flat-ground forward-walking reward is used for reward inference to obtain one latent, which is then evaluated across multiple terrains. The policy does not reproduce a fixed walking pattern: it autonomously changes step height, stride length, body posture, and foothold selection per perceived terrain geometry — high lifting and discrete footholds on stairs, smooth continuous contact adjustment on slopes. No terrain-specific demonstrations are provided during training; the adaptive behaviors emerge from autonomous exploration.

Figure 3: Terrain-adaptive behavior from a single forward-walking intent across terrains.

Ablation: both ERI instructions are necessary. Removing heading or velocity instructions degrades the corresponding metrics substantially:

Method	Seen motions					Unseen motions
	$E_{\mathrm{jpos}}$ (rad)↓	$E_{\mathrm{vel}}^{\mathrm{root}}$ (m/s)↓	$E_{\mathrm{head}}^{\mathrm{root}}$ (°)↓	FS (cm)↓	RR (%)↑	$E_{\mathrm{jpos}}$ (rad)↓	$E_{\mathrm{vel}}^{\mathrm{root}}$ (m/s)↓	$E_{\mathrm{head}}^{\mathrm{root}}$ (°)↓	FS (cm)↓	RR (%)↑
w/o head	0.12±0.04	0.28±0.14	19.3±14.2	0.045±0.026	20.7	0.13±0.06	0.22±0.12	14.6±14.3	0.026±0.018	39.5
w/o vel	0.12±0.04	0.41±0.26	7.7±6.9	0.032±0.018	22.4	0.13±0.06	0.32±0.19	6.3±4.9	0.020±0.012	42.9
P-BFM	0.11±0.04	0.25±0.12	6.9±5.4	0.043±0.026	41.2	0.13±0.06	0.21±0.12	6.7±4.7	0.026±0.016	61.4

Table 2: Ablation of global instructions. Without heading instructions the robot drifts; without velocity instructions it steps in place.

A counterintuitive detail deserves attention: dropping the velocity instruction yields the lowest foot sliding in the whole table (0.032/0.020 cm) — the robot barely moves forward, so there is little to slide. The seesaw between heading and velocity errors confirms the division of labor: no heading instruction means drift; no velocity instruction means marching in place. The full model preserves reference behavior while maintaining intended heading and forward progress.

Figure 4: Qualitative ablation of the two instructions.

Real-world validation. The deployment platform is a Unitree G1 with a RealSense D435i depth camera, covering teleoperation, motion tracking, goal reaching, reward inference, and terrain-adaptive recovery from perturbations. Teleoperation is most telling: the same teleop input becomes high lifting with discrete footholds on stairs and smooth continuous contact adjustment on slopes; the robot preserves the intended whole-body behavior despite frequent body-terrain contact, and recovers from unexpected falls.

Figure 5: Real-world teleoperation with perception-driven terrain adaptation.

Figure 6: Real-world goal reaching and terrain-aware recovery.

Limitations

Author-stated limitation. The conclusion is explicit: behavioral generalization remains limited by the diversity of motion priors and terrains explored during training. Broader motion datasets and more challenging terrains — particularly those requiring complex whole-body contacts — are left to future work. In other words, the intent space's vocabulary is bounded by what training exploration covered; intents outside that space cannot be encoded no matter how strong the perceptive policy is.

Evaluation caveat. The main comparison is not strictly matched: PMT has one released trajectory while UFO and P-BFM use 40 LaFAN sequences, and the paper itself states the differing evaluation sets "preclude a strictly matched comparison". Table 1 is directional rather than definitive; PMT's lower seen-motion velocity error (0.20 vs 0.25 m/s) makes the point.

ERI's applicability boundary. ERI depends on valid-target masks $m_{h,t},m_{v,t}\in\{0,1\}$: random or purely goal-conditioned latents do not automatically receive reference-velocity targets, and zero velocity counts as a valid stopping instruction. ERI's gains therefore concentrate on reference-anchored tracking tasks; for purely exploratory intents, global-progress supervision falls back on the FB value itself.

Conclusion and Outlook

P-BFM's contribution compresses into one sentence: make "intent" a terrain-agnostic constant and leave "execution" to a terrain-aware function. Unsupervised RL with FB representations supplies the terrain-free intent space, depth perception supplies per-terrain realization, and ERI supplies the global-progress supervision that unsupervised learning lacks. Together they yield one frozen policy supporting motion tracking, goal reaching, reward inference, teleoperation, and terrain-aware recovery, transferring zero-shot to a physical G1.

The route answers the "skill repertoire × environment" combinatorial explosion without piling up demonstrations, and naturally raises follow-ups: how does the intent space preserve structure as training data scales? Is ERI's planar-velocity constraint sufficient when behaviors demand complex whole-body contact (get-up after a fall, balance loss while carrying)? And depth reliability under drastic lighting changes, transparent and reflective surfaces still needs validation on broader hardware.

Golden line: "Teach the robot what to do; let it decide how." P-BFM encodes behavioral intent as a terrain-free constant and leaves the "how" to perception — the same "walk forward" is a leg lift on stairs and a contact adjustment on a slope. The gait is not taught; the terrain grows it.