PAPER DEEP DIVE
A Replay-Constrained Simulation Framework for Personalization of Powered Knee–Ankle Prosthesis Controllers
Personalization of impedance controllers for powered prosthetic legs is critical to accommodating individual gait biomechanics but remains challenging. Existing methods rely on time-intensive human-in-the-loop exploration and/or constrain optimization to low-dimensional, single-joint parameter subspaces. Sim-to-real transfer has enabled high-dimensional locomotion control for legged robots, but in assistive device control the human partner remains un-modelable. We present a replay-constrained simulation framework: a MuJoCo-based simulator reproduces prosthetic knee-ankle dynamics while replaying recorded hip kinematics and feedback-based ground reaction forces from individual walking data, bypassing the need to model complex human neuromuscular control mechanisms. We demonstrate the framework with a deep reinforcement learning policy that personalizes phase-dependent stiffness, damping, and equilibrium angle at both joints simultaneously, maximizing a biomimicry-based reward computed solely from onboard prosthesis measurements. Experiments with three participants with transfemoral amputation during level-ground walking at 0.8 m/s demonstrate strong simulation-to-hardware predictive validity (Pearson r=0.96–$0.997$). The best-performing policy on hardware was consistently predicted within the top five simulation policies for all participants. The learned controllers improved overall biomimicry rewards by 42–59% relative to the unpersonalized baseline. The framework supports scalable high-dimensional personalization of powered prosthetic legs and is amenable to extension to higher-dimensional controller parameterizations such as neural-network controllers.
One-Sentence Summary
A replay-constrained simulation framework where MuJoCo replays recorded hip kinematics and feedback-based ground reaction forces, bypassing human neuromuscular modeling, enabling DRL to personalize phase-dependent stiffness, damping, and equilibrium angle at both knee and ankle simultaneously, improving biomimicry rewards by 42-59% with sim-to-hardware Pearson r=0.96-0.997.
Figure 1 — Replay-constrained simulation: replay individual hip kinematics and ground reaction forces, simulating prosthesis knee-ankle dynamics.
1. Background and Motivation
Approximately 2.3 million individuals with limb loss in the US, projected to double by 2060. Powered knee-ankle prostheses can improve gait biomechanics, reduce metabolic demand, and achieve net-positive joint work, but realizing these benefits depends on personalized control strategies adapted to user-specific anatomy, socket fit, alignment, and compensatory patterns.
Existing limitations: manual tuning takes ~20 minutes exposing users to suboptimal controllers; HIL-RL automates personalization but is limited to low-dimensional discrete FSM parameters (12 for knee-only); CMA-ES/Bayesian optimization face hardware iteration bottlenecks. No method enables full-dimensional, multi-joint, continuous phase-dependent impedance personalization.
2. Core Method
2.1 Replay-Constrained Simulation
Core innovation: MuJoCo simulator replays recorded hip kinematics and feedback-based ground reaction forces from individual walking data, simulating prosthesis knee-ankle dynamics, bypassing the need to model complex human neuromuscular control mechanisms.
Impedance control equation in the simulator:
$$\tau_j(\phi) = K_j(\phi) \left[\theta_{eq,j}(\phi) - \theta_j\right] + D_j(\phi) \dot{\theta}_j$$
where $\phi$ is the gait phase variable, $K_j(\phi)$, $D_j(\phi)$, $\theta_{eq,j}(\phi)$ are phase-dependent stiffness, damping, and equilibrium angle for joint $j$.
2.2 Deep RL Personalization
DRL policy simultaneously optimizes phase-dependent stiffness, damping, and equilibrium angle at both knee and ankle (high-dimensional parameter space), maximizing a biomimicry-based reward computed solely from onboard prosthesis sensor measurements.
Figure 2 — Train multiple policy parameters in simulation, select best for hardware deployment.
flowchart TB
A["Recorded Walking Data
Hip kinematics + GRF"] --> B["Replay-Constrained MuJoCo Sim"]
B --> C["Prosthesis Dynamics
Knee-Ankle"]
C --> D["DRL Policy
φ-dependent K, D, θ_eq"]
D --> E["Biomimicry Reward
(onboard sensors only)"]
E -->|train| D
D --> F["Top-5 Sim Policies"]
F --> G["Hardware Deployment
with transfemoral amputees"]
G --> H["Best Hardware Policy"]
style B fill:#e0e7ff,stroke:#2563eb
style D fill:#fef3c7,stroke:#d97706
style H fill:#dcfce7,stroke:#16a34a
3. Experimental Results
3 participants with transfemoral amputation, level-ground walking at 0.8 m/s.
| Metric | Participant 1 | Participant 2 | Participant 3 |
|---|---|---|---|
| Sim-HW Pearson r | 0.997 | 0.96 | 0.98 |
| Top-5 contains best | ✓ | ✓ | ✓ |
| Biomimicry improvement | +59% | +42% | +51% |
Best hardware policy consistently predicted within simulation Top-5, validating sim-to-hardware predictive validity.
4. Main Contributions
- Replay-constrained simulation: Replays hip kinematics and ground reaction forces, bypassing human neuromuscular modeling.
- High-dimensional multi-joint personalization: Simultaneously optimizes knee and ankle phase-dependent stiffness, damping, and equilibrium angle without dimensionality reduction.
- Onboard-sensor-only reward: Biomimicry reward computed from prosthesis onboard sensors, no external motion capture needed.
- Strong sim-to-hardware predictive validity: Pearson r=0.96-0.997, best hardware policy in sim Top-5.
5.
Joint torque
$$ K_{j}(s_{\mathrm{st}})\,\big(\theta_{j}(t)-\theta^{\mathrm{eq}}_{j}(s_{\mathrm{st}})\big) $$
Polynomial trajectory
$$ p_{i}(s_{\mathrm{st}})=\sum_{k=0}^{3}w_{i,k}\,s_{\mathrm{st}}^{k} $$
Normalized stride phase
$$ s_{\mathrm{st}}(t)=s(t)/\bar{s}_{\mathrm{TO}} $$
Weighted blending
$$ w_{i}=\alpha_{i}w_{i,\mathrm{data}}+k_{p,i}(\phi_{\mathrm{ref}}-\phi_{\mathrm{sim}}) $$
| Parameter | Value | Description |
|---|---|---|
| Learning rate | 1e-4 | Adam optimizer |
| Batch size | 256 | Training batch |
| Discount $\gamma$ | 0.99 | Reward discount |
| Epochs | 5000 | Training iterations |
Figure 2: Additional results from the paper.
Limitations and Future WorkLimitations:
- Only 3 participants, limited sample size.
- Only validated for level-ground walking, not stairs or ramps.
- Replay constraint assumes hip kinematics unchanged post-personalization, which may not be fully accurate.
Future work: Framework extensible to higher-dimensional parameterizations (e.g., neural-network controllers) and more gait modes.
6. Conclusion
The core idea is using replay-constrained simulation to bypass the human modeling challenge — MuJoCo replays individual walking data's hip kinematics and ground reaction forces, simulating prosthesis knee-ankle dynamics, enabling DRL to train personalized policies in high-dimensional parameter space. The biomimicry reward computed solely from onboard sensors ensures practicality. In 3 amputee participants, biomimicry rewards improved by 42-59%, sim-to-hardware Pearson r reached 0.96-0.997, with best hardware policy consistently predicted within simulation Top-5.
SOURCE LINKS
