KlaskTron: An Open-Source Platform for Physical Adversarial Multi-Agent RL

Authors: Aswin Ramachandran, Jona Schulz, Maurus Derungs, Carlo Angelini, Tobias Meier, and Raffaello D'Andrea.

Affiliation: Institute for Dynamic Systems and Control, ETH Zurich.

Paper: arXiv:2611.03138, November 2, 2026.

Open source: klask_hardware, klask_software, and klask_rl.

One-sentence summary

KlaskTron turns a tabletop KLASK game into a low-cost physical benchmark for adversarial multi-agent reinforcement learning, then uses a learned actuator model to transfer a PPO policy from Isaac Lab to hardware with zero-shot deployment.

Figure 1: The KlaskTron hardware. Each player has a two-axis gantry below the board, and magnetic coupling moves the striker above the playing surface.

Why build another robot game platform?

Multi-agent reinforcement learning has produced impressive behavior in digital games, where agents can collect enormous numbers of interactions and where the environment can be copied without replacing a motor. Physical multi-agent learning carries a different burden. Experiments consume hardware time, state must be estimated from imperfect sensors, contacts can be discontinuous and energetic, and a competitive policy must adapt to an opponent rather than execute a fixed trajectory.

The authors argue that the field lacks an accessible physical benchmark with the right combination of traits. Industrial robot sports can reach high speed, but their cost, space, and safety requirements keep most laboratories out. Many lower-cost tabletop systems focus on one agent and one manipulation objective. Digital MARL environments are reproducible and fast, but they do not test the hardware effects that make real competition difficult.

KLASK is chosen as the task substrate because it compresses several hard problems into a small arena. Two players control magnetic strikers beneath the table. They must attack, defend, predict rebounds, and handle the ball near goals and walls. The rules are easy to understand, episodes can reset automatically, and the platform does not require compressed air or a specialized sports hall. Each episode in the paper lasts about 31 seconds and contains an average of 12.4 dynamic contacts.

KlaskTron is therefore not just a small robot that plays a game. It is a complete experimental stack: a dual-gantry mechanism costing less than 2,500 USD, an estimated 20-hour build, CAD and bill-of-materials files, ROS 2 runtime nodes, an Isaac Lab digital twin, a PPO training pipeline, and released policy weights. The paper treats the platform itself as the primary contribution and the policy as a reference implementation.

That distinction matters when interpreting the results. The baseline is good enough to demonstrate transfer, sustained play, and a measurable competitive gap against a human. It is not presented as a new state-of-the-art MARL algorithm. The research question is whether a shared physical environment can make future algorithmic comparisons more credible, not whether one particular PPO checkpoint has solved KLASK.

Mechanical design and sensing

The hardware uses two opposing CoreXY gantries beneath a standard KLASK board. In a CoreXY mechanism, the motors remain attached to the frame while belts transmit motion to the carriage. Reducing moving mass makes rapid acceleration easier. The paper reports a mean acceleration near 6.25 m/s², enough to cross a large portion of the play area in less than a second.

Each player's striker is magnetically coupled to an under-board carriage. This preserves the feel of the commercial game and removes the need for a direct mechanical arm above the table. The authors tested stepper-motor options and observed stalls, jerk, or slip under aggressive commands. The final system uses brushless DC motors, encoders, ODrive S1 drives, and CAN communication. Measured velocity-tracking RMSE is 0.013 m/s.

The perception stack uses one overhead camera operating at 120 frames per second. The control observation stream is described as 50 Hz, so the high camera rate does not eliminate latency in detection, filtering, communication, or actuation. The policy receives a time-dependent estimate rather than an oracle state. This is intentional: high-speed play exposes delays that slower tabletop tasks may hide.

A CoreXY coordinate system is not identical to the two-dimensional striker coordinates seen by the policy. The player.cpp node first clamps the requested planar velocity and then converts it into left and right motor velocities:

// klask_hardware/ros/src/klask_motor_commander_pkg/src/player.cpp:176-213
v_x = std::clamp(v_x, -max_velocity_, max_velocity_);
v_y = std::clamp(v_y, -max_velocity_, max_velocity_);
const float meters_per_rotations = 1.0f / distance_per_revolution_;
control_r_msg.input_vel = sign * (v_x + v_y) * meters_per_rotations;
control_l_msg.input_vel = sign * (v_y - v_x) * meters_per_rotations;

This source fragment connects the policy action to the physical mechanism. The planar command is not sent to a single motor. A diagonal move requires a coordinated difference between belt axes, and the hardware layer applies its own safety clamp. A reproducibility report should therefore distinguish the raw network output, the rescaled policy action, the command accepted by the ROS node, and the actual measured striker velocity.

The hardware repository exposes a maximum striker velocity of 0.4 m/s. The runtime inference configuration includes an action-scaling boundary of 0.8 that belongs to deployment configuration, not to the paper's stated training range. Treating that scalar as the physical velocity limit, or using it to rewrite the paper's action interval, would confuse three separate levels of the system.

Figure 2: Training and deployment share the policy interface but not the source of state or execution dynamics. Simulation uses the physics engine and neural actuator, while hardware uses EKF and motor control.

flowchart LR
    A[Overhead camera] --> B[Rectified board image]
    B --> C[Adaptive EKF]
    C --> D[20-D ego observation]
    D --> E[Frozen PPO actor]
    E --> F[Planar velocity]
    F --> G[CoreXY inverse mapping]
    G --> H[ODrive motors]
    H --> I[Magnetic striker]
    I --> J[Ball and contacts]
    J --> A

    K[Isaac Lab] --> L[Simulation state]
    L --> M[PPO training actor]
    M --> N[Neural actuator model]
    N --> K
    M -. export weights .-> E
  

The diagram separates the two loops that the paper tries to align. In simulation, the environment directly supplies state, and the learned actuator changes the dynamics between an action and the next velocity. On hardware, the camera and EKF produce the state, while the physical motor and magnetic coupling produce the execution response. Zero-shot transfer means the exported actor crosses from one loop to the other without additional real-world fine-tuning.

The Isaac Lab digital twin

The simulation is implemented as a GPU-native Isaac Lab environment. It models the board, walls, ball, and two strikers with primitive geometry and uses PhysX for contacts and friction. Thousands of environments can run in parallel, which changes the economics of PPO training. The paper reports approximately 12 minutes for 20 million PPO steps on one RTX 4090.

The baseline temporarily removes the magnetic biscuit obstacles from the commercial game. This reduces the current task to a cleaner two-player attack-and-defense problem. It also creates an honest boundary for the results: the released agent has not demonstrated navigation around the non-convex obstacle layout, and reintroducing those pieces would change perception, reward design, and contact sequencing.

The simulation is not treated as a perfect model that merely needs enough random perturbations. The authors report that ideal PD actuation plus domain randomization failed to transfer. That failure motivated the neural actuator component. The digital twin is therefore a training engine plus a measurable hypothesis about the command-response gap, not a claim that the real table has been reproduced at every physical scale.

Figure 3: The KLASK environment in Isaac Lab. Two agents observe ball and striker state and emit continuous planar velocity commands.

MDP, observations, and action space

The paper formulates a game as a two-player zero-sum Markov decision process with state space, action space, reward, transition kernel, and discount factor. Each agent receives an observation centered on the table and rotated into its own ego frame. This means the same learned policy can be assigned to either side, with the right-side action negated before it is converted into the global frame.

The raw observation contains positions and velocities for the controlled striker, opponent striker, and ball, plus the two goal positions. The runtime then adds four angles and four distances. The resulting vector has 20 values. The extra geometry gives the network direct cues about shooting lanes, defensive alignment, and the relation between striker, ball, and goal.

$$ \mathbf{o}_t=\left[p_S,p_O,\dot{p}_S,\dot{p}_O,p_B,\dot{p}_B,\phi_1,\phi_2,\phi_3,\phi_4,d_1,d_2,d_3,d_4\right] $$

Here, $p_S$ and $p_O$ are the controlled and opponent striker positions, $p_B$ is the ball position, dot notation denotes velocity, $\phi_i$ are angular features, and $d_i$ are distance features. The vector is normalized with running statistics before entering the actor. The normalization is part of the policy contract and must be loaded with the checkpoint.

The software path is consistent with the paper's description. The observation mapping first centers the board, selects or mirrors the player perspective, swaps player and opponent when necessary, and negates coordinates for the right-hand view. The resulting base vector is extended with geometric features before inference.

# klask_software/.../klask_player_pkg/utils.py:39-68
if player_side == "left":
    obs = np.concatenate([left_peg_pos, left_peg_vel,
                          right_peg_pos, right_peg_vel,
                          ball_pos, ball_vel,
                          left_goal_pos, right_goal_pos])
else:
    obs = np.concatenate([-right_peg_pos, -right_peg_vel,
                          -left_peg_pos, -left_peg_vel,
                          -ball_pos, -ball_vel,
                          -right_goal_pos, -left_goal_pos])

The repository also preserves a historical feature bug. One vector intended to represent ball-to-goal direction is computed from the controlled striker position rather than the ball position, with the comment "Match the original bug!" at line 97. This does not invalidate the released policy, because the same feature convention was used during training and inference. It does mean that a cleanup commit which silently changes the geometry can break checkpoint compatibility even if the new formula looks more correct.

The action is a two-axis continuous velocity command. The paper's training range is bounded by 0.4 m/s on each axis. The policy objective is the standard discounted return, with the opponent optimizing the opposite outcome in a zero-sum game.

$$ \mathbf{a}_t=(v_x,v_y)\in[-0.4,0.4]^2\ \text{m/s} $$ $$ \pi^*=\arg\max_\pi\mathbb{E}_\pi\left[\sum_{t=0}^{\infty}\gamma^t r_t\right] $$

The action range is a compromise between speed and controllability. A larger command may produce a stronger strike, but it also increases magnetic slip, contact uncertainty, and the time needed to reverse near a boundary. The paper's later hardware analysis shows that stopping and reversing consume several centimeters, so a policy that ignores actuator history can choose a geometrically plausible action that arrives too late.

Adaptive EKF state estimation

On hardware, the policy does not receive direct physics state. A camera image is rectified, objects are detected, and an extended Kalman filter estimates position and velocity. Each tracked object uses a four-dimensional state with planar position and planar velocity.

$$ \mathbf{x}_k=[x_k,y_k,\dot{x}_k,\dot{y}_k]^T $$

A fixed process-noise model cannot serve both the quiet middle of the board and a violent collision near a wall. The authors therefore partition the table into regions and adjust the process covariance according to the predicted ball location. In open areas, the motion model receives more trust. Near boundaries or strikers, the filter receives a larger covariance and relies more heavily on new measurements.

$$ Q_{\text{ball}}=\operatorname{diag}(q_x,q_y,q_{\dot{x}},q_{\dot{y}}) $$

Figure 4 shows the region-specific matrices. A normal region uses 30 variance for each velocity component. A vertical collision region raises the relevant velocity variance to 1030, and the corner region raises both. The filter is not trying to smooth away real collisions. It changes its confidence so that a sudden post-contact trajectory can be followed quickly.

This estimator is one reason the physical result is not a direct replay of the simulation. On hardware, the state passed to the actor includes filtering delay and covariance-dependent corrections. The paper attributes part of the residual offensive miss to this uncertainty, together with magnetic coupling and gantry damping. A useful follow-up experiment would separate perception error from policy error by replaying logged states through the same actor.

Figure 4: Adaptive process-noise regions. Larger covariance near collisions makes the EKF trust new measurements more strongly.

Learned actuator dynamics

The paper's central sim-to-real claim is that command execution should be learned from the real system before adversarial policy training. An idealized PD model treats a velocity command as something the hardware can realize almost immediately. The real carriage experiences friction, torque-speed limits, magnet lag, motor transients, and a finite control period. These effects are especially visible when a striker must reverse during a fast exchange.

The authors collect scripted circular, diagonal, linear-ramp, and random-walk trajectories. Commands range up to plus or minus 0.7 m/s, beyond the 0.4 m/s policy range, so the model sees saturation and high-frequency transients. The neural actuator receives a history of ten command and velocity steps and predicts the next velocity.

$$ v_{t+1}=f_{NN}(v_t,a_{t-1},u_{t-1},\ldots) $$

The ten-step history lets the network represent latency and response shape without an explicit delay buffer. The model uses three hidden layers of 64 units. The paper reports an ablation over 1, 5, and 10 history steps and 32, 64, and 128 hidden units across eight seeds. The chosen 10-step, 3 by 64 configuration gives the lowest velocity-prediction error among the compact candidates.

Global validation RMSE is 0.013 m/s. Representative high-frequency trajectories reach peak errors of 0.032 m/s in x and 0.049 m/s in y. The average is small relative to the 0.4 m/s action limit, while the peak y error is about 12 percent of that limit. This combination explains why the policy can execute recognizable attacks while still missing contact by roughly 0.5 to 1.5 cm.

The public implementation and the paper differ in one architectural detail. The appendix lists ReLU activation for the three hidden layers, but the current actuator_network.py uses torch.nn.Softsign. The hidden width and depth remain 64 and three, and the paper's reported training results belong to the version used for that study. Anyone retraining from the current repository should record the activation choice and run a direct comparison rather than silently assuming equivalence.

Figure 5: Neural actuator validation. Solid curves are hardware responses, dashed curves are model outputs, and the shaded bands show variation across eight seeds.

The appendix also gives two different dataset sizes. Section IV-C describes 37,500 time-series samples, about 12.5 minutes at 50 Hz. Appendix C reports 69,669 samples, about 23.2 minutes. The discrepancy likely reflects different data collection or filtering stages, but the paper does not reconcile it. A reproduction should pin the dataset version and publish the exact count, split, and preprocessing used for the reported RMSE.

Validation item	Reported value	What it means for transfer
Global actuator RMSE	0.013 m/s	Average velocity tracking is close enough for recognizable play
Peak x error	0.032 m/s	Fast changes can still shift contact timing
Peak y error	0.049 m/s	About 12.25% of the 0.4 m/s action bound
Paper Section IV-C data	37,500 samples, about 12.5 minutes	One reported collection count
Paper Appendix C data	69,669 samples, about 23.2 minutes	A second reported count, not reconciled by the authors
Model history and width	10 steps, 3 x 64 units	Compact temporal model selected across eight seeds

Reward curriculum and self-play

Sparse goal rewards produced passive policies. The authors therefore start with shaping terms that teach useful interaction: move the ball toward the opponent goal, stay between the ball and the own goal, approach the ball, and generate speed. The shaping contribution decays exponentially over training, leaving the terminal game outcome to dominate later optimization.

$$ r_{\text{total}}(t)=r_{\text{term}}+\sum_i R_i\exp\left(-\frac{t}{N}\right)r_{\text{shape},i} $$

The terminal term includes goals for, goals against, and self-goal errors. The decay constant $N$ controls how quickly the agent loses the exploratory guidance. This is a curriculum over the reward function, not a fixed weighted sum throughout training. The intent is to prevent the policy from exploiting a proxy such as ball proximity after the proxy stops correlating with winning.

The offensive, defensive, and contact shaping terms make the geometry explicit. The ball should approach the opponent goal, stay away from the player's own goal, and be close enough to the controlled striker for a useful touch.

$$ r_{\text{attack}}=\exp\left(-\frac{\lVert p_B-p_O\rVert}{D}\right) $$ $$ r_{\text{defense}}=-\exp\left(-\frac{\lVert p_B-p_P\rVert}{D}\right) $$ $$ r_{\text{contact}}=\exp\left(-\frac{\lVert p_{PS}-p_B\rVert}{C}\right) $$

In the paper's round-robin comparison, the dense ball-to-goal curriculum reaches Elo around 1283, sparse rewards around 1240, and collision-only rewards around 1220. The differences are not huge, but the ordering supports the claim that useful intermediate signals improve early exploration. The Elo values are comparisons among trained policies, not evidence of human-level play.

Enhanced Self-Play supplies opponents from a historical checkpoint pool. Recent checkpoints receive more weight, with geometric decay:

$$ w_i=0.9^{(N-i-1)} $$

The pool is updated when the average score difference exceeds 0.7. The authors compare this against random sampling and training only against the best opponent. Random sampling is comparable, while best-only training is unstable because the opponent distribution lacks diversity. This result is useful for platform users: self-play strength is not monotonic in opponent severity if the learner loses exposure to different strategies.

PPO baseline and source-level correspondence

The formal paper configuration uses gamma 0.99, GAE lambda 0.95, learning rate 0.0003, PPO clip 0.2, minibatch 4096, and entropy coefficient 0.01. The inference network is a shared three-layer ELU trunk with 256, 128, and 64 units, a two-dimensional action mean, and a value head. At deployment it loads only the actor and its observation normalization buffers.

# klask_software/.../ppo/ppo_network.py:14-28
self.actor_mlp = nn.Sequential(
    nn.Linear(20, 256), nn.ELU(),
    nn.Linear(256, 128), nn.ELU(),
    nn.Linear(128, 64), nn.ELU(),
)
self.mu = nn.Linear(64, 2)
self.value = nn.Linear(64, 1)

The public training configuration does not match every number in the paper. It currently shows gamma 0.996, learning rate 0.0001, and entropy 0.0, while keeping minibatch 4096 and clip 0.2. These are legitimate implementation values for a checked-in training setup, but they are not the values reported in the paper's PPO table. The correct interpretation is version-dependent: use the paper values to reason about the published experiment, and the repository values to resume the public checkpoint or reproduce the current training recipe.

Parameter	Paper experiment	Current public configuration
Discount factor	0.99	0.996
GAE lambda	0.95	0.95
Learning rate	0.0003	0.0001
PPO clip	0.2	0.2
Minibatch size	4096	4096
Entropy coefficient	0.01	0.0
Maximum planar velocity	0.4 m/s	0.4 m/s

The runtime action path adds another layer of configuration. The network first maps its mean output to a bounded normalized action and then rescales it with the configured low and high values. The converted result is in the robot's planar frame, with right-side actions mirrored and axes reordered before publication to the hardware. A complete experiment log must save these values together with the checkpoint and normalization statistics.

The paper's mapping from component to repository is unusually direct. klask_hardware contains CAD, ROS 2 state estimation, camera handling, and CoreXY motor commands. klask_software contains the 20-dimensional observation builder and PPO inference. klask_rl contains Isaac Lab tasks, reward configuration, the actuator network, and multiple RL framework wrappers. A reader can trace a command from the action head to the motor, but only if all three revisions are recorded.

Physical experiments and human benchmark

The paper evaluates the platform in three ways: actuator-model validation, reliability over more than 30 consecutive physical games, and a ten-game match against a recreational human. Hardware failure is defined as a motor connection error, camera dropout, EKF divergence, peg decoupling, or the ball leaving the board. Software resets occur and are logged, but they do not invalidate the protocol.

The robot wins one of ten games. The authors do not present this as a strong competitive result. The policy nevertheless produces sustained attack and defense, reactive goalkeeping, and bank shots from the table geometry. The remaining error is concentrated in offensive contact. The policy often chooses a reasonable direction but the physical striker misses the ball by 0.5 to 1.5 cm.

Across the ten games, the average episode duration is 31.3 seconds. The agent records 7.4 defensive blocks, 5.0 offensive strikes, 1.8 threatening hits, and 43.6 percent possession. A total of 12.4 dynamic contacts per episode shows that the system is not merely pushing a stationary ball. The gap between five offensive strikes and fewer than two threatening hits is the more revealing statistic.

Metric	Ten-game average	Interpretation
Robot win rate	10%	The baseline wins one game and remains below the human opponent
Episode duration	31.3 s	Fast enough for repeated physical evaluation
Dynamic contacts	12.4	Evidence of repeated adversarial exchange
Defensive blocks	7.4	Reactive defense and board geometry are visible
Offensive strikes	5.0	The agent regularly reaches attacking actions
Threatening hits	1.8	Conversion from contact to threat remains weak
Possession	43.6%	Ball control is competitive but not dominant

The hardware measurements explain why small policy errors matter. The magnetic coupling lags by about 0.4 cm at direction changes. From a peak striker speed of 0.5 m/s, stopping takes 77 ms and 2 cm, while reversing to the same speed takes 111 ms and 7 cm. These are not software bugs that can be removed by a perfect controller. They are physical constraints the policy must model and anticipate.

The long-run reliability evidence is directional but limited. More than 30 consecutive games without a hardware-level failure supports the claim that the platform can sustain an evaluation session. Two students independently retraining the baseline on different days and reaching comparable play adds initial evidence of software reproducibility. Neither result is a multi-lab study or a long-term maintenance record.

Limitations and open questions

The authors explicitly remove the magnetic biscuit obstacles for the baseline. This makes the current task a cleaner adversarial arena and leaves the non-convex, multi-stage obstacle problem for future work. Reintroducing the biscuits is not a cosmetic change. It alters occlusion, contact sequences, valid shooting lanes, reward shaping, and the policy's need for longer-horizon planning.

The evaluation sample is also small. A ten-game human match can characterize the baseline and expose failure modes, but it cannot estimate broad human-relative strength. The paper's algorithm comparisons rely on simulation round robins and Elo. A stronger follow-up would vary human skill, initial conditions, random seeds, and hardware units, then report confidence intervals rather than one aggregate win rate.

A third limitation is version drift between the paper and the public code. The actuator activation, PPO hyperparameters, and dataset count do not all align. This does not make the platform non-reproducible, but it raises the standard for release engineering: each reported result needs a manifest that links the paper table to a code commit, trained checkpoint, normalization values, reward config, and exact dataset version.

The independent interpretation is that KlaskTron is strongest as a measurement instrument, not as a finished game AI. Its CoreXY hardware creates fast contact, its EKF reveals perception lag, its actuator model exposes command-response mismatch, and its self-play engine creates an opponent distribution. Those are exactly the variables an algorithm paper should control and report. A leaderboard built on this platform would be more informative if it published hardware variance, reset counts, and cross-unit transfer alongside Elo.

The paper also leaves representation learning open. The baseline relies on hand-engineered angles and distances. A future agent could learn from images or richer histories, but that would add a new perception problem and reduce direct comparability with the released baseline. The platform's value comes from allowing both kinds of work while preserving a shared physical protocol.

What the work contributes

At the hardware level, KlaskTron shows that a competitive physical multi-agent task can fit on a desk, use accessible components, and reset without human intervention. At the software level, it connects camera estimation, ROS 2 control, Isaac Lab training, neural actuator modeling, and PPO inference into one released stack. At the methodological level, it demonstrates a particularly clean sim-to-real diagnosis: model the execution layer before asking a larger policy to absorb every mismatch through domain randomization.

The platform is not yet a complete benchmark ecosystem. It needs more users, more independently built units, larger human studies, obstacle variants, and standardized reporting. The paper acknowledges most of these gaps and treats the current agent as a starting point. That makes the contribution durable: future work can replace the policy or reward while keeping the physical testbed, and comparisons can focus on algorithm differences rather than incompatible hardware demonstrations.

For robot learning teams, the most transferable lesson is the separation between policy intelligence and actuator fidelity. A policy cannot reliably learn a contact-rich task if simulation gives it an execution model that the hardware never had. KlaskTron turns that abstract warning into a measurable pipeline, with a 0.013 m/s global validation RMSE, explicit peak errors, and a physical benchmark that reveals where those errors still matter.

Conclusion

KlaskTron is a reproducible physical arena for adversarial MARL, built around dual CoreXY gantries, magnetic strikers, camera-based state estimation, and an Isaac Lab digital twin. Its PPO baseline transfers without hardware fine-tuning because a ten-step neural actuator model captures the real command response, even though the paper's best policy disables broad domain randomization. The result is a robot that can sustain fast physical exchanges and exhibit defensive and offensive behavior, while still losing nine of ten games to a recreational human.

The honest limitations are part of the value. The current agent is not elite, the obstacles are removed, the human sample is small, and the public repository does not exactly match every reported hyperparameter. These are concrete starting points for the next paper rather than reasons to dismiss the platform. KlaskTron succeeds because it makes those gaps measurable on common hardware and releases enough of the stack for another group to challenge, reproduce, or improve the result.