#block-causal (1)

LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics

LeVJEPA performs video self-supervised pretraining with a single encoder, a single loss and one fixed hyperparameter (λ=0.02): an invariance loss plus SIGReg regularization provably rule out representation collapse, with no target encoder, predictor, stop-gradient or pixel reconstruction. It uses 5.6–20.8× less training compute than V-JEPA 2, leads by 7.6 points on ImageNet-1K under a FLOP-matched budget, and gets block-causal attention for free — paving the way to streaming perception and autoregressive world models.