MUNDO: A Few Photos Become an Interactive, Simulatable World
MUNDO turns 1 to 12 photos of a real place into a world you can explore, edit and simulate. A 3D foundation model recovers camera poses and geometry, a 3D detector proposes object boxes, and a VLM editing agent refines them in an analysis-by-synthesis loop against a render engine, with a separate reviewer agent checking every input view. The result is a proxy world of 3D bounding boxes plus a scene program, a code world model storing objects, poses and physical properties such as mass and friction. Simulations run in MuJoCo, covering collisions, contact and robot manipulation, and the state is rendered into flat-colour proxy frames. A video diffusion model finetuned from MiniMax-H3 Ref2VA on about 40k real-and-proxy video pairs (124 frames, 640x480) renders proxy frames plus context views into photorealistic video, streaming long sequences in 124-frame chunks with the last 34 frames as a fixed prefix. The world supports new-viewpoint exploration, object edits, and Real-to-Sim robot navigation and manipulation tests.
