MirroS-Lab/Code-as-World

What it solves

It addresses the gap between raw visual observations (pixels) and a true understanding of the physical world. While pixels show how things look, they don't explain the underlying rules, structures, or dynamics that govern how objects behave. This project enables models to move beyond simple visual recognition to quantitative physical reasoning.

How it works

Code-as-World uses code as an executable representation of the physical world. It employs an agentic process to discover these representations through iterative simulation and verification. By converting raw observations into explicit states, dynamics, and mechanisms in code, the system can simulate and verify the physical world that could have produced the observed data, providing scalable physical supervision for AI models.

Who it’s for

Researchers and developers working on physical reasoning, embodied intelligence, and the intersection of computer vision and physical simulation.

Highlights

• Executable World Models: Represents the physical world as code that can be run and verified.
• Agentic Discovery: Uses an iterative process of simulation and verification to find the correct physical representations.
• Physical Reasoning: Achieves state-of-the-art performance on quantitative physical reasoning benchmarks like QuantiPhy.
• Pre-trained Checkpoints: Provides VL (Vision-Language) models in 4B and 9B parameter sizes.