google-deepmind/physics-IQ-benchmark

What it solves

Physics-IQ addresses the difficulty of evaluating whether generative video models actually understand physical laws or are simply producing visually plausible but physically incorrect sequences. It provides a standardized, high-quality benchmark to measure a model's ability to simulate realistic physical phenomena.

How it works

The project provides a comprehensive dataset of real-world videos (not rendered) capturing various physical events. It offers two versions: the original Physics-IQ and a "Verified" version with improved prompts and metrics. The benchmark tests models by comparing their generated outputs against these real-world captures across diverse scenarios, including gravity, fluid dynamics, collisions, and magnetism. Each scenario is filmed from multiple perspectives and recorded multiple times to account for natural physical variation.

Who it’s for

Researchers and developers building generative video models who need a rigorous way to quantify their model's physical accuracy and track progress against a leaderboard of other frontier models.

Highlights

• Real-world data: Uses high-quality camera captures rather than synthetic renders.
• Broad physical scope: Covers collisions, fluid dynamics, gravity, material properties, light, shadows, and magnetism.
• High fidelity: Videos are recorded at 4K resolution (3840×2160) and 30 frames per second.
• Multi-perspective: Each scenario is filmed from three different angles to ensure robustness.