rerun: a multimodal data layer and visual debugger for physical AI and robotics

What it solves

Rerun is a visual and temporal debugger for multimodal data. It solves the problem of debugging complex AI and robotics systems where text logs are insufficient. Instead of relying on a simple log, developers can visualize how a robot's internal representations of the world—such as camera feeds, lidar scans, and 3D maps—evolve over time, allowing them to pinpoint exactly when and why a system fails.

How it works

Built in Rust using a purpose-built column-chunk storage system, Rerun ingests multi-rate, multimodal data (including images, point clouds, tensors, and joint states) from various sources like robot logs, simulations, and web video. It provides a Python, Rust, and C++ SDK to log data, and a dedicated viewer that renders these streams in sync in real-time. The data is also queryable via dataframes or SQL, which allows developers to extract clean datasets for training and evaluation.

Who it’s for

It is designed for developers working with sensors, 2D/3D state evolving over time, and specifically those in robotics, computer vision, and simulation.

Highlights

• Multimodal Support: Handles images, point clouds, time series, tensors, transforms, and joint states.
• Temporal Debugging: A built-in viewer allows users to scrub through episodes and compare sensors side-by-side.
• Training Integration: Data can be streamed directly into training pipelines without the need for export jobs.
• Cross-Language SDKs: Full support for Python, Rust, and C++.
• Flexible Ingestion: Compatible with formats like MCAP and LeRobot.