RoboDojo-Benchmark/RoboDojo

What it solves

RoboDojo is a unified benchmark designed to evaluate generalist robot manipulation policies. It addresses the gap between simulation and real-world performance by providing a standardized set of tasks and environments that probe specific robotic capabilities rather than simple visual variations.

How it works

The project provides a simulator client, benchmark tasks, and asset validation tools. It leverages Isaac Sim for heterogeneous parallel simulation, allowing different tasks and scenes to run concurrently for fast feedback. The benchmark is split into 42 simulation tasks and 18 real-world tasks across three different robot embodiments. Evaluation is integrated with XPolicyLab, which provides a unified interface for testing various policies.

Who it’s for

Researchers and developers working on generalist robot manipulation policies who need a rigorous, reproducible way to measure performance across both simulated and real-world environments.

Highlights

• Sim-and-Real Unity: Combines 60 total tasks across simulation and real-world settings.
• Capability-Driven Design: Evaluates five dimensions: Generalization, Memory, Precision, Long-Horizon, and Open.
• Scalable Simulation: Uses heterogeneous parallel simulation on Isaac Sim for efficiency.
• Physically Grounded: Supports rigid, articulated, and deformable objects in configuration-driven scenes.
• Leaderboard Integration: Includes seed-controlled layouts and automated aggregation for reproducible rankings.