NVlabs/RoboLab

A simulation benchmarking platform for evaluating generalist robot policies

What it solves

RoboLab addresses the need for a standardized, high-fidelity evaluation framework for generalist robot manipulation policies. It provides a consistent environment to test how well different AI models perform on diverse robotic tasks, such as pick-and-place, stacking, and tool use, ensuring results are reproducible and comparable.

How it works

Built on NVIDIA Isaac Lab, the project uses a server-client architecture where a robot policy runs as a standalone server and connects to the RoboLab simulation client. It supports multi-environment parallel evaluation, allowing researchers to run many episodes simultaneously. The system includes automated success detection using composable predicates and provides a web-based dashboard for visualizing episode videos and analyzing experiment results.

Who it’s for

This tool is designed for robotics researchers and AI engineers working on general-purpose robot manipulation policies who need a robust, scalable simulation benchmark to validate their models.

Highlights

  • 120+ pre-built manipulation tasks with automated success and failure tracking.
  • Agnostic design that supports any robot compatible with NVIDIA Isaac Lab.
  • AI-assisted workflow for generating new scenes and tasks using natural language.
  • Multi-environment parallel execution for efficient, large-scale benchmarking.
  • Integrated web dashboard for replaying episodes and comparing cross-experiment performance.

Related

  • Project
    isaac-sim/IsaacLab-Arena

    Isaac Lab‑Arena is an open‑source, alpha‑stage framework that sits on top of NVIDIA Isaac Lab/Sim. It lets you compose robot simulation environments from reusable scene, robot (embodiment), and task blocks, then run massive parallel policy evaluations on a GPU or across nodes, collecting fine‑grained metrics and sensitivity analyses. Features include relational object placement, sequential task chaining, experimental natural‑language‑driven environment generation, and a client‑server policy API. Installation uses the `uv` Python manager or provided Docker scripts, and the library is Python‑centric (≥ 3.12, Linux).

  • Project
    isaac-sim/IsaacLab

    Isaac Lab is a GPU-accelerated, open-source robotics simulation framework built on NVIDIA Isaac Sim, designed to simplify and scale robot learning tasks like reinforcement learning and motion planning.

  • Project
    RoboDojo-Benchmark/RoboDojo

    RoboDojo is an open‑source benchmark that evaluates robot‑manipulation policies in both Isaac Sim and on real robots. It provides 42 simulated and 18 real tasks, parallel simulation, and a one‑command workflow that talks to policies hosted via XPolicyLab, producing scores for a public leaderboard.

  • Project
    fan-ziqi/robot_lab

    robot_lab is an Isaac Lab extension that provides a large suite of Gym‑compatible reinforcement‑learning environments for many real‑world robot platforms (quadrupeds, wheeled, humanoids). It supports multiple RL back‑ends, scalable multi‑GPU training, Docker deployment, and easy addition of new robots.

  • Project
    XPolicyLab/XPolicyLab

    XPolicyLab is a unified framework that decouples robot policies from evaluation environments, allowing researchers to test and deploy a wide range of models across different simulators and real-world robot setups using a standardized interface.