robocurve/inspect-robots

What it solves

Inspect Robots provides a standardized evaluation framework for physical AI, allowing developers to test robotics policies (such as LLM agents or Vision-Language-Action models) across different robot hardware (embodiments) or simulators without rewriting the benchmark for each combination.

How it works

The framework decouples the task definition from the robot hardware and the AI policy. It uses a plugin system where "embodiments" (adapters for specific robots like Franka or Unitree G1) and "policies" (the AI models driving the robot) are registered as separate packages. Before a run, the system performs a compatibility check to ensure the policy's action/observation spaces match the robot's capabilities.

Runs are executed via a CLI or Python API, with integrated Rerun visualization for streaming camera feeds and telemetry. Every trial is recorded into an immutable EvalLog for auditable, offline re-scoring and analysis.

Who it’s for

Robotics researchers and engineers developing VLAs, LLM-based agents, or code-as-policy agents who need a rigorous, reproducible way to evaluate their models on real hardware or in simulation.

Highlights

• Hardware Agnostic: Supports a wide range of real robots (via dedicated plugins or a general ROS adapter) and simulators like Isaac Lab.
• VLA-Native: Built-in support for action chunking, open-loop execution, and temporal ensembling.
• Auditable Logging: Generates schema-versioned logs and HTML reports for every run, including transcripts for LLM agents.
• Live Visualization: First-class integration with Rerun for real-time streaming of images, 3D poses, and joint time-series.
• Safety Guardrails: Includes default bounds clamping and delta limits to prevent erratic robot movement.
• Learning Loop: Ability to summarize failed runs into markdown "learnings" files that can be passed back into subsequent agent runs.