google-deepmind/bsuite

bsuite is a collection of carefully-designed experiments that investigate core capabilities of a reinforcement learning (RL) agent

What it solves

bsuite addresses the difficulty of evaluating reinforcement learning (RL) agents using consistent, scalable, and informative benchmarks. It provides a standardized set of experiments designed to isolate and test core RL capabilities, helping researchers identify specific weaknesses in learning algorithms and facilitate reproducible research.

How it works

The library consists of a collection of carefully designed RL environments. Each experiment includes a specific environment definition, a set of configuration settings (difficulty, seeds), and analysis tools.

Key technical features include:

  • Automated Logging: It uses load_and_record functions to automatically log agent performance data in a format compatible with the provided analysis tools.
  • Interface Compatibility: Environments follow the dm_env interface and include a wrapper for OpenAI Gym compatibility.
  • Analysis Pipeline: A pre-made Jupyter notebook processes the logged data to generate scores and radar plots that visualize agent behavior across different tasks.
  • Baseline Agents: The suite includes several common RL agent implementations to serve as a starting point for comparison.

Who it’s for

RL researchers and developers who need to rigorously test their agents against a shared set of benchmarks to understand their general capabilities and failures.

Highlights

  • Standardized Benchmarks: A curated set of experiments targeting key issues in RL algorithm design.
  • C-CPU Friendly: Environments have small observation sizes, allowing them to run efficiently on CPUs.
  • Automated Reporting: Includes tools to generate a one-page summary appendix compatible with major ML conference formats.
  • Flexible Logging: Supports CSV and terminal logging, with the ability to implement custom logging mechanisms.

Related

  • Project
    DLR-RM/rl-baselines3-zoo

    A training framework for Stable Baselines3 that provides scripts for training, evaluating, and tuning RL agents, along with a collection of pre-tuned hyperparameters for common environments.

  • Dispatch
    OpenAI Benchmarking Safe Exploration in Deep Reinforcement Learning

    OpenAI introduces the Safety Gym benchmark suite to standardize constrained reinforcement learning as a primary formalism for safe exploration in high-dimensional continuous control environments.

  • Project
    JuliaReinforcementLearning/ReinforcementLearning.jl

    A comprehensive Julia package for reinforcement learning research that provides extensible components for implementing, benchmarking, and reproducing RL algorithms.

  • Project
    google-deepmind/acme

    Acme is a research framework for reinforcement learning that provides scalable building blocks and baseline agents for developing and testing RL algorithms.

  • Project
    Farama-Foundation/Gymnasium

    A standard Python API and environment suite for developing and comparing reinforcement learning algorithms.