Denys88/rl_games

TL;DR

RL Games is an open‑source Python library (built on PyTorch) that provides fast, GPU‑accelerated reinforcement‑learning training pipelines. It supports popular algorithms such as PPO and SAC, works with high‑throughput environment back‑ends like EnvPool, and includes ready‑made configs for many benchmarks (MuJoCo, Atari, StarCraft II, Isaac Gym, etc.).

What it is

Domain: Reinforcement learning (RL) – a core sub‑field of AI dealing with agents that learn by trial‑and‑error.
Goal: Offer a high‑performance, easy‑to‑use framework for training RL agents on a variety of environments, from classic control tasks to large‑scale robot simulators.
Implementation: Pure Python, built on PyTorch ≥ 2.2. Optional extras add support for Atari, MuJoCo, EnvPool, and the pufferlib multi‑agent suite.
Key design: Emphasises GPU‑centric pipelines (e.g., Triton‑fused kernels for GAE, vectorized env execution via EnvPool) to squeeze maximum speed out of modern hardware.

Core features (as described in the README)

Feature	Details
Algorithms	PPO (including asymmetric actor‑critic), SAC, and legacy TensorFlow implementations of Rainbow DQN, A2C, PPO.
Multi‑agent & self‑play	Decentralised and centralised critic variants; built‑in support for StarCraft II multi‑agent challenge.
GPU‑accelerated pipelines	Direct integration with MJLab (MuJoCo Warp), Isaac Lab, and legacy Isaac Gym (≤ v1.6.5).
EnvPool integration	High‑throughput vectorised execution for MuJoCo, Atari, DeepMind Control, etc.
Triton kernels	Optional Triton‑based fused kernels for Generalized Advantage Estimation, giving noticeable speedups.
ONNX export	Utilities to export trained policies (discrete, continuous, LSTM) to ONNX for deployment.
Population‑Based Training (PBT)	DexPBT‑style observers for automated hyper‑parameter evolution.
Experiment tracking	Native hooks for Weights & Biases.
Multi‑GPU support	Simple `torchrun` launch scripts for data‑parallel training.
Extensive docs & examples	Colab notebooks for Mujoco, MJLab, ONNX export; markdown docs for each supported backend.

Typical workflow (quick‑start)

Install the package (optionally with extras for your env):

pip install rl-games               # base
pip install -e "[mujoco,envpool]" # GPU‑heavy workloads

Pick a config from rl_games/configs/… (e.g., atari/ppo_pong.yaml).

Run training:

python runner.py --train --file rl_games/configs/atari/ppo_pong.yaml

Play / evaluate a checkpoint:

python runner.py --play --file rl_games/configs/atari/ppo_pong.yaml \
    --checkpoint nn/PongNoFrameskip.pth

(Optional) Track with Weights & Biases by adding --track and setting WANDB_API_KEY.

Export a policy to ONNX for inference elsewhere:

python notebooks/train_and_export_onnx_example_continuous.ipynb

Who might use it?

Research labs needing a fast baseline for new RL algorithms or for reproducing published results (the README shows benchmark tables matching or exceeding reference scores).
Robotics teams that train dexterous manipulation or locomotion policies in GPU‑accelerated simulators such as Isaac Gym, MJLab, or Isaac Lab.
Game‑AI developers wanting to experiment with Atari or StarCraft II agents without writing low‑level training loops.
Practitioners who want to export a trained policy to ONNX for deployment on edge devices or in other runtimes.

Installation & environment notes

Python: 3.11 – 3.14 (recommended for the upcoming 2.0 release). Older releases support Python ≤ 3.8 for legacy Isaac Gym.
CUDA: Required for the performance‑focused paths; PyTorch ≥ 2.2 with CUDA is recommended.
Optional extras:
- atari – Atari ROM handling.
- mujoco – MuJoCo bindings (needs a MuJoCo licence).
- envpool – High‑throughput env execution.
- pufferlib – Multi‑agent environments.
Package manager: The README suggests using uv for fast environment creation, but pip works as well.

Documentation highlights

Env-specific docs (docs/ISAAC_GYM.md, docs/MJLAB.md, docs/SMAC.md, etc.) give step‑by‑step instructions for each backend.
Benchmark tables (docs/SAC_BENCHMARKS.md) provide reproducibility evidence for continuous‑control tasks.
Config reference table explains the hierarchical YAML structure (seed, algo, model, network, etc.).
Triton performance guide (benchmarks/bench_triton_gae.py).

Citation

If you publish work that uses RL Games, cite the repository as:

@misc{rl-games2021,
  title = {rl-games: A High-performance Framework for Reinforcement Learning},
  author = {Makoviichuk, Denys and Makoviychuk, Viktor},
  month = {May},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/Denys88/rl_games}},
}

Bottom line

RL Games is a genuine, actively maintained reinforcement‑learning library that focuses on speed (GPU‑centric kernels, EnvPool, Triton) and breadth (support for many environments and algorithms). It is well‑suited for researchers and engineers who need a performant training stack without building everything from scratch.