Denys88/rl_games

RL implementations

What it solves

RL Games is a high-performance reinforcement learning (RL) library designed to accelerate the training of agents in complex environments, particularly for robotics and multi-agent systems. It focuses on maximizing throughput and efficiency, enabling the training of policies for tasks like quadruped locomotion, dexterous manipulation, and multi-agent challenges.

How it works

Implemented primarily in PyTorch, the library provides high-performance implementations of algorithms such as PPO (Proximal Policy Optimization) and SAC (Soft Actor-Critic). It leverages GPU-accelerated training pipelines (such as MJLab and Isaac Lab) and integrates with high-throughput environment execution engines like EnvPool. To further boost performance, it uses custom Triton kernels to replace slow Python loops with fused GPU kernels for operations like Generalized Advantage Estimation (GAE).

Who it’s for

It is intended for researchers and developers working in reinforcement learning, specifically those focusing on robot learning, embodied intelligence, and high-throughput simulation-to-reality transfer.

Highlights

  • High-Throughput Training: Support for end-to-end GPU accelerated pipelines and vectorized environments via EnvPool.
  • Advanced RL Algorithms: Includes PPO (with asymmetric actor-critic variants) and SAC, with benchmarks showing competitive or superior performance on MuJoCo continuous control.
  • Robotics Focus: Extensive results in dexterous manipulation and locomotion (e.g., Unitree Go1, G1 Humanoid).
  • Multi-Agent & Self-Play: Support for multi-agent training with decentralized and centralized critic variants, as well as self-play frameworks.
  • Performance Optimization: Automatic use of Triton kernels for critical operations and support for multi-GPU orchestration via torchrun.

Related

  • Project
    rlcode/reinforcement-learning

    A collection of easy-to-read, standalone PyTorch implementations of core reinforcement learning algorithms, from basic Grid World to deep RL for Atari games.

  • Project
    rl-tools/rl-tools

    A fast, portable C++ library for deep reinforcement learning and continuous control, optimized for everything from microcontrollers to GPUs.

  • Project
    ServiceNow/Fast-LLM

    A high-performance library for training large language models that uses 3D parallelism and optimized kernels to reduce training time and cost.

  • Project
    KellerJordan/modded-nanogpt

    A collaborative speedrun to find the fastest way to train a language model to a specific loss target on 8xH100 GPUs, reducing training time from 45 minutes to under 3 minutes.

  • Project
    meta-pytorch/KernelAgent

    A multi-agent system that converts PyTorch programs into verified Triton kernels and iteratively optimizes their performance using hardware profiling and LLM-guided bottleneck diagnosis.