rl-tools/rl-tools

The Fastest Deep Reinforcement Learning Library

What it solves

RLtools provides a high-performance, portable library for deep reinforcement learning (DRL), specifically targeting continuous control. It addresses the need for RL libraries that can run efficiently across a wide range of hardware, from powerful GPUs and CPUs to resource-constrained microcontrollers and embedded platforms, where traditional Python-based frameworks are often too slow or too heavy.

How it works

Written primarily in C++, the library implements several core DRL algorithms including TD3, PPO, Multi-Agent PPO, and SAC. It leverages hardware-specific acceleration (such as Apple's Accelerate framework or OpenBLAS) to maximize training speed on CPUs. It also supports CUDA for GPU acceleration and provides a Python interface via PyPI for easier integration with Gymnasium environments, though native C++ environments offer the best performance.

Who it’s for

It is designed for researchers and engineers working on continuous control, robotics, and embedded AI, particularly those who need to deploy RL models to microcontrollers (like ESP32 or Teensy) or other embedded systems (like iOS or PX4).

Highlights

  • Extreme Portability: Supports a vast range of platforms including macOS, Ubuntu, iOS, and various microcontrollers.
  • High Efficiency: Capable of training complex tasks (like quadrotor flight) in seconds on consumer hardware.
  • Broad Algorithm Support: Includes implementations of SAC, TD3, PPO, and Multi-Agent PPO.
  • Embedded Ready: Specifically optimized for inference and training on embedded platforms.

Related

  • Project
    alibaba/ROLL

    An efficient scaling library for reinforcement learning with LLMs that optimizes training and inference across large GPU clusters using a distributed Ray-based architecture.

  • Project
    leggedrobotics/rsl_rl

    A GPU-accelerated, lightweight learning library for robotics research that enables rapid prototyping of robot learning algorithms.

  • Project
    uxlfoundation/oneDAL

    A C++ and DPC++ library that provides hardware-accelerated machine learning routines for tabular data across CPUs, GPUs, and distributed systems.

  • Project
    sail-sg/envpool

    A high-performance C++-based environment pool for reinforcement learning that enables massive parallelization and significantly higher simulation throughput than Python-based vector environments.

  • Project
    Denys88/rl_games

    RL Games is a PyTorch‑based, high‑performance reinforcement‑learning library. It offers fast GPU‑accelerated training (PPO, SAC, multi‑agent, self‑play), integrates with EnvPool and Triton kernels, supports many simulators (MuJoCo, Isaac Gym, MJLab, Atari, StarCraft II), provides Colab notebooks and ONNX export, and includes utilities for experiment tracking and population‑based hyper‑parameter search.