PaddlePaddle/PARL

A high-performance distributed training framework for Reinforcement Learning

What it solves

PARL provides a flexible and high-efficiency framework for reinforcement learning (RL), addressing the difficulty of implementing reproducible algorithms and scaling training across large-scale compute resources like multi-GPU and CPU clusters.

How it works

The framework uses three primary abstractions to organize the RL process:

  • Model: Defines the forward network (policy or critic networks).
  • Algorithm: Manages the mechanism for updating the model parameters.
  • Agent: Acts as the data bridge, handling I/O and preprocessing between the environment and the algorithm.

For scaling, PARL uses a decorator-based API (@parl.remote_class) and a connection method (parl.connect) to offload computation to remote CPUs or machines, allowing users to write distributed code as if it were multi-threaded.

Who it’s for

It is designed for researchers and developers building RL agents for complex tasks, ranging from simple problems like CartPole to advanced applications like autonomous driving simulation.

Highlights

  • Reproducible: Includes implementations of many influential RL algorithms.
  • Large Scale: Supports high-performance parallelization across thousands of CPUs and multiple GPUs.
  • Reusable: Algorithms can be adapted to new tasks by simply defining a new forward network.
  • Extensible: Allows for the quick creation of new algorithms by inheriting abstract base classes.
  • Distributed Training: Simplifies the transition to parallel versions of code via a compact API.

Related

  • Project
    alibaba/ROLL

    ROLL is an open‑source library for reinforcement‑learning training of large language models at scale. It provides Ray‑based distributed execution, supports many RL algorithms (PPO, GRPO, Reinforce++, etc.), works with Megatron‑Core, vLLM, SGLang, and LoRA, and offers pipelines for RL‑based fine‑tuning (RLVR) and multi‑turn agentic RL. The project includes extensive docs, hardware‑agnostic support (NVIDIA, AMD, Ascend), and built‑in observability.

  • Project
    NVIDIA/physicsnemo

    NVIDIA PhysicsNeMo is a PyTorch-based framework for building and scaling AI models for scientific and engineering simulations, supporting diverse data structures like meshes and grids.

  • Project
    QData/TextAttack

    A Python framework for generating adversarial examples, augmenting data, and training robust NLP models to identify and fix vulnerabilities in text-based AI.

  • Project
    ethen8181/machine-learning

    A continuously updated repository of Jupyter notebooks documenting a personal journey through machine learning, deep learning, and data science topics, with a balance between mathematical foundations, from-scratch Python implementations, and real-world library usage.