agi-brain/xuance

XuanCe: A Comprehensive and Unified Deep Reinforcement Learning Library

What it solves

Deep Reinforcement Learning (DRL) algorithms are often difficult to implement and tune due to their sensitivity to hyperparameters and unstable training processes. XuanCe provides a unified, high-quality, and modularized library of DRL algorithm implementations to make these "magic" policies easier to learn, use, and reproduce.

How it works

XuanCe acts as a comprehensive ensemble of DRL algorithms that is compatible with multiple deep learning backends, including PyTorch, TensorFlow, and MindSpore. It supports a wide range of tasks, including single-agent DRL, Multi-Agent Reinforcement Learning (MARL), Model-Based RL (MBRL), and Contrastive RL (CRL). The library is designed for high modularity, allowing users to flexibly combine models and run training across parallel environments or distributed multi-GPU setups.

Who it’s for

Researchers and developers working with reinforcement learning who need a reliable, standardized implementation of various DRL/MARL algorithms across different deep learning frameworks.

Highlights

  • Multi-Backend Support: Compatible with PyTorch, TensorFlow2, and MindSpore.
  • Extensive Algorithm Zoo: Includes a vast array of implementations such as DQN, PPO, SAC, QMIX, MAPPO, and DreamerV3.
  • Broad Environment Compatibility: Supports Classic Control, Box2D, MuJoCo, Atari, Minigrid, Drones, MetaDrive, MPE, Robotic Warehouse, SMAC, and Google Research Football.
  • Advanced Training Features: Supports automatic hyperparameter tuning, distributed training with multi-GPUs, and visualization via TensorBoard or WandB.
  • Cros-Platform: Runs on Linux, Windows, MacOS, and EulerOS.

Related

  • Project
    MushroomRL/mushroom-rl

    MushroomRL is a modular Python library for reinforcement learning that provides a wide array of classical and deep RL algorithms and integrates with popular tensor libraries and benchmarks.

  • Project
    vwxyzjn/cleanrl

    A Deep Reinforcement Learning library providing single-file, transparent implementations of popular algorithms to simplify research, debugging, and prototyping.

  • Project
    rl-tools/rl-tools

    A fast, portable C++ library for deep reinforcement learning and continuous control, optimized for everything from microcontrollers to GPUs.

  • Project
    Tencent-Hunyuan/UniRL

    UniRL is a reinforcement learning framework that provides a unified post-training loop for multimodal models, including image and video diffusion, autoregressive VLMs, and LLMs.

  • Project
    RLinf/RLinf

    A flexible and scalable reinforcement learning infrastructure designed to streamline the training of Embodied and Agentic AI across diverse hardware and simulators.