Eclectic-Sheep/sheeprl

Distributed Reinforcement Learning accelerated by Lightning Fabric

What it solves

SheepRL is a PyTorch-based reinforcement learning (RL) framework designed to be simple and scalable. It addresses the common problem where RL algorithms are tightly coupled with their environments, which often limits their extensibility beyond standard interfaces like OpenAI Gym. SheepRL allows users to easily decouple algorithms from environments, making them compatible with a wider range of setups.

How it works

The framework uses Lightning Fabric to accelerate training across different hardware configurations. It provides a suite of out-of-the-box algorithms (such as PPO, SAC, and Dreamer V1-V3) that support various observation types (vector and pixel) and action spaces (continuous, discrete, and multi-discrete). It includes built-in wrappers for numerous environments, including Atari, Mujoco, MineRL, and Crafter.

Who it’s for

It is intended for developers and researchers working with reinforcement learning who need a scalable framework that supports multiple algorithms and Dreamer-style world models, and who want the flexibility to use their own custom environments.

Highlights

  • Scalable Acceleration: Powered by Lightning Fabric for easy multi-device and multi-node training.
  • Wide Algorithm Support: Includes implementations of PPO, SAC, A2C, DroQ, and the Dreamer series (V1, V2, V3).
  • Environment Flexibility: Supports a vast array of environments including Classic Control, Box2D, Atari, DeepMind Control, and Minecraft-based environments (MineRL, MineDojo).
  • Decoupled Architecture: Specifically designed to separate the RL agent from the environment interface.
  • Integrated Evaluation: Includes a dedicated evaluation tool (sheeprl-eval) to test trained agents from checkpoints.

Related

  • Project
    Lightning-AI/pytorch-lightning

    A deep learning framework that abstracts away the engineering boilerplate of PyTorch, enabling researchers to pretrain and finetune AI models at scale across various hardware.

  • Project
    meta-pytorch/torchforge

    A PyTorch-native agentic RL library that separates infrastructure concerns from model concerns to simplify scalable reinforcement learning experimentation.

  • Project
    kengz/SLM-Lab

    A modular PyTorch framework for Deep Reinforcement Learning that allows users to configure and run RL experiments via JSON specs across various Gymnasium environments.

  • Project
    yihaosun1124/OfflineRL-Kit

    A PyTorch library for offline reinforcement learning that provides a standardized framework and implementations of state-of-the-art model-free and model-based algorithms.

  • Project
    PrimeIntellect-ai/prime-rl

    A framework for large-scale asynchronous reinforcement learning designed to train 1T+ parameter MoE models across 1000+ GPUs using FSDP2 and vLLM.