facebookresearch/ReAgent

A platform for Reasoning systems (Reinforcement Learning, Contextual Bandits, etc.)

What it solves

ReAgent provides a complete platform for applied reinforcement learning (RL), specifically designed for large-scale, distributed tasks where a simulator is unavailable. It addresses the challenges of training policies offline on batches of data and safely deploying them in real-world environments.

How it works

Built with Python and PyTorch, the platform uses TorchScript for model serving. It integrates a full workflow for deep RL, including data preprocessing, feature transformation, distributed training, and optimized serving. To handle the lack of a simulator, it focuses on off-policy algorithms and uses counterfactual policy evaluation (CPE) to estimate a new policy's performance without needing to deploy it first.

Who it’s for

It is intended for developers and researchers working on large-scale optimization and recommendation systems that require offline RL training and safe policy deployment.

Highlights

  • Broad Algorithm Support: Includes classic off-policy algorithms (DQN, TD3, SAC, PPO), RL for recommender systems (Seq2Slate, SlateQ), and Multi-Arm/Contextual Bandits.
  • Counterfactual Evaluation: Implements techniques like Doubly Robust and MAGIC to evaluate policies offline.
  • Real-world Application Tools: Features a Domain Analysis Tool for feature importance and Behavior Cloning to bootstrap learning policies safely.
  • End-to-End Workflow: Covers everything from data preprocessing to distributed training and model serving.

Related

  • Project
    RLinf/RLinf

    A flexible and scalable reinforcement learning infrastructure designed to streamline the training of Embodied and Agentic AI across diverse hardware and simulators.

  • Project
    OpenPipe/ART

    An open-source RL framework that uses GRPO to train multi-step AI agents to learn from experience and improve reliability in real-world tasks.

  • Project
    microsoft/agent-lightning

    A lightweight reinforcement learning framework that allows developers to train AI agents using their existing harnesses with minimal code changes.

  • Project
    areal-project/AReaL

    A large-scale asynchronous reinforcement learning system designed for training high-performance reasoning and agentic AI models.

  • Dispatch
    Decision Transformer Integration in Hugging Face Transformers

    Hugging Face has integrated the Decision Transformer, an offline reinforcement learning method that treats RL as a conditional-sequence modeling problem, into the transformers library and Hugging Face Hub.