meta-pytorch/torchforge

PyTorch-native post-training at scale

What it solves

It removes the complexity of infrastructure management from reinforcement learning (RL) research. By separating model logic from the underlying hardware and communication patterns, researchers can focus on developing RL algorithms without needing to be experts in distributed systems or GPU placement.

How it works

Torchforge provides a set of clear RL abstractions and a scalable implementation of those abstractions. It allows users to switch between asynchronous and synchronous training across thousands of GPUs, while still providing low-level primitives for power users who need fine-grained control over fault handling, training load redirection, and communication patterns.

Who it’s for

It is designed for RL researchers and power users who need a scalable, PyTorch-native library for agentic RL that can scale from a few GPUs to thousands.

Highlights

  • Infrastructure Abstraction: Isolates the RL loop from the underlying infrastructure to speed up research.
  • High Scalability: Supports scaling across thousands of GPUs with the flexibility to shift between async and synchronous training.
  • Hackability: All parts of the RL loop can be modified without needing to interact with the infrastructure layer.
  • PyTorch Native: Built to integrate deeply with the PyTorch ecosystem, including support for ROCm and integration with vLLM and torchtitan.

Related

  • Project
    yihaosun1124/OfflineRL-Kit

    A PyTorch library for offline reinforcement learning that provides a standardized framework and implementations of state-of-the-art model-free and model-based algorithms.

  • Project
    rlcode/reinforcement-learning

    A collection of easy-to-read, standalone PyTorch implementations of core reinforcement learning algorithms, from basic Grid World to deep RL for Atari games.

  • Project
    VLARLKit/VLARLKit

    A PyTorch library for training Vision-Language-Action (VLA) models using reinforcement learning, featuring a decoupled architecture to prevent dependency conflicts between models and simulators.

  • Project
    Eclectic-Sheep/sheeprl

    A scalable PyTorch reinforcement learning framework accelerated by Lightning Fabric that decouples RL algorithms from their environments for greater flexibility.

  • Project
    kengz/SLM-Lab

    A modular PyTorch framework for Deep Reinforcement Learning that allows users to configure and run RL experiments via JSON specs across various Gymnasium environments.