alex-petrenko/sample-factory

What it solves

Sample Factory is a high-throughput reinforcement learning (RL) library designed to minimize training time and hardware requirements while achieving state-of-the-art performance. It addresses the challenge of slow RL training by providing highly optimized implementations of policy gradients, specifically Proximal Policy Optimization (PPO).

How it works

The library uses a highly optimized architecture for maximum learning throughput, supporting both synchronous and asynchronous training regimes. It can operate in serial mode for debugging and performs optimally on both CPU-based and GPU-accelerated environments. The system automatically creates model architectures based on action and observation space specifications, but also supports custom model architectures and custom environments.

Who it’s for

Researchers and practitioners who need an efficient, scalable RL framework for training agents in diverse environments, including pixel-based (like VizDoom) and physics-based (like Mujoco or IsaacGym).

Highlights

• High Throughput: Optimized for maximum learning speed and minimal hardware overhead.
• Flexible Training: Supports synchronous, asynchronous, and serial training modes.
• Broad Compatibility: Works with discrete, continuous, and hybrid action spaces, as well as vector-based, image-based, and dictionary observation spaces.
• Advanced RL Features: Includes support for single- and multi-agent training, self-play, Population-Based Training (PBT), and multi-policy training.
• Integration: Integrated with HuggingFace for model sharing and provides detailed monitoring via WandB and Tensorboard.