PrimeIntellect-ai/prime-rl

What it solves

prime-rl provides a scalable framework for large-scale reinforcement learning (RL), enabling the training of massive models (up to 1T+ parameters) across thousands of GPUs. It addresses the complexity of coordinating high-throughput agentic training, multimodal support, and the deployment of frontier models on multi-node clusters.

How it works

The framework utilizes a fully asynchronous RL architecture to maximize throughput. It leverages PyTorch FSDP2 for training and vLLM for inference, incorporating advanced optimization techniques such as FP8 inference, P/D disaggregation, and Expert Parallelism (EP) and Context Parallelism (CP) for Mixture-of-Experts (MoE) models and long sequences. It integrates natively with the Prime Intellect Environments Hub for agentic and SWE (Software Engineering) environments and supports deployment via Slurm and Kubernetes.

Who it’s for

It is designed for AI researchers and engineers training frontier-scale models, specifically those working with large MoE models, multimodal VLMs, or complex agentic tasks that require massive compute resources.

Highlights

• Massive Scalability: Capable of scaling to 1000+ GPUs for training models with 1T+ parameters.
• High-Performance Stack: Combines FSDP2 and vLLM with FP8 inference and specialized kernels like quack-kernels.
• End-to-End Pipeline: Supports the full post-training lifecycle, including SFT, RL training, and evaluations.
• Broad Model Support: Optimized for various MoE families (e.g., GLM-5, Qwen3, Nemotron) and supports multimodal VLMs like Qwen3-VL.
• Flexible Deployment: One-line SLURM deployment and native Kubernetes support.