areal-project/AReaL

The RL Bridge for LLM-based Agent Applications. Made Simple & Flexible.

What it solves

AReaL is a reinforcement learning (RL) infrastructure designed to bridge the gap between training foundation models and deploying them as agentic applications. It addresses the inefficiency and complexity of training large-scale reasoning and agentic models, making the process more accessible and cost-effective for developers.

How it works

AReaL employs a fully asynchronous RL training paradigm to optimize for speed and scalability. In version 2.0, it transitioned to a microservice architecture, separating the system into independent services for training, inference, and agent execution, and weight updates. It supports a wide array of RL algorithms (such as GRPO, PPO, and DPO) and integrates with various training backends like Megatron and PyTorch FSDP to handle large-scale model training across multiple nodes and GPUs.

Who it’s for

It is designed for AI researchers and developers building large-scale reasoning models, coding agents, search agents, and vision-language models (VLMs) who require a high-performance, scalable RL training framework.

Highlights

  • Asynchronous Training: Achieves industry-leading speed and stability in RL training, significantly reducing training time compared to synchronous systems.
  • Microservice Architecture: Decouples training, inference, and agent services for better modularity and and scalability.
  • Broad Algorithm Support: Implements a variety of cutting-edge RL algorithms including GRPO, GSPO, PPO, and DAPO.
  • Agentic RL Flexibility: Allows seamless customization for black-box agent applications by simply replacing the base_url.
  • Hardware Support: Supports training on NVIDIA GPUs and Huawei Ascend NPUs.
  • Lightweight Version: Offers AReaL-lite for rapid prototyping and algorithm-first development.