OpenPipe/ART

Agent Reinforcement Trainer: train multi-step agents for real-world tasks using GRPO. Give your agents on-the-job training. Reinforcement learning for Qwen3.6, GPT-OSS, Llama, and more!

What it solves

ART (Agent Reinforcement Trainer) addresses the difficulty of improving the reliability of multi-step AI agents. It allows developers to move beyond static datasets by enabling LLMs to learn from experience through reinforcement learning (RL), specifically using Group Relative Policy Optimization (GRPO), without requiring extensive DevOps or infrastructure management.

How it works

ART splits its functionality between a client and a server:

  1. Inference: An OpenAI-compatible client executes agentic workflows. Completion requests are routed to an ART server running the model's latest LoRA via vLLM. Each interaction is stored as a "Trajectory."\n2. Reward Assignment: Once a rollout finishes, the developer assigns a reward to the trajectory based on the agent's performance.
  2. Training: Trajectories are grouped and sent to the server, which trains the model using GRPO. The server then saves the new LoRA checkpoint and reloads it into vLLM for the next round of inference.

It also offers a "Serverless RL" option via W&B Training to automate infrastructure management, reducing costs and increasing training speed.

Who it’s for

Developers building multi-step AI agents who want to improve their model's reasoning and tool-usage capabilities through reinforcement learning but want to avoid the complexity of managing GPU clusters and RL training loops.

Highlights

  • GRPO Integration: Provides an ergonomic harness for integrating Group Relative Policy Optimization into Python applications.
  • Broad Model Support: Compatible with most vLLM and HuggingFace-transformers causal language models (via Unsloth).
  • Flexible Deployment: Supports running the ART server on local GPUs or using a managed serverless backend.
  • Extensible Integrations: Works with LangGraph, MCP servers, and observability tools like Langfuse and OpenPipe.

Related

  • Project
    AgentR1/Agent-R1

    A modular framework for Agentic Reinforcement Learning that trains multi-step LLM agents by treating each interaction turn as a step-level MDP transition.

  • Project
    langfengQ/verl-agent

    An extension of veRL for training LLM agents via reinforcement learning, featuring a step-independent rollout mechanism to efficiently handle long-horizon, multi-turn tasks.

  • Project
    RLinf/RLinf

    A flexible and scalable reinforcement learning infrastructure designed to streamline the training of Embodied and Agentic AI across diverse hardware and simulators.

  • Project
    agentscope-ai/Trinity-RFT

    A general-purpose framework for reinforcement fine-tuning of LLMs that decouples exploration, training, and data buffering to support diverse RL algorithms and agentic workflows.