verl-project/uni-agent

What it solves

Uni-Agent addresses the difficulty of training and evaluating long-horizon AI agents at scale. It provides a unified way to bring existing agent harnesses into reinforcement learning (RL) workflows, allowing developers to collect traceable trajectories and run thousands of concurrent sessions without managing fragmented infrastructure.

How it works

The framework uses a "Uni-Agent Gateway" that acts as a model endpoint, allowing any OpenAI- or Anthropic-compatible agent harness to plug in and output training tokens. It decouples the system into reusable abstractions—Agent, Tool, Task, and Sandbox—which allows users to customize agent logic and environments independently. For scaling, it employs distributed workers, pooled gateway sessions, and isolated sandboxes with asynchronous scheduling to handle massive amounts of stateful sessions.

Who it’s for

It is designed for AI researchers and developers building complex, multi-step agents who need to perform large-scale benchmarking, trajectory generation for Supervised Fine-Tuning (SFT), or Reinforcement Learning (RL) training.

Highlights

• Harness Agnostic: Supports plugging in existing harnesses like Claude Code and Mini-SWE-Agent.
• Massive Scalability: Capable of running 1,000+ concurrent long-horizon sessions.
• RL Integration: Supports agent RL training using the same interaction stack as inference, including GRPO/GSPO-style objectives.
• Modular Architecture: Separates agent logic, tools, tasks, and sandbox backends for maximum flexibility.