verl-project/uni-agent

Uni‑Agent: Train Long‑Horizon Agents at Scale

What it is – Uni‑Agent is an open‑source framework that lets you plug any LLM‑based “agent harness” (e.g., Claude Code, Mini‑SWE‑Agent) into a unified training and inference pipeline. It provides abstractions for agents, tools, tasks, and sandboxed execution environments, then runs thousands of stateful sessions in parallel so you can collect traceable trajectories for supervised‑fine‑tuning (SFT) or reinforcement learning (RL).

Key ideas

• Gateway – a thin service that receives a prompt string, forwards it to the chosen model endpoint (OpenAI‑ or Anthropic‑compatible), and returns the generated tokens. Any existing harness can be hooked up to this gateway.
• Modular abstractions – Agent, Tool, Task, and Sandbox are separate, reusable components. You can swap a tool (e.g., a code executor) or a task definition without touching the rest of the stack.
• Massive concurrency – Distributed workers launch isolated sandboxes (via veFaaS or Modal), pool gateway sessions, and schedule asynchronous interactions. The system can handle 1 000+ concurrent long‑horizon sessions while keeping each trajectory, log, and reward linked to its session.
• Reproducible recipes – The repo ships ready‑to‑run example configurations, benchmark scripts, and result tables so you can verify the numbers yourself.

Typical workflow

1. Install the library (the docs cover the verl extras like colocate_async).
2. Launch a sandbox – either locally or on a cloud provider; the sandbox isolates the agent’s tool use and state.
3. Run inference – fire many agents through the gateway to generate evaluation data or benchmark scores.
4. Train with RL – use the same interaction stack for reinforcement‑learning loops (GRPO/GSPO‑style objectives, partial rollouts). Example training scripts are in examples/quickstart/training/.

Performance highlights (from the README)

• Parallel inference on benchmarks such as SWE‑Bench Verified and Terminal‑Bench, achieving scores like 64.2 % with a 480 B Qwen3‑Coder model.
• RL improvements – e.g., ReAct agents on the R2E‑Gym dataset improve from a base score of 22.2 % to 36.8 % after RL; similar gains are shown for other model/agent combos.
• The Qwen3.5‑4B RL run used 64 H100 GPUs and peaked at step 50 on SWE‑Bench Verified.

Why it matters – Training agents that can plan over many steps (hundreds of turns, > 100 K tokens) is computationally heavy. Uni‑Agent abstracts away the plumbing (sandbox management, async scheduling, data collection) so researchers can focus on the agent logic, reward design, or model improvements.

Where it runs – Supports serverless back‑ends like veFaaS (Volcengine) and Modal for elastic sandbox provisioning, but can also be run locally for development.

Getting started – The docs provide a quick‑start guide covering installation, sandbox launch, inference, and RL training. Example configs are fully runnable, and the roadmap (issue #79) lists upcoming features.

License – Apache 2.0.

---

All details are taken directly from the repository’s README; no additional claims are made.