NVIDIA-NeMo/ProRL-Agent-Server

Agentic RL on Any Harness at Scale

What it solves

Polar is a reinforcement learning (RL) rollout framework designed for real-world agent harnesses. It solves the problem of scaling asynchronous RL training for multi-turn LLM agents by providing a "Rollout-as-a-Service" architecture, allowing developers to generate trajectories (rollouts) without needing to modify their existing agent harnesses.

How it works

Polar operates as a distributed server system. A central Rollout Server manages and dispatches requests to distributed Gateway Nodes. These nodes asynchronously prepare the runtime environment, execute agents, build trajectories, and evaluate them. A proxy sits between the agent execution processes and inference servers (like vLLM or SGLang) to handle communication, making the system trainer-agnostic and compatible with various training frameworks.

Who it’s for

Researchers and developers training LLM agents using RL, who need a scalable way to generate and evaluate trajectories across different environments and harnesses without changing their agent's core code.

Highlights

  • Harness-Agnostic: Integrates existing agent harnesses as RL-ready environments without requiring code changes.
  • Scalable Architecture: Uses parallel Rollout Staging and Runtime Pooling to optimize GPU usage.
  • Trainer-Agnostic: Works with any training framework via HTTP server boundaries.
  • Observability: Includes a dedicated dashboard for monitoring and health checks.
  • Broad Support: Compatible with vLLM and SGLang inference servers.

Related

  • Project
    alibaba/ROLL

    An efficient scaling library for reinforcement learning with LLMs that optimizes training and inference across large GPU clusters using a distributed Ray-based architecture.

  • Project
    rllm-org/rllm

    An open-source framework for training language agents with reinforcement learning, allowing the same agent code to be used across different backends, sandboxes, and benchmarks.

  • Project
    langfengQ/verl-agent

    An extension of veRL for training LLM agents via reinforcement learning, featuring a step-independent rollout mechanism to efficiently handle long-horizon, multi-turn tasks.

  • Project
    OpenPipe/ART

    An open-source RL framework that uses GRPO to train multi-step AI agents to learn from experience and improve reliability in real-world tasks.

  • Project
    nvidia-cosmos/cosmos-rl

    A scalable Reinforcement Learning framework specialized for Physical AI applications, providing a decoupled architecture for large-scale training workloads.