microsoft/agent-lightning

What it solves

Agent Lightning is a lightweight framework designed to train AI agents using Reinforcement Learning (RL) without requiring significant changes to the agent's existing code. It solves the difficulty of integrating RL training into complex agentic workflows that involve tools, context, and specific control flows.

How it works

The framework uses a simple three-component architecture to capture and update the model:

• Trainer: Manages the training process using verl and vLLM to build training samples and update the policy.
• API Gateway: Acts as a proxy for model requests, capturing the interaction data needed for training.
• Rollout Controller: Launches and manages agents, either locally or as Kubernetes Jobs, allowing them to interact with their real environments.

Who it’s for

It is intended for developers and researchers who want to apply RL to improve the performance of AI agents—particularly coding agents, search agents, and reasoning agents—within their actual operational harnesses.

Highlights

• Minimal Codebase: The entire framework consists of approximately 3,500 lines of code.
• Zero-Change Integration: Agents interact via a proxy, allowing them to keep their existing tools and environments without modification.
• Native Kubernetes Support: Agents can be run directly as Kubernetes Jobs for scalable rollouts.
• Proven Performance: Demonstrated significant gains in coding agents (e.g., improving SWE-bench Verified scores for Qwen3.5 models) using relatively small training sets.