verl-project/verl-recipe

What it solves

verl-recipe provides a centralized, version-controlled collection of community-contributed recipes for the verl (Versatile Reinforcement Learning) library. It solves the problem of managing dependencies and environment compatibility for diverse Reinforcement Learning (RL) experiments, ensuring that users can easily reproduce specific research setups without version conflicts.

How it works

Each recipe in the repository includes a REQUIRED_VERL.txt file that acts as a manifest. This file pins the exact version of the core verl library (via git commit hash or release tag) and the recipe itself. The project provides an install_verl.sh script that reads these manifests to automatically set up the correct environment, either by installing via pip or by cloning the verl source code as a git submodule for editable development.

Who it’s for

This project is for AI researchers and engineers who use the verl framework to train LLMs using reinforcement learning techniques. It is particularly useful for those who need to replicate specific RL algorithms, agent-based workflows, or performance optimizations documented in research papers.

Highlights

• Reproducibility: Every recipe is strictly pinned to compatible versions of the core library to prevent breaking changes.
• Automated Setup: The install_verl.sh script simplifies the complex process of matching library versions with specific research implementations.
• Diverse Implementations: Includes recipes for advanced techniques like Single-stream Policy Optimization (SPO), multi-turn agent rollouts with LangGraph, and GPU-efficient partial rollouts.
• Flexible Deployment: Supports both standard pip installations and editable git-based workflows for users who need to modify the underlying framework.