vllm-project/vime

What it solves

Vime is an LLM post-training framework designed for RL (Reinforcement Learning) scaling. It bridges the gap between high-performance training stacks and efficient inference engines, allowing developers to scale RL training workflows while maintaining flexibility in how training data is generated.

How it works

Vime integrates three primary components to create a training loop:

• Training (Megatron): Handles the core training process and synchronizes model parameters to the rollout module.
• Rollout (vLLM + router): Uses vLLM as the default backend to launch inference engines and route generation requests. This module can be customized to include multi-turn loops, tool calls, and environment interactions.
• Data Buffer: Acts as a bridge that manages prompt initialization and handles the generation of samples, including complex agentic workflows.

Who it’s for

It is built for researchers and engineers performing post-training on large language models, specifically those who want to leverage the vLLM ecosystem for efficient rollouts and support for models like Qwen, DeepSeek, and Llama 3.

Highlights

• vLLM Integration: Uses vLLM and vllm-router as the default rollout backend for high performance.
• Flexible Data Generation: Supports custom interfaces for arbitrary data generation workflows, including fully asynchronous rollouts.
• Agentic RL Support: Provides built-in examples for multi-agent generation and end-to-end coding agents with sandboxed tool use and test-based rewards.
• Broad Model Support: Compatible with the Qwen series, DeepSeek V3/R1, and Llama 3.