THUDM/slime

slime is an LLM post-training framework for RL Scaling.

What it solves

slime is an LLM post-training framework designed for RL (Reinforcement Learning) scaling. It addresses the complexity of managing disconnected trainers, rollout services, and agent frameworks by unifying high-performance training and flexible data generation into a single, streamlined dataflow.

How it works

slime connects Megatron (for training) with SGLang (for rollouts/inference). It uses a "pass-through" design that allows users to use native arguments from both upstream engines without adding abstraction layers. The system consists of three primary modules:

  • Training (Megatron): Handles the main training process and synchronizes parameters to the rollout module.
  • Rollout (SGLang + router): Generates new data, including rewards and verifier outputs, which can be extended with custom functions for multi-turn loops or tool calls.
  • Data Buffer: Acts as the bridge managing prompt initialization and the flow of generated samples.

Who it’s for

It is intended for researchers and engineers performing large-scale RL post-training on frontier models (such as the GLM, Qwen, DeepSeek, and Llama series) and those building agentic RL workflows involving sandboxes, verifiers, or multi-modal interactions.

Highlights

  • Production Validated: Used in the training of the GLM family (GLM-4.5 through GLM-5.3).
  • Native Engine Integration: Direct pass-through for Megatron and SGLang arguments, ensuring upstream optimizations are immediately available.
  • Agentic Flexibility: Supports complex data generation workflows, including multi-agent systems, search/RAG, and sandboxed coding agents, without modifying the training kernel.
  • Advanced Deployment: Features PD (Prefill-Decode) disaggregation, delta weight synchronization for efficient updates, and support for external rollout engines.
  • Engineering Rigor: Includes comprehensive CI, GPU end-to-end tests, and dedicated tools for tracing and profiling.

Related

  • Project
    vllm-project/vime

    An LLM post-training framework for RL scaling that integrates Megatron for training and vLLM for high-performance rollouts and data generation.

  • Project
    radixark/miles

    An enterprise-grade reinforcement learning framework for large-scale model post-training that integrates SGLang for rollouts and Megatron-LM for scalable training.

  • Dispatch
    vime RL Framework Release

    vLLM introduces vime, an open-source RL post-training framework that integrates Megatron training with vLLM inference to provide a stable and efficient pipeline for LLM reinforcement learning.

  • Project
    XYZ-AI-Lab/axrl

    An agentic RL post-training framework that coordinates high-throughput rollouts via SGLang and large-scale training via Megatron for multi-turn agent workflows.

  • Project
    PrimeIntellect-ai/prime-rl

    A framework for large-scale asynchronous reinforcement learning designed to train 1T+ parameter MoE models across 1000+ GPUs using FSDP2 and vLLM.