baidu-baige/LoongSage

What it solves

LoongSage addresses the complexity of training large language models (LLMs) using reinforcement learning (RL) by providing a production-grade, scalable framework. It solves the challenge of decoupling agent business logic from the underlying training infrastructure, while ensuring consistency between training and inference to prevent model collapse during large-scale RL fine-tuning.

How it works

LoongSage uses a modular architecture built on three primary technologies: Ray for distributed scheduling, Megatron-Core for model training, and SGLang for high-throughput inference. The framework features an "AgentFlow" module that allows any agent to interact with the system via a standard chat interface without requiring modifications to the framework. It supports fully asynchronous training, where sampling and training occur in parallel, and uses advanced techniques like Router Replay and importance sampling to maintain probability distribution consistency between the training and inference engines.

Who it’s for

This framework is designed for researchers and engineers working on post-training optimization for frontier-scale LLMs. It is particularly suited for teams needing to implement complex RL strategies (like online distillation or agentic code repair) on large models (such as MoE architectures) while maintaining high throughput and system stability.

Highlights

• Zero-Intrusion Agent Access: Decouples agent logic from the RL framework, allowing seamless integration via standard chat APIs.
• Fully Asynchronous Training: Decouples sampling and training to eliminate bottlenecks, improving throughput by up to 39%.
• System-Level Consistency: Implements specific alignment techniques to keep training-inference probability bias at the 1e-4 magnitude, preventing training collapse.
• Production-Ready Recipes: Includes validated configurations for complex tasks like math reasoning and agentic code repair, tested on large-scale models like DeepSeek-V4-Flash and Qwen3.
• Highly Extensible: Provides 12 core extension points for custom agents, reward models, and RL algorithms without requiring changes to the framework core.