agentscope-ai/Trinity-RFT

Trinity-RFT is a general-purpose, flexible and scalable framework designed for reinforcement fine-tuning (RFT) of large language models (LLM).

What it solves

Trinity-RFT provides a unified and flexible framework for the reinforcement fine-tuning (RFT) of Large Language Models. It simplifies the complex process of improving LLM capabilities—especially for agentic workflows—by decoupling the data generation, model training, and data pipelining into manageable, interchangeable components.

How it works

The framework operates through three coordinated components:

  • Explorer: Handles agent-environment interaction to generate experience data.
  • Trainer: Updates the model weights by minimizing losses based on the collected data.
  • Buffer: Manages the data pipeline, handling processing, cleaning, and augmentation throughout the RFT lifecycle.

It supports a wide variety of RFT modes, including synchronous/asynchronous, on-policy/off-policy, and online/offline RL. It also integrates with agent frameworks like AgentScope to train multi-step agentic workflows.

Who it’s for

  • Agent Application Developers: Those looking to train LLM-powered agents for specific domain capabilities.
  • RL Researchers: Researchers who need to design and validate new RL algorithms using plug-and-play modules.
  • Data Engineers: Professionals building RFT datasets and data pipelines for cleaning or human-in-the-loop scenarios.

Highlights

  • Diverse RFT Modes: Supports a broad spectrum of RL configurations, including asynchronous rollout and training to scale independently across devices.
  • Agentic RL Support: Capable of training both concatenated and general multi-step agentic workflows.
  • Full-Lifecycle Data Pipelines: Features active data management, including prioritization and online task curriculum construction.
  • Extensive Algorithm Library: Includes implementations of PPO, GRPO, DAPO, DPO, SFT, and several specialized research algorithms like CHORD and REC.

Related

  • Project
    IntelLabs/RAG-FiT

    A library for improving LLM performance in Retrieval Augmented Generation (RAG) by creating augmented datasets and fine-tuning models using PEFT.

  • Project
    OpenManus/OpenManus-RL

    An open-source framework for optimizing LLM agents using reinforcement learning to enhance their reasoning, tool use, and decision-making capabilities.

  • Project
    opendilab/LightRFT

    LightRFT is an open‑source, distributed reinforcement‑learning framework for fine‑tuning large language and multimodal generative models. It unifies training back‑ends (DeepSpeed, FSDP) with rollout engines (SGLang, vLLM), supports a range of RL algorithms (GRPO, REINFORCE++, DAPO, etc.), and works across text, image, video, and audio modalities. The library provides ready‑to‑run examples (e.g., GSM8K with Qwen2.5), extensive docs, and Docker images, making it suitable for researchers and engineers building RL‑based alignment pipelines.

  • Project
    TsinghuaC3I/MARTI

    An open-source framework for training LLM-based multi-agent systems using reinforcement learning, featuring tree search augmentation for complex reasoning and code generation.

  • Project
    rllm-org/rllm

    An open-source framework for training language agents with reinforcement learning, allowing the same agent code to be used across different backends, sandboxes, and benchmarks.