X-GenGroup/Flow-Factory

A unified framework for easy reinforcement learning in Flow-Matching models

What it solves

Flow-Factory is a unified framework designed to make reinforcement learning (RL) easier for diffusion and flow-matching models. It provides a standardized way to fine-tune these models to improve specific outcomes—such as aesthetic quality, text-image alignment, or text rendering—using various RL algorithms and reward models.

How it works

The framework decouples models from algorithms, allowing users to apply a wide range of RL techniques to various generative models. It supports a flexible reward system that can be pointwise (independent scores), pairwise (comparisons), or groupwise (ranking-based). Users can integrate built-in reward models like PickScore or CLIP, or connect to remote VLM-as-Judge servers via OpenAI-compatible APIs.

Who it’s for

It is intended for researchers and developers working with generative AI who want to optimize the output quality of text-to-image, image-to-image, text-to-video, and audio-video models through RL fine-tuning.

Highlights

  • Broad Model Support: Compatible with a wide array of models including FLUX, Stable Diffusion 3.5, Wan2.1, and MiniMax H3.
  • Diverse RL Algorithms: Implements numerous algorithms such as GRPO, DPO, DPPO, and DiffusionNFT.
  • Multimodal Capabilities: Supports fine-tuning for image, video, and synchronized audio-video content.
  • Flexible Reward System: Includes built-in aesthetic and alignment scorers and supports complex reasoning reward models (e.g., RationalRewards).
  • Acceleration: Integrates with torch.compile and multiple attention backends (Flash-Attention, xformers) for optimized training.

Related

  • Project
    Tencent-Hunyuan/UniRL

    UniRL is a reinforcement learning framework that provides a unified post-training loop for multimodal models, including image and video diffusion, autoregressive VLMs, and LLMs.

  • Project
    ReinFlow/ReinFlow

    A policy gradient framework for fine-tuning flow matching policies using online reinforcement learning, enabling efficient optimization of robotic control policies with minimal denoising steps.

  • Project
    hmorimitsu/ptlflow

    A unified PyTorch Lightning framework for training and evaluating a wide collection of state-of-the-art optical flow estimation models.

  • Project
    agentscope-ai/Trinity-RFT

    A general-purpose framework for reinforcement fine-tuning of LLMs that decouples exploration, training, and data buffering to support diverse RL algorithms and agentic workflows.

  • Project
    verl-project/verl-omni

    A general RL training framework designed for multimodal generative models, providing fast and stable post-training for diffusion and omni-modality models.