X-GenGroup/Flow-Factory

A unified framework for easy fine-tuning in Flow-Matching models

Flow‑Factory – Unified RL & fine‑tuning for diffusion & flow‑matching models

What it is – A research‑grade Python framework that lets you train, fine‑tune, and evaluate large diffusion‑style generative models (text‑to‑image, text‑to‑video, audio‑video, etc.) using a single configuration language. It combines online reinforcement‑learning style algorithms (GRPO, DPO, DPPO, etc.) with classic offline supervised‑fine‑tuning (SFT) and offline DPO, all built on top of the Hugging‑Face diffusers library.

Key capabilities

Feature Details
Supported model families Stable‑Diffusion 3.5, FLUX 1/2, Qwen‑Image (including 2.1), Z‑Image, BAGEL, SenseNova, Wan2.x (text‑to‑video & image‑to‑video), LTX‑2, MiniMax H3, plus many image‑editing variants.
Tasks Text‑to‑image, image‑to‑image, multi‑image‑to‑image, text‑to‑video, image‑to‑video, first/last‑frame‑to‑video, text‑to‑audio‑video, image‑to‑audio‑video, ordered‑reference‑to‑audio‑video.
Algorithms SFT, Offline‑DPO, Online‑DPO, GRPO, DiffusionNFT, AWM, DGPO, GRPO‑Guard, DPPO, CRD, DiffusionOPD, DMD2, TDM, TDM‑R1.
Training back‑ends PyTorch 2.10+, torch.compile, DeepSpeed ZeRO‑2, FSDP2, optional flash‑attention kernels, custom Muon optimizer.
Acceleration Unified acceleration: block to pick attention back‑ends (native, xformers, flash variants) and combine with torch.compile.
Experiment tracking Built‑in integration with Weights & Biases, SwanLab, TensorBoard.
Dataset format Strict JSONL v2 schema for supervised demonstrations and preference pairs; also simple line‑or‑JSONL prompts for pure text‑to‑image/video.
Reward models Pluggable reward system (built‑in and remote servers) used by DPO/GRPO pipelines.
Docker image CUDA‑enabled training image (Python 3.12, PyTorch 2.10, cu129, DeepSpeed, WandB).
Extensibility Guides for adding new diffusion/flow‑matching models and new datasets.

How you use it

  1. Install – pip install -e . (Python ≥ 3.10, PyTorch ≥ 2.10). Optional extras for DeepSpeed, flash‑attention, or specific adapters.
  2. Pick a config – Example: ff-train examples/grpo/lora/flux1/default.yaml runs GRPO + LoRA on FLUX 1.
  3. Run – The ff-train CLI reads the YAML, builds the model, loads the dataset, creates the chosen algorithm (e.g., trainer_type: grpo), and starts distributed training.
  4. Log – Set logging_backend: wandb (or swanlab) in the config to stream sample images/video and metric curves.
  5. Add your own model or data – Follow guidance/new_model.md and guidance/datasets.md to register a new Diffusers model or custom JSONL schema.

Why it matters

  • Unified interface – No need to switch repositories when moving from SFT to RL fine‑tuning; the same ff-train command works for all supported algorithms.
  • Broad modality support – Extends diffusion fine‑tuning beyond images to synchronized audio‑video, a niche that few open‑source toolkits cover.
  • Research‑ready – Provides a “smoke‑matrix” of 120 model/backend/algorithm combos that have been verified to run, giving confidence for reproducibility.
  • Modular acceleration – The acceleration: block lets you experiment with flash‑attention, xformers, or native kernels without changing model code.

Typical users

  • Researchers exploring RL‑based alignment of diffusion models.
  • Engineers who need a single pipeline to fine‑tune many diffusion families (Stable‑Diffusion, FLUX, Qwen‑Image, etc.).
  • Teams building custom reward models for text‑to‑image/video generation.

Limitations

  • Focused on diffusion/flow‑matching models; not a general‑purpose LLM trainer.
  • Requires substantial GPU resources (most examples assume multi‑GPU or DeepSpeed/FSDP setups).
  • Some optional back‑ends (flash‑attention variants) need extra system‑level libraries.

All information above is taken directly from the repository’s README; no features have been invented.

Related

  • Project
    Lei-Kun/RL-100

    A real-world reinforcement learning framework for robotic manipulation that enables a unified pipeline from imitation learning to offline/online RL and one-step policy distillation.

  • Project
    hmorimitsu/ptlflow

    A unified PyTorch Lightning framework for training and evaluating a wide collection of state-of-the-art optical flow estimation models.

  • Project
    ReinFlow/ReinFlow

    A policy gradient framework for fine-tuning flow matching policies using online reinforcement learning, enabling efficient optimization of robotic control policies with minimal denoising steps.

  • Project
    SalesforceAIResearch/DiffusionDPO

    A training framework for Diffusion-DPO that aligns image generation models like Stable Diffusion with human preferences using Direct Preference Optimization.

  • Project
    opendilab/LightRFT

    LightRFT is an open‑source, distributed reinforcement‑learning framework for fine‑tuning large language and multimodal generative models. It unifies training back‑ends (DeepSpeed, FSDP) with rollout engines (SGLang, vLLM), supports a range of RL algorithms (GRPO, REINFORCE++, DAPO, etc.), and works across text, image, video, and audio modalities. The library provides ready‑to‑run examples (e.g., GSM8K with Qwen2.5), extensive docs, and Docker images, making it suitable for researchers and engineers building RL‑based alignment pipelines.