NVlabs/FastGen

NVIDIA FastGen: Fast Generation from Diffusion Models

What it solves

FastGen addresses the slow generation speeds typically associated with diffusion models. It provides a framework to "distill" or compress large, complex models into smaller, faster versions that can produce high-quality images, videos, and audio in significantly fewer steps.

How it works

Built on PyTorch, the framework acts as a central hub for various acceleration and distillation techniques. It allows users to train models using methods like trajectory matching and distribution matching. The framework is designed to be agnostic to the specific network architecture or data type, meaning it can be applied to a wide range of existing models (such as FLUX or Wan) and modalities (like image-to-video or video-to-video) through a flexible configuration system.

Who it’s for

It is designed for researchers and developers working on generative AI who need to optimize models for faster inference. It is particularly useful for those looking to implement large-scale training (20B+ parameters) or those who need to integrate custom architectures and datasets into a standardized, high-performance training pipeline.

Highlights

  • Supports diverse generation tasks including text-to-image, image-to-video, and multimodal video+audio.
  • Includes a wide library of distillation methods like DMD2, flow maps, and self-forcing.
  • Scales to large models using distributed training tools like DDP and FSDP2.
  • Provides integration with ComfyUI for interactive, real-time testing of student models.
  • Highly modular design allows for easy addition of new network architectures and custom datasets.

Related

  • Project
    AlibabaResearch/SparkDiffusion

    SparkDiffusion is an open-source framework for accelerating video generation with Diffusion Transformer (DiT) models, targeting Wan 2.1 and Wan 2.2. It combines sparse low-rank attention, few-step distillation, and custom operators to speed up inference by over 200x while preserving quality.

  • Project
    hao-ai-lab/FastVideo

    FastVideo is an open‑source framework for training, distilling, and running video diffusion models at real‑time speeds. It offers sparse‑attention back‑ends, LoRA finetuning, and a ready‑to‑use web app (Dreamverse) for streaming video generation, and runs on NVIDIA GPUs, Apple Silicon, and DGX Spark.

  • Project
    lucidrains/denoising-diffusion-pytorch

    A PyTorch implementation of Denoising Diffusion Probabilistic Models (DDPM) for generating high-quality images and 1D sequences.

  • Project
    dc-ai-projects/DC-Gen

    DC-Gen is an acceleration framework that transfers pre-trained diffusion models into a deeply compressed latent space, enabling up to 53.8x faster inference and native 4K image generation.

  • Project
    thu-ml/Causal-Forcing

    A framework for high-quality real-time interactive video generation using Causal ODE and Consistency Distillation to optimize autoregressive diffusion distillation.