huggingface/diffusers

🤗 Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch.

What it solves

Diffusers is a library designed to simplify the use and training of state-of-the-art pretrained diffusion models. It provides a modular toolbox that allows users to generate images, audio, and 3D molecular structures without needing to build complex systems from scratch.

How it works

The library is built around three core components:

  • Diffusion Pipelines: High-level APIs that allow users to run inference with just a few lines of code.
  • Noise Schedulers: Interchangeable components that control the diffusion speed and the quality of the output.
  • Pretrained Models: Modular building blocks that can be combined with schedulers to create custom end-to-end diffusion systems.

Who it’s for

It is intended for developers and researchers who want to implement diffusion-based generation tasks—such as text-to-image, image-to-image, inpainting, and super-resolution—either through simple inference or by training their own models.

Highlights

  • Broad Modality Support: Supports generating images, audio, and 3D molecular structures.
  • Modular Design: Prioritizes customizability and usability over strict abstractions.
  • Extensive Model Hub: Access to over 30,000 pretrained checkpoints via the Hugging Face Hub.
  • Optimization Tools: Includes guides and tools for reducing memory consumption and increasing inference speed.

Related

  • Dispatch
    Hugging Face Diffusers First Anniversary Overview

    Hugging Face celebrates the first anniversary of the Diffusers library, highlighting its evolution into a modular toolbox for text-to-image, video, 3D, and audio generation.

  • Project
    yuanchenyang/smalldiffusion

    A lightweight diffusion library for training and sampling from diffusion and flow models, featuring a minimal core for easy experimentation with U-Net and DiT architectures.

  • Project
    huggingface/finetrainers

    A library for the accessible training and fine-tuning of diffusion models, focusing on memory-efficient text-to-video and text-to-image generation.

  • Project
    modelscope/DiffSynth-Studio

    An open-source diffusion model engine that enables high-performance inference and training of image, video, and audio generative models on consumer-grade GPUs.

  • Project
    ZHZisZZ/dllm

    A library that unifies the training, inference, and evaluation of diffusion language models, providing scalable pipelines and recipes for converting AR models into diffusion models.