xdit-project/xDiT

xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism

What it solves

xDiT addresses the computational bottlenecks of Diffusion Transformers (DiTs), where the Attention mechanism's demand grows quadratically with input context length. This makes real-time high-quality image and video generation difficult on single GPUs, necessitating scalable multi-GPU and multi-machine deployments.

How it works

xDiT is an inference engine that implements several parallelization and acceleration strategies:

  • Parallel Inference: It uses a hybrid approach combining Sequence Parallelism (USP), sequence-level pipeline parallelism (PipeFusion), Data Parallelism, and CFG Parallel (Split Batch) to optimize communication across hardware.
  • Cache Acceleration: It integrates methods like TeaCache and DiTFastAttn to exploit computational redundancies between diffusion steps.
  • Computing Acceleration: It utilizes kernel acceleration, torch.compile, and onediff to boost single-GPU performance.
  • Attention Backends: It supports a wide array of backends including FlashAttention (v2, v3, v4), cuDNN, and AITER for AMD GPUs.

Who it’s for

Developers and researchers deploying large-scale Diffusion Transformers for image and video generation who need to reduce latency and increase throughput using multiple computing devices.

Highlights

  • Broad Model Support: Compatible with a wide range of DiTs including Flux, HunyuanVideo, Wan2.1, and Stable Diffusion 3.
  • Hybrid Parallelism: Ability to combine multiple parallel methods to match the specific number of available devices.
  • Flexible Integration: Provides wrappers to adapt models from the huggingface/diffusers library.
  • Multi-Hardware Support: Optimized for NVIDIA GPUs, AMD GPUs (via AITER), and Ascend NPUs.

Related

  • Project
    AlibabaResearch/SparkDiffusion

    A video-generation acceleration framework for Diffusion Transformer (DiT) models that uses sparse low-rank attention and few-step distillation to achieve over 200x inference speedups.

  • Project
    Jasonzzt/ComfyUI-CacheDiT

    A ComfyUI extension that accelerates Diffusion Transformer (DiT) models through intelligent caching, providing 1.4x to 2.0x speedups for image and video generation.

  • Project
    vipshop/cache-dit

    Cache‑DiT is a PyTorch‑native inference engine that accelerates diffusion transformer (DiT) models with hybrid caching, various parallelism strategies, quantization, and CPU off‑load. Built on Hugging Face Diffusers, it supports 40+ model families (image, video, audio, 3‑D) and integrates with tools like SGLang, vLLM‑Omni, TensorRT‑LLM, and ComfyUI. The package offers up to 9× speed‑up, works on NVIDIA, AMD and Ascend hardware, and provides a GitHub SKILL for easy agent‑assisted model integration.

  • Dispatch
    Optimizing Stable Diffusion XL (SDXL) for Inference Speed and Memory

    Hugging Face explores several optimization techniques for Stable Diffusion XL (SDXL), demonstrating how to reduce memory usage from 28GB to as low as 11.47GB and decrease inference latency from 72.2 seconds to approximately 10.3 seconds.

  • Dispatch
    Accelerating Stable Diffusion XL Inference with JAX on Cloud TPU v5e

    Hugging Face Diffusers now supports serving Stable Diffusion XL (SDXL) using JAX on Cloud TPU v5e, delivering up to 2.4x greater performance per dollar compared to TPU v4.