xdit-project/xDiT

What it solves

xDiT addresses the high computational cost and quadratic growth of attention mechanisms in Diffusion Transformers (DiTs), which often make real-time high-quality image and video generation difficult on single GPUs. It enables these large models to be deployed across multiple GPUs and machines to meet the performance requirements of online services.

How it works

xDiT is a scalable inference engine that utilizes a variety of parallelization and acceleration techniques:

• Parallel Inference: It implements several strategies that can be combined in a hybrid manner:
  • Unified Sequence Parallel (USP): Combines DeepSpeed-Ulysses and Ring-Attention.
  • PipeFusion: A sequence-level pipeline parallelism that leverages temporal redundancy in diffusion models.
  • Data Parallel: Parallelizes across multiple prompts or images.
  • CFG Parallel: A split-batch approach for classifier-free guidance.
• Cache Acceleration: Integrates methods like TeaCache and DiTFastAttn to exploit redundancies between diffusion steps.
• Computing Acceleration: Uses kernel optimizations, torch.compile, and onediff to boost single-GPU performance.
• Attention Backends: Supports a wide array of backends (e.g., FlashAttention, cuDNN, AITER for AMD GPUs) and a hybrid attention schedule to balance precision and speed.

Who it’s for

It is designed for developers and researchers deploying large-scale Diffusion Transformers for image and video generation who need to reduce latency and increase throughput using multi-GPU or multi-machine setups.

Highlights

• Hybrid Parallelism: Ability to combine sequence, pipeline, data, and CFG parallelism to optimize for specific hardware.
• Broad Model Support: Compatible with a wide range of DiTs including Flux, HunyuanVideo, Wan2.1, and Stable Diffusion 3.
• Hardware Flexibility: Optimized for both NVIDIA and AMD GPUs (via AITER).
• Extensible API: Provides simple wrappers to adapt models from the diffusers library.