NVlabs/Sana

SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer

What it solves

SANA provides an efficiency-oriented framework for high-resolution image and video generation. It addresses the high computational cost and memory requirements typically associated with generating 4K images or long-form videos, making these capabilities accessible on consumer-grade hardware (including laptop GPUs with less than 8GB VRAM).

How it works

SANA employs several key architectural optimizations to reduce the burden on hardware:

  • Linear Attention: Replaces standard attention in Diffusion Transformers (DiT) to maintain efficiency at high resolutions.
  • DC-AE: Uses 32x image compression (compared to the standard 8x) to significantly reduce the number of latent tokens.
  • Decoder-only Text Encoder: Utilizes a modern LLM for better text-image alignment and in-context learning.
  • Specialized Video Modules: Uses Block Causal Linear Attention and Causal Mix-FFN for long video generation, and sCM distillation for one-step generation (SANA-Sprint).
  • Quantization: Supports 4-bit and 8-bit quantization to lower VRAM usage.

Who it’s for

SANA is designed for researchers and developers building high-resolution generative AI applications, as well as users who want to run high-quality image and video generation on limited hardware.

Highlights

  • High Resolution: Supports text-to-image generation up to 4K resolution.
  • Extreme Speed: SANA-Sprint can generate 1024px images in as little as 0.1s on an H100.
  • Versatile Modalities: Includes specialized models for image generation, video generation (SANA-Video), world modeling (SANA-WM), and real-time streaming video editing (SANA-Streaming).
  • Hardware Accessible: Capable of running on GPUs with < 8GB VRAM via 4-bit quantization.
  • RL Integration: Includes Sol-RL for faster reinforcement learning convergence during post-training.