NVlabs/Sana

What is SANA?

SANA (and its many extensions) is an open‑source codebase from NVIDIA Research that implements a family of efficient diffusion models for high‑resolution image and video generation. The repository ships end‑to‑end training and inference pipelines, model checkpoints, and integration helpers for popular ecosystems such as Hugging Face Diffusers, ComfyUI, SGLang, and Cosmos‑RL.

---

Core components

Component	What it does	Typical size / resolution
SANA	Text‑to‑image diffusion, up to 4K. Claims ~20× fewer parameters and ~100× faster inference than Flux‑12B.	1 K‑4 K px images
SANA‑1.5	Scales compute at training and inference time for higher quality without changing the model architecture.	Same as SANA
SANA‑Sprint	One‑step / few‑step generation via sCM distillation. ~0.1 s per 1024 px image on an H100.	1 K‑2 K px images
SANA‑Video / LongSANA	Diffusion video generation using block‑linear attention; supports real‑time minute‑long video (≈27 FPS).	720p‑1080p, up to 1 min
SANA‑Video 2.0	5 B (and upcoming 14 B) text‑to‑video / text‑image‑to‑video model with hybrid linear/softmax attention. Demo for 5 B 720p is online.	720p, 8 s clips
SANA‑WM	Controllable world‑model (2.6 B) that can generate 1‑minute 720p videos with 6‑DoF camera control.	720p video
SANA‑Streaming	Real‑time video‑to‑video editing (2 B params) for 720p, minute‑scale streams.	720p video
Sol‑RL	Reinforcement‑learning wrapper (NVFP4 rollout, BF16 training) that speeds up diffusion training by >4×.	Works with SANA, FLUX‑1, SD3.5‑L

Key technical ideas

• Linear / Block‑Causal Linear Attention – replaces the quadratic‑cost attention in DiT, enabling high‑resolution generation with modest GPU memory.
• DC‑AE (32× compression) – a lightweight auto‑encoder that shrinks images to far fewer latent tokens, further cutting compute.
• Decoder‑only Text Encoder – leverages modern LLM architectures for stronger text‑image alignment and in‑context learning.
• sCM Distillation – a training recipe that lets the model generate high‑quality samples in a single diffusion step.
• Inference‑time scaling (SANA‑1.5) – dynamically adjusts compute (e.g., token‑wise or layer‑wise) at run‑time for a quality‑speed trade‑off.

---

Who might use this?

• Researchers exploring efficient diffusion, linear‑attention, or multimodal generation.
• Developers building image/video generation services that need low latency or run on limited hardware (e.g., 4‑bit or 8‑bit quantized models that fit in 8 GB VRAM).
• Creative artists who want high‑resolution (2 K‑4 K) image generation or short video synthesis with fast turnaround.
• Robotics / Embodied‑AI teams interested in world‑model generation (SANA‑WM) or RL‑based fine‑tuning (Sol‑RL).

---

Getting started (quick‑start guide)

1. Clone the repo

   git clone https://github.com/NVlabs/Sana.git && cd Sana
   

2. Install dependencies (the repo provides an environment.yml/requirements.txt).
3. Pick a model – checkpoints are hosted on Hugging Face under the Efficient‑Large‑Model organization (e.g., SANA-Video_2.0_5B_720p).
4. Run a demo – several Hugging Face Spaces are linked in the README; for local inference you can use the Diffusers pipelines:

   from diffusers import SanaPipeline
   pipe = SanaPipeline.from_pretrained("Efficient-Large-Model/SANA-Video_2.0_5B_720p")
   video = pipe("a sunrise over mountains", num_frames=24)
   

5. Fine‑tune or RL‑train – follow the docs/sol_rl/ and docs/sana/ sections for DDP/FSDP scripts, 4‑bit quantization (CAME‑8bit), or RL rollout with Cosmos‑RL.

---

Community & support

• Discord: https://discord.gg/rde6eaE5Ta – active discussion, troubleshooting, and announcements.
• Hugging Face Model Zoo: all released checkpoints and Diffusers integrations.
• Demo sites: web demos for SANA‑Sprint, SANA‑Video 2.0, SANA‑WM, and SANA‑Streaming are linked in the README.
• Paper & citations – multiple arXiv pre‑prints (e.g., 2509.24695, 2605.30409) are listed; they provide the scientific background.

---

License

The code is released under the Apache 2.0 license (changed on 2025‑01‑11), allowing commercial use, modification, and redistribution.

---

TL;DR

SANA is a production‑grade, efficiency‑focused diffusion framework that lets you train and run state‑of‑the‑art image and video generators at high resolutions with dramatically lower compute than traditional models. It comes with ready‑to‑use checkpoints, integration with major ML libraries, and a suite of extensions for fast single‑step generation, video streaming, world‑model control, and RL‑based fine‑tuning.