hao-ai-lab/FastVideo

FastVideo – Real‑time, post‑training video generation framework

What it is – FastVideo is an open‑source library that lets you fine‑tune, distill, and run state‑of‑the‑art video diffusion models at high speed. It supports both post‑training (e.g., sparse‑distillation, data‑free DMD2) and real‑time inference on a wide range of hardware (NVIDIA RTX/H100/A100, Apple Silicon via MLX, DGX Spark, Windows/Linux/macOS).

Key capabilities

• Training & finetuning – Full‑model and LoRA finetuning for open video DiTs, a data‑pipeline for video/image/text, and advanced distillation methods (Video Sparse Attention, Sparse‑Distillation, Self‑Forcing causal distillation). Scales with FSDP2, sequence parallelism, and activation checkpointing.
• Inference speedups – Sparse attention back‑ends, sequence‑parallel distributed inference, and a CLI/Python API that automatically picks the best kernel (e.g., VSA, FlashAttention). The library can generate 1080p video in under real‑time on a single consumer GPU.
• Hardware flexibility – Pre‑built wheels for CUDA 12/13, optional MLX support for Apple Silicon, and ARM‑compatible builds for DGX Spark. The same code runs on Windows, Linux and macOS.
• Realtime app – Dreamverse – A bundled web‑app that streams “vibe‑directed” video, deployable locally, via Docker, on a B200 server, or server‑less with Modal.

How to get started

# create a clean uv environment (Python 3.12)
uv venv --python 3.12 --seed
source .venv/bin/activate

# install the CUDA‑12 build (or cu130 for CUDA 13)
UV_TORCH_BACKEND=cu126 uv pip install fastvideo

Apple Silicon: uv pip install -e '.[mlx]' from a clone and follow the MLX install guide. DGX Spark (ARM): install from source with UV_TORCH_BACKEND=cu130 uv pip install -e . and add the separate FlashAttention wheel.

Simple generation example

import os
from fastvideo import VideoGenerator

os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "VIDEO_SPARSE_ATTN"

gen = VideoGenerator.from_pretrained(
    "FastVideo/FastWan2.1-T2V-1.3B-Diffusers", num_gpus=1
)
video = gen.generate_video(
    "A curious raccoon peers through a vibrant field of yellow sunflowers",
    output_path="my_videos/",
    save_video=True,
)

Run with python example.py.

Models & resources

• FastH3 (4‑step, 8‑step, V2) – distilled from MiniMax‑H3, runs on RTX 5090/4090, DGX Spark, and Apple Silicon. Trimmed 4.2× smaller version fits in 8 GB GPU memory.
• Kandinsky 6 – text‑to‑video + synchronized audio, with up‑to‑4× super‑resolution.
• FastWan‑QAD – 5 s video generated in 1.8 s (FP8, 1.3 B).
• All models are hosted on Hugging Face (e.g., FastVideo/FastWan2.1‑T2V‑1.3B‑Diffusers).

Community & docs

• Full documentation site, cookbook with step‑by‑step recipes, and quick‑start guide.
• Weekly dev meetings, Slack, and WeChat channels for real‑time help.
• Contributions welcomed via the standard GitHub flow; a detailed contribution guide and roadmap are provided.

Why it matters – Video diffusion is notoriously slow. FastVideo’s sparse‑attention and data‑free distillation pipelines cut denoising time by >50×, making real‑time generation feasible on consumer hardware and opening the door to interactive video‑AI applications.