hao-ai-lab/FastVideo

A unified inference and post-training framework for accelerated video generation.

What it solves

FastVideo is designed to accelerate video generation by providing a unified framework for both post-training and real-time inference. It addresses the high computational cost and slow generation speeds typically associated with high-resolution video diffusion models.

How it works

The framework employs several optimization techniques to reduce latency and increase throughput:

  • Post-Training & Distillation: It supports full and LoRA finetuning for video DiTs, as well as Distribution Matching Distillation (DMD2) and sparse distillation to achieve significant denoising speedups (over 50x).
  • Attention Optimizations: It implements Video Sparse Attention (VSA) and Sliding Tile Attention to reduce the complexity of the attention mechanism.
  • Inference Acceleration: It utilizes sequence parallelism for distributed inference and supports multiple high-performance attention backends.
  • Hardware Compatibility: The system is optimized for a wide range of hardware, including NVIDIA GPUs (H100, A100, 4090) and Apple Silicon (via MLX).

Who it’s for

It is intended for AI researchers and developers building video generation models, as well as those looking to deploy real-time video generation and editing applications (such as the included Dreamverse platform).

Highlights

  • Real-time Capabilities: Includes Dreamverse for "vibe directing" and streaming video in real-time.
  • Extreme Speedups: Sparse distillation techniques can achieve over 50x denoising speedup.
  • Broad Hardware Support: Native support for Linux, Windows, and macOS (Apple Silicon).
  • Comprehensive Tooling: Provides a full pipeline from data preprocessing and training/finetuning to optimized inference.

Related

  • Project
    ModelTC/LightX2V

    LightX2V is an open‑source inference framework for fast image and video generation (T2V, I2V, T2I, I2I, T2AV, etc.). It supports many state‑of‑the‑art models (MiniMax‑H3, Wan, HunyuanVideo, Qwen‑Image, etc.) and provides speed‑up techniques such as 4‑step distilled LoRAs, FP8/NVFP4 quantization, tensor/sequence parallelism, and disaggregated deployment on a wide range of hardware. Benchmarks show up to 3.9× faster inference than competing frameworks on H100 GPUs. The project offers Docker images, pip install, Gradio/ComfyUI front‑ends, extensive docs, and an online demo.

  • Project
    OpenImagingLab/FlashVSR

    FlashVSR is a one-step streaming diffusion framework for real-time video super-resolution that achieves high-speed upscaling and scalability to ultra-high resolutions.

  • Project
    NVlabs/FastGen

    A PyTorch-based framework for accelerating diffusion models using various distillation and acceleration techniques for image and video generation.

  • Project
    thu-ml/TurboDiffusion

    TurboDiffusion is a video generation acceleration framework that speeds up end-to-end diffusion generation by 100-200x using attention acceleration and timestep distillation.

  • Project
    thu-ml/Causal-Forcing

    A framework for high-quality real-time interactive video generation using Causal ODE or Causal Consistency Distillation to enable efficient few-step autoregressive diffusion.