chenfengxu714/StreamDiffusionV2

What it solves

StreamDiffusionV2 addresses the latency and throughput challenges of generating high-quality video in real-time. It enables dynamic and interactive video-to-video generation, allowing creators and platforms to produce streaming video content with high frames-per-second (FPS) across various GPU configurations.

How it works

The system implements a streaming diffusion pipeline that processes video in chunks rather than as a whole. It utilizes a Causal DiT (Diffusion Transformer) and a Ring-buffer KV Cache to enable efficient sliding window attention. The pipeline supports multiple execution modes: single-GPU streaming, single-GPU without stream-batching, and multi-GPU pipeline-parallel inference. To further increase speed, it offers optional acceleration via TensorRT and the lightweight TAEHV-VAE decoder.

Who it’s for

This tool is designed for AI researchers, video creators, and developers building real-time interactive video applications who need high-throughput video generation on NVIDIA GPUs.

Highlights

• High Performance: Optimized for real-time streaming with support for high FPS.
• Flexible Scaling: Scales from single-GPU setups to multi-GPU pipeline-parallel configurations.
• Hardware Optimized: Includes initial support for NVIDIA Blackwell GPUs and optional Flash-Attention integration.
• Interactive Capabilities: Designed specifically for dynamic and interactive video generation tasks.