OpenImagingLab/FlashVSR

⚡ FlashVSR – Real‑time diffusion‑based video super‑resolution

What it is

• A research‑grade library that upsamples low‑resolution video (typically 4×) to high‑resolution video using a diffusion model that has been heavily distilled for speed.
• The authors claim ~17 FPS on a single NVIDIA A100 for 768 × 1408 video, making it the first diffusion‑based VSR system that can run in (near) real‑time.

Why it matters

• Diffusion models give excellent visual quality but are usually far too slow for video. FlashVSR solves this with three engineering tricks:
  1. Three‑stage distillation pipeline – trains a large teacher model and then compresses it into a streaming‑friendly student that can process frames one‑by‑one.
  2. Locality‑Constrained Sparse Attention (LCSA) – only attends to a small spatial‑temporal neighbourhood, cutting redundant computation and narrowing the gap between training‑time and inference‑time resolutions.
  3. Tiny conditional decoder – a lightweight reconstruction head that restores fine details without the heavy cost of a full diffusion decoder.
• A new VSR‑120K dataset (120 k videos, 180 k images) is provided for large‑scale training.

Key components

Component	Role
Three‑stage distillation	Turns a heavyweight diffusion teacher into a streaming student that can be run frame‑by‑frame.
LCSA	Sparse attention mask that limits each token’s view to nearby pixels/frames, dramatically reducing FLOPs.
Tiny Conditional Decoder	Fast up‑sampling head that takes the distilled latent and produces the final high‑res frame.
Block‑Sparse‑Attention backend	External library (MIT‑HAN‑Lab) that implements the custom sparse kernels needed for LCSA.
VSR‑120K dataset	Large, diverse video‑image collection used to train the model; released on Hugging Face.

Typical workflow

1. Install the Python package (pip install -e .) and the required Block‑Sparse‑Attention library.
2. Download the pre‑trained weights (v1.1 is the recommended, more stable version) from Hugging Face using Git LFS.
3. Run one of the provided inference scripts (infer_flashvsr_v1.1_full.py, *_tiny.py, or the long‑video variant) on a folder of low‑resolution video frames.
4. The script streams frames through the distilled diffusion model, applying LCSA on‑the‑fly, and writes out the up‑scaled video.

Use cases

• Content creators who need fast, high‑quality up‑scaling of 4K/8K footage for streaming platforms.
• Post‑production pipelines where real‑time preview of up‑scaled video is valuable.
• Research on efficient diffusion models, streaming video processing, or sparse attention mechanisms.

Installation highlights

# Clone repo
git clone https://github.com/OpenImagingLab/FlashVSR && cd FlashVSR

# Create conda env (Python 3.11)
conda create -n flashvsr python=3.11 && conda activate flashvsr

# Install package & deps
pip install -e .
pip install -r requirements.txt

# Install Block‑Sparse‑Attention (outside repo)
git clone https://github.com/mit-han-lab/Block-Sparse-Attention && cd Block-Sparse-Attention
pip install packaging ninja
python setup.py install

GPU note: Tested on NVIDIA A100/A800 (Ampere) and H200 (Hopper). Performance on consumer RTX 40/50 cards is undocumented.

Running inference (example)

cd examples/WanVSR
# download v1.1 weights with Git LFS first (see README)
python infer_flashvsr_v1.1_full.py   # full‑size decoder, best quality
# or
python infer_flashvsr_v1.1_tiny.py    # tiny decoder, faster, slightly lower quality

The scripts automatically locate the weight folder (FlashVSR‑v1.1/) and output an up‑scaled video.

Community & ecosystem

• Several ComfyUI wrappers exist, though early versions omitted the LCSA module and suffered quality loss.
• Third‑party cloud services (fal.ai, WaveSpeed AI, Segmind, etc.) host the model for easy web‑based inference.
• The authors provide a project page, Hugging Face model repos, and a downloadable dataset.

Citation

@article{zhuang2025flashvsr,
  title={FlashVSR: Towards Real-Time Diffusion-Based Streaming Video Super-Resolution},
  author={Zhuang, Junhao and Guo, Shi and Cai, Xin and Li, Xiaohui and Liu, Yihao and Yuan, Chun and Xue, Tianfan},
  journal={arXiv preprint arXiv:2510.12747},
  year={2025}
}

---

Bottom line: FlashVSR is a genuine, cutting‑edge AI project that makes diffusion‑based video up‑scaling fast enough for interactive use by combining distillation, sparse attention, and a tiny decoder. It is suitable for anyone needing high‑quality, real‑time video super‑resolution and for researchers exploring efficient diffusion architectures.