xin1u/UltraFlash

What it solves

Ultra Flash addresses the challenge of generating high-resolution (1K and 2K) video in real-time. Traditional video generation models often struggle with the computational intensity required for high-resolution output, typically resulting in slow generation speeds. This framework enables streaming video generation at approximately 30 FPS for 1K resolution and 18 FPS for 2K resolution on a single GPU.

How it works

The system uses a cascaded architecture that builds upon a low-resolution streaming generator. It processes video through four main stages: a self-forcing generator produces low-resolution latents, a causal latent upsampler increases the resolution, a sparse super-resolution (SR) Diffusion Transformer (DiT) refines the details, and a tiny decoder converts these latents into high-resolution pixel output. The framework incorporates dynamic cache management and block-sparse attention to maintain temporal coherence and speed while minimizing memory overhead.

Who it’s for

This project is for researchers and developers working on real-time generative video applications who need to balance high visual fidelity with low-latency performance. It is designed for users with high-end NVIDIA hardware (24GB+ VRAM) who want to deploy or experiment with efficient, streaming-capable video generation pipelines.

Highlights

• Achieves real-time performance: 1K at ~30 FPS and 2K at ~18 FPS on a single GPU.
• Includes Dynamic Cache Management (DCM) to optimize inference speed with negligible quality loss.
• Supports both short and long video generation (up to ~10 seconds).
• Provides a modular pipeline including a tiny decoder for faster latent-to-pixel conversion compared to standard VAEs.