taco-group/SparkVSR

[ECCV 2026] SparkVSR: Interactive Video Super-Resolution via Sparse Keyframe Propagation

What it solves

SparkVSR addresses the "black box" nature of traditional Video Super-Resolution (VSR) models, where users cannot correct artifacts or control the output. It provides an interactive framework that allows users to guide the restoration process using a small set of high-quality keyframes, ensuring the final video is grounded in both the original motion and the user's desired visual quality.

How it works

SparkVSR uses a two-stage training pipeline based on the CogVideoX1.5-5B-I2V model:

  1. Latent-Space Adaptation: The model learns to fuse low-resolution (LR) video latents with sparsely encoded high-resolution (HR) keyframe latents to enable robust propagation across the sequence.
  2. Pixel-Space Refinement: The model refines perceptual details in pixel space, balancing adherence to the provided keyframes with the ability to perform "blind" restoration when keyframes are missing or imperfect.

At inference, users can provide keyframes generated by any off-the-shelf image super-resolution (ISR) model or API. The system propagates these priors across the entire video while maintaining temporal consistency.

Who it’s for

This project is for researchers and developers working on video restoration, old-film restoration, and video style transfer who need controllable, high-quality upscaling of low-resolution video sequences.

Highlights

  • Interactive Control: Allows the use of sparse keyframes as control signals to guide the super-resolution process.
  • Flexible Reference Modes: Supports reference frames from commercial APIs (e.g., nano-banana-pro) or open-source models (e.g., PiSA-SR), as well as a no-reference fallback mode.
  • Temporal Consistency: Improves temporal stability and restoration quality over baselines, with significant gains in metrics like CLIP-IQA and DOVER.
  • Generic Framework: Can be applied to tasks beyond VSR, such as style transfer and old-film restoration.