cvlab-kaist/VideoMaMa

Official implementation of "VideoMaMa: Mask-Guided Video Matting via Generative Prior", CVPR 2026

What it solves

Video matting is the task of extracting a precise foreground (e.g., a person or object) from each video frame, including accurate edges and fine details like hair or fur. Traditional methods often struggle with video domains they haven't seen before, producing unstable or coarse results. VideoMaMa addresses this by using a video generative prior—knowledge from a video diffusion model—to guide the matting process, leading to more stable and detailed matting across a wide range of video content.

How it works

The framework takes an input image and a user-provided mask (a rough outline of the foreground) as guidance. It uses Stable Video Diffusion (a pre-trained video generation model) as the generative prior, and incorporates SAM2 (Segment Anything 2) for training the matting head. The model output is a fine-grained alpha matte that separates foreground from background with precise edges. The repository includes scripts for setup, inference (using a single command), and training, along with a Hugging Face demo.

Who it's for

Researchers and developers working on video editing, VFX, or any application requiring high-quality automatic foreground extraction. It is also useful for those interested in leveraging generative models for vision tasks. The project provides code, checkpoints, and an interactive demo, making it accessible for both experimentation and integration.

Highlights

  • Leverages a video generative prior from Stable Video Diffusion for robust matting across diverse domains.
  • Fine-grained matting quality preserving details like hair and edges.
  • Includes pre-trained model checkpoint on Hugging Face and an interactive web demo.
  • Provides full training code and data pipeline documentation.
  • Integrated into third-party tools like ComfyUI, CorridorKey, and Sammie-Roto-2.
  • Published at CVPR 2026 with an open-source release under CC BY-NC 4.0 (code) and Stability AI community license (checkpoints).

Related

  • Project
    TencentARC/WorldCrafter

    WorldCrafter is a research‑grade video generation model that creates camera‑controllable, consistent videos from a single image or a text prompt. It stores an implicit 3‑D‑aware memory so the scene stays coherent across viewpoints. The repo provides a full‑size Base model and a distilled Fast model, scripts for image‑to‑video and text‑to‑video inference, automatic prompt generation via Qwen‑VL, and an open‑source interactive web demo. Setup uses Python 3.11 with PyTorch 2.10 (CUDA 12.8) via uv or conda, and model weights are hosted on Hugging Face.

  • Project
    KnowClip-ai/knowclip

    An AI-powered tool that uses local ASR and LLMs to automatically identify and export highlight clips from long-form videos.

  • Project
    tschnz/Live-Video-Magnification

    A real-time video magnification tool that amplifies subtle motion and color changes using Eulerian video magnification techniques.

  • Project
    mrsarac/ff-tracking

    A Rust‑based demo that programmatically renders a short video of a virtual camera tracking every glyph an AI‑agent types in a terminal‑style UI. It uses the fframes library, a custom SkSL shader for realistic lens effects, and Python‑generated sound, all driven by deterministic JSON data.