OpenGVLab/VideoChat-Flash

[ICLR2026] VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling

What it solves

VideoChat-Flash addresses the challenge of processing ultra-long videos (up to three hours) while maintaining high inference speed and accuracy. It solves the problem of computational inefficiency in long-context video modeling by reducing the number of tokens required to represent video frames.

How it works

The project implements a hierarchical compression architecture that encodes each video frame into just 16 tokens. This approach allows the model to handle massive inputs—up to 10,000 frames—while remaining 5 to 10 times faster than previous iterations. It is built using a video encoder (InternVideo) and a Large Language Model (Qwen).

Who it’s for

This is for researchers and developers working on multimodal large language models (MLLMs) specifically focused on long-form video understanding, temporal localization, and high-efficiency video analysis.

Highlights

  • Ultra-long context support: Capable of processing videos up to three hours long.
  • High efficiency: Encodes frames into 16 tokens, significantly increasing inference speed.
  • Strong retrieval performance: Achieves 99.1% accuracy on needle-in-a-haystack evaluations for 10,000 frames.
  • Diverse model options: Offers multiple model sizes (2B and 7B) and specialized versions for super long video input or short-term temporal understanding.

Related

  • Project
    MCG-NJU/VideoChat3

    VideoChat3 is a 4B generalist video MLLM designed for efficient video understanding, covering everything from subtle motion and long-form reasoning to live stream proactive responses.

  • Project
    OpenMOSS/MOSS-VL

    MOSS‑VL is an open‑weight 11 B‑parameter family of video‑language models that can understand and converse about live video streams in real time. It uses a cross‑attention architecture with timestamped frames and a 3‑D rotary embedding (XRoPE) to achieve low‑latency, interruptible dialogue, proactive silence, and dynamic correction. Three variants are released (Realtime, Instruct, Base), with quantized checkpoints for single‑GPU use, integration with FlashAttention‑3, SGLang, ms‑swift, and LoRA fine‑tuning. The project includes benchmarks showing state‑of‑the‑art streaming performance and provides scripts for both realtime and offline inference.

  • Project
    Zefan-Cai/KVCache-Factory

    A unified playground for implementing and evaluating KV cache compression, retrieval, merging, and quantization methods to optimize long-context LLM inference memory usage.

  • Project
    Robbyant/lingbot-map

    A feed-forward 3D foundation model for streaming 3D reconstruction that enables real-time generation of point clouds and camera trajectories from video.