microsoft/DCVC

Deep Contextual Video Compression

What it solves

DCVC-UF addresses the high computational complexity of neural video codecs (NVCs), which often makes them impractical for real-world deployment. It aims to provide a high compression ratio while achieving ultra-fast encoding and decoding speeds.

How it works

The project implements a chunk-based coding framework. Instead of processing frames sequentially, it encodes a chunk of multiple frames into a single compact latent representation and decodes them simultaneously. This is supported by:

  • Cross-frame interaction modules: Used for joint spatial-temporal modeling.
  • Frame-specific decoders: Enable parallel reconstruction of frames.
  • Streamlined entropy coding: A mechanism that reduces decoding overhead by consolidating bit-stream interactions into a single step.

Who it’s for

Researchers and developers working on video compression, neural video codecs, and high-performance video processing pipelines.

Highlights

  • Ultra-Fast Speed: Significantly outperforms previous leading codecs in encoding and decoding throughput.
  • Wide Bitrate Range: A single model with 64 QP levels allows for fine-grained bitrate adjustments.
  • Versatile Color Spaces: Supports both YUV420 and RGB content coding.
  • High Efficiency: Achieves bitrate reductions compared to VTM-17.0 while maintaining high quality.

Related

  • Project
    OpenGVLab/VideoChat-Flash

    VideoChat-Flash is a multimodal large language model designed for efficient long-context video understanding, capable of processing videos up to three hours long using hierarchical compression.

  • Project
    microsoft/mlvc

    A neural video codec that provides real-time encoding and decoding performance on consumer NPUs from Apple, Intel, and Qualcomm.

  • Project
    descriptinc/descript-audio-codec

    A high-fidelity neural audio codec that compresses 44.1 kHz audio into discrete codes at 8 kbps, providing approximately 90x compression for speech, music, and environmental sounds.

  • Project
    FAST-Imaging/FAST

    An open-source framework for high-performance medical image processing, neural network inference, and real-time visualization using CPUs and GPUs.

  • Project
    NVIDIA-RTX/RTXNTC

    A neural texture compression SDK that compresses multiple PBR texture channels into a single neural set to reduce VRAM and bandwidth usage in 3D rendering.