UVA-Computer-Vision-Lab/OmniShotCut

What it solves

OmniShotCut is designed to solve the problem of Shot Boundary Detection (SBD) across diverse video sources, including anime, vlogs, games, and sports. It specifically addresses the challenge of accurately identifying both sudden jump cuts and gradual transitions (such as dissolves, fades, and wipes) while reducing false positives caused by continuous camera motion or lighting changes.

How it works

The project implements a Shot-Query-based Video Transformer. This architecture allows the model to recognize relational patterns in video frames to distinguish between actual shot changes and visual noise. It can be deployed as a Python library via pip and supports inference on video files, numpy arrays, or torch tensors.

Who it’s for

This tool is for developers and researchers working with video processing, video editing automation, and data curation for video-based AI models (such as world-model pretraining or fine-tuning data preparation).

Highlights

• Broad Compatibility: Works across various content types like screen recordings, shorts, and anime.
• Holographic Detection: Detects both clean cuts and complex transitions (dissolves, fades, wipes).
• Robustness: The v1.5 model specifically reduces false cuts triggered by strobe lighting or camera movement.
• Flexible Input: Supports direct processing of video files or raw tensor/numpy frame data.