a312863063/Video-Auto-Wipe

What it solves

This project provides a way to automatically remove unwanted fixed-pattern content from videos, such as subtitles, logos, and animated icons, and then fill in the missing areas to restore the video's appearance.

How it works

The system uses a two-step training process. First, it employs temporal perception training to help the model identify the foreground elements (the content to be erased). It then integrates this into an erasure model for end-to-end fine-tuning.

For different types of content, it uses different detection methods:

• Subtitles: Identified by recognizing areas with consistent text styles.
• Logos: Identified by detecting pixel blocks that remain static over time.
• Dynamic Icons: Identified by detecting fixed pixel blocks that flicker or move dynamically.

Who it’s for

Users looking to clean up videos by removing distracting overlays, subtitles, or watermarks without manual masking.

Highlights

• Automatic Detection: Automatically perceives the location of subtitles and logos rather than requiring manual input.
• Temporal Consistency: Uses temporal search and completion capabilities to restore details and text-free areas.
• Custom Training Data: Built using a dataset of over 3,000 movie and variety show clips with synthetic foreground overlays.