notepower2k1/CapCap

What it solves

CapCap is a Windows application designed to simplify the video editing and localization process. It automates the tedious tasks of speech recognition, translation, dubbing, and visual editing, allowing users to create localized content for different languages (specifically English and Vietnamese) more efficiently.

How it works

The application follows a structured, step-by-step workflow: Preparation $\to$ Transcription $\to$ Translation $\to$ Text-to-Speech (TTS) $\to$ Export. It leverages several AI models and services for these tasks:

• Transcription: Uses Faster-Whisper or SenseVoice for speech-to-text.
• Translation: Supports various Cloud/API translation services, with Google Translate as a fallback.
• Dubbing: Integrates multiple TTS engines including Piper TTS, Edge TTS, CapCut TTS, and VieNeu TTS, and supports voice cloning.
• Visuals: Uses OCR to extract existing subtitles from videos.
• Export: Allows direct export to CapCut Drafts for advanced post-production, or direct render using hardware-accelerated encoding (NVIDIA NVENC / CPU libx264).

Who it’s for

Content creators and video editors who need to localize their videos into English or Vietnamese, adding subtitles and dubbing while maintaining a control over the timeline and visual elements like logos and masks.

Highlights

• Integrated AI Pipeline: Combines transcription, translation, and TTS in one tool.
• Voice Customization: Features speaker diarization to assign different voices to different people and voice cloning capabilities.
• Visual Editing Tools: Includes a timeline for managing subtitles, blur regions, logos, masks, and text.
• CapCut Integration: Ability to export directly to CapCut Drafts for further editing.
• Optimized Rendering: Offers multiple quality profiles (Low to Very High) with hardware acceleration support.