Huanshere/VideoLingo

What it solves

VideoLingo is designed to overcome the limitations of standard machine translation for videos, which often result in stiff, multi-line subtitles and poor synchronization. It provides an all-in-one pipeline for high-quality video translation, localization, and dubbing that aims for "Netflix-quality" results, ensuring subtitles are single-line and translations are cinematic in quality.

How it works

The tool uses a multi-stage pipeline to process videos:

1. Transcription: It utilizes WhisperX for word-level, low-illusion subtitle recognition.
2. Segmentation: NLP and AI-powered tools segment subtitles for better readability.
3. Translation: A three-step "Translate-Reflect-Adaptation" process, combined with custom and AI-generated terminology, ensures coherent and natural translation.
4. Dubbing: The system integrates with various Text-to-Speech (TTS) providers such as GPT-SoVITS, Azure, and OpenAI to generate high-quality voiceovers.
5. Interface: Everything is managed through a Streamlit-based UI, allowing users to download YouTube videos via yt-dlp and control tasks (pause, resume, stop) at each step.

Who it’s for

Content creators and users who want to translate and localize videos for a global audience while maintaining professional-grade subtitle formatting and natural-sounding dubbing.

Highlights

• Netflix-standard subtitles: Guarantees single-line subtitles only, avoiding the cluttered look of multi-line text.
• Cinematic translation: Uses a reflective adaptation process to refine translation quality beyond simple machine translation.
• Comprehensive TTS support: Compatible with a wide range of dubbing engines including GPT-SoVITS, Azure, and OpenAI.
• Flexible deployment: Supports one-click startup via uv, Docker, or Conda.
• Integrated workflow: Handles everything from YouTube downloading to final dubbing in one interface.