SmartSub: an all-in-one desktop tool for video transcription, translation, AI dubbing, and subtitle burning

What it solves

SmartSub provides a complete, automated pipeline for video processing, solving the fragmented workflow of converting speech to text, translating subtitles, and generating AI voiceovers. It allows users to transform foreign language videos into localized content with synchronized dubbing and burned-in subtitles without needing multiple disconnected tools.

How it works

The application operates as a multi-step pipeline: it downloads videos (via yt-dlp/lux), transcribes audio using local engines like whisper.cpp or FunASR, translates text using various services (including local LLMs via Ollama), and performs TTS (Text-to-Speech) for dubbing. Finally, it uses FFmpeg to burn subtitles into the video or mux them as soft subtitles.

Who it’s for

• Content Creators looking to localize videos for international audiences.
• Language Learners needing bilingual subtitles for foreign lectures or videos.
• Podcast/Meeting Organizers who need to batch-transcribe audio into SRT files.
• Privacy-Conscious Users who prefer running transcription and TTS models locally on their own hardware.

Highlights

• Full Pipeline: Covers downloading, transcription, translation, proofreading, dubbing, and video synthesis.
• Local-First Privacy: Supports offline processing using local models (Whisper, Kokoro, VITS) to keep files on the user's machine.
• Hardware Acceleration: Supports NVIDIA CUDA, AMD/Intel Vulkan, and Apple Silicon (Core ML/Metal).
• Extensive Integrations: Connects to 20+ translation services and various cloud ASR/TTS providers.
• Voice Cloning: Features zero-shot voice cloning using local or cloud-based engines.