buxuku/SmartSub

What it solves

SmartSub is an open-source desktop application that packs a complete audio-video subtitle and dubbing pipeline into one tool: speech-to-text, subtitle translation, proofreading, TTS voice synthesis with voice cloning, and subtitle burning. It solves the problem of dealing with foreign-language videos, creating multilingual content, adding voiceovers, and automating video production without juggling multiple separate tools or sending files to the cloud. The entire pipeline can run fully free and offline using local models.

How it works

The core workflow is: download online videos (from platforms like YouTube or Bilibili) or import local files, transcribe speech into subtitles using local ASR engines (whisper.cpp, faster-whisper, FunASR, Qwen3-ASR, FireRedASR, NVIDIA Parakeet, etc.), translate subtitles via built-in free services or 20+ cloud/LLM providers, proofread and refine subtitles in an integrated editor, synthesize TTS dubbing (local Kokoro/VITS voices, ZipVoice zero-shot voice cloning, or cloud services like Edge TTS, Azure, ElevenLabs), and finally burn subtitles into the video (hard or soft subs) with ffmpeg. Beyond the GUI, SmartSub includes an AI copilot with context awareness, multimodal screenshot diagnosis, and direct tool-calling, plus a full MCP server and CLI exposing 111 tools, enabling external AI agents (Cursor, Claude Code, Codex) or shell scripts to drive the whole pipeline headlessly. Local processing keeps files on-device; cloud services are opt-in with privacy confirmations. GPU acceleration is supported via CUDA, Vulkan, and Apple Core ML/Metal.

Who it's for

SmartSub is built for content creators, translators, educators, podcasters, video editors, and developers who need to add subtitles or dubbing to videos, translate existing subtitles, repurpose content for international audiences, or automate media workflows. It suits both non-technical users who want a friendly GUI and developers/AI enthusiasts who want to connect it to AI agents or CI/CD scripts.

Highlights

• All-in-one pipeline: download, transcribe, translate, proofread, dub with voice cloning, and burn subtitles in one app
• Fully local and free path: offline transcription, built-in free translation sources, local TTS with voice cloning, local ffmpeg—no API key or usage limits for local steps
• AI copilot with deep context awareness: understands current playback, selected subtitle lines, task logs, and can edit subtitles or troubleshoot via natural language
• Multimodal screenshot diagnosis: capture the workspace and let a vision LLM analyze error popups and UI states
• Agentic automation: 111 MCP tools and CLI commands let external AI agents (Cursor, Claude Code, Codex) or terminal scripts fully automate production pipelines
• Broad model support: 8 ASR engines, 20+ translation services, 6 TTS providers, and multiple LLM endpoints (DeepSeek, Qwen, Gemini, OpenAI-compatible, etc.)
• Privacy-focused: local processing, optional cloud, no telemetry, no usage data collection
• Cross-platform (Windows, macOS, Linux) with GPU acceleration and a one-click installer / Homebrew support