liuzhao1225/YouDub-webui

YouDub WebUI – Open‑source video localization workbench

What it is

• A desktop‑style web application that lets you import a video, automatically transcribe the speech, translate it, and generate a dubbed version with subtitles. It works sentence‑by‑sentence, cloning the original voice timing for each translated line.

Key capabilities

Mode	Output
Original audio + subtitles	Keeps the source soundtrack and burns translated subtitles onto the video
Dub only	Generates a voice‑over in the target language (optionally keeping background music)
Dub + subtitles	Produces a video with both dubbed audio and translated subtitles (with optional forced‑alignment)

• Sentence‑level voice cloning – each translated sentence is synthesized using the exact audio segment of the source sentence as a reference, preserving timing and prosody.
• Accurate segmentation – Whisper provides word‑level timestamps; NLTK Punkt groups them into natural sentences.
• Subtitle handling – splits on punctuation, merges very short fragments, and hides trailing punctuation for clean display.
• Task manager – shows video cover, processing steps, errors, and final assets; supports cancel, retry, re‑run from a specific step, and download via API.

Architecture

• Built on Cordis, a plugin‑based framework. The whole stack – client UI, authentication, task engine, workflow, model providers, and the Python compute process – are plugins that can be swapped.
• Workflow plugins declare the sequence of operations (ASR → translate → TTS → align → mix).
• Provider plugins implement a concrete operation (e.g., Whisper ASR, VoxCPM2 TTS, Qwen3 forced aligner). Adding a new model only requires a plugin that follows the contract.
• The heavy media work (FFmpeg, model inference) runs in a managed Python subprocess communicating over stdio.

Getting started (macOS / Linux, also works on Windows with CUDA)

1. Clone & install

   git clone --branch main --recurse-submodules https://github.com/liuzhao1225/YouDub-webui.git
   cd YouDub-webui
   npm ci               # install Node deps
   npm --prefix apps/web ci
   

2. Create config

   cp .env.example .env
   ln .env env.txt   # both files are read, env.txt is ignored by Git
   

3. First run – the app will automatically download the default models (Whisper‑tiny, VoxCPM2, Demucs, Qwen3‑Aligner) and verify them.
4. Start the services

   npm run build:plugins   # compile built‑in plugins
   npm start               # host (backend + plugin manager)
   npm --prefix apps/web run dev -- --hostname 127.0.0.1 --port 3000   # front‑end
   

5. Open http://127.0.0.1:3000, configure an OpenAI‑compatible translation API key in Settings, then import a video and launch a task.

Extending the platform

• Install, enable, disable, or remove plugins via the npm run plugins CLI (local folder, Git repo, or npm package).
• Create a workflow plugin to define a new processing pipeline.
• Create a provider plugin (JavaScript/TypeScript or pure Python) to plug in a different ASR, translation, or TTS engine.
• The SDK (packages/sdk) contains the shared types and contracts; it is bundled with the repo but not yet published to npm.

Supported formats & limits

• Video: MP4, MOV, MKV, WEBM up to 4 GiB, 10 min, 1920 px max, 60 fps.
• Languages: English, Chinese, Japanese out‑of‑the‑box; additional languages depend on the translation/TTS provider.
• Hardware: Detects CUDA, Apple MPS, AMD ROCm, with CPU fallback.

License & community

• Apache 2.0 license.
• QQ group 618246010 for discussion and support.
• Releases, issue tracking, and PRs are hosted on GitHub; a Chinese mirror is available on AtomGit.

Bottom line YouDub WebUI is a fully open‑source, plugin‑centric solution for turning any short video into a localized version with dubbed audio and subtitles, leveraging state‑of‑the‑art open models (Whisper, VoxCPM2, Demucs, Qwen3) and offering a clear path to swap in your own models or services.