bakrianoo/mazinger

End-to-end video dubbing pipeline

What it solves

Mazinger Dubber provides an end-to-end pipeline for dubbing videos into different languages. It automates the complex process of downloading a video, transcribing the speech, translating the text, cloning the original speaker's voice (or using a preset), and generating a final dubbed video or audio file while maintaining synchronization with the original timing.

How it works

The system chains ten resumable stages: Download, Transcribe, Thumbnails, Describe, Review, Translate, Re-segment, Speak, Assemble, and Subtitle. It uses voice-activity detection on a vocals stem (separated via Demucs) to create a speech map, ensuring dubbed lines start and end exactly where the original speech did. The pipeline supports various backends for transcription (CohereX, Faster Whisper, Deepgram) and TTS (Qwen3-TTS, OmniVoice), and includes a Gradio-based web UI called Mazinger Studio for visual management and chunk-level editing.

Who it’s for

Content creators, translators, and developers who need to localize video content with high-fidelity voice cloning and precise timing without manually managing multiple separate AI tools.

Highlights

  • One-Command Workflow: Handles everything from video download to final assembly in a single command or via a Web UI.
  • Voice Cloning: Can automatically clone the original speaker's voice from the source video or use pre-defined voice themes and profiles.
  • Precise Synchronization: Syncs dubbed audio to the original speech spans rather than just subtitle slots, preserving original background music and effects.
  • Chunk-Level Editor: Allows users to manually correct transcriptions, translations, or timing for specific segments and re-dub only the changed parts.
  • Flexible Backends: Supports local LLMs via Ollama and multiple transcription engines to balance between privacy, cost, and GPU availability.

Related

  • Project
    Huanshere/VideoLingo

    An AI-powered video translation and dubbing tool that automates speech recognition, subtitle translation, and voiceover generation in a single pipeline.

  • Project
    buxuku/SmartSub

    SmartSub is an open-source desktop tool that packages video-to-subtitle transcription, translation, proofreading, TTS dubbing with voice cloning, and subtitle burning into a single pipeline, supporting both local offline processing and AI-agent automation via MCP/CLI.

  • Project
    liuzhao1225/YouDub-webui

    YouDub WebUI is an open‑source, plugin‑based web application that localizes videos: it transcribes speech with Whisper, translates via an OpenAI‑compatible API, and generates sentence‑by‑sentence dubbed audio with VoxCPM2, optionally keeping background music. It can also burn translated subtitles onto the video. Built on the Cordis plugin framework, every component (ASR, translation, TTS, alignment, UI) is a replaceable plugin, allowing custom models or services. The repo provides scripts to install Node and Python dependencies, auto‑download default models, and run a host backend plus a Next.js front‑end. Supports MP4/MOV/MKV/WEBM up to 4 GiB, 10 min, and runs on CPU, CUDA, Apple MPS, or AMD ROCm. Licensed under Apache 2.0, with a QQ community for support.

  • Project
    abus-aikorea/voice-pro

    An AI-powered multimedia tool that integrates speech recognition, translation, and voice cloning for automated dubbing and multilingual content creation.

  • Project
    WEIFENG2333/VideoCaptioner

    An all-in-one video subtitle tool that uses LLMs for speech recognition, subtitle optimization, and translation to automate the video captioning process.