meangrinch/MangaTranslator

What it solves

It automates the tedious process of translating manga and comics. Instead of manually erasing text and re-typing translations, this tool handles the entire pipeline: detecting text bubbles, removing the original text, translating the content, and rendering new text back into the image.

How it works

The application uses a multi-stage AI pipeline:

1. Detection: It identifies speech bubbles and text outside of bubbles using models like YOLO and SAM 2.1/3.
2. Cleaning: It removes original text (inpainting) using FLUX models or OpenCV to create a clean canvas.
3. Translation: It performs OCR and translates the text into one of 60 supported languages using LLMs or VLMs (via various API providers or local endpoints).
4. Rendering: A custom engine places the translated text back into the bubbles with adjustable alignment and custom font packs.
5. Upscaling: It can upscale the text regions or the entire page using 2x-AnimeSharpV4 for better visual quality.

Who it’s for

• Manga and comic readers who want to read untranslated works in their own language.
• Scanlation groups looking to automate the cleaning and typesetting phases of their workflow.

Highlights

• One-click automation: Handles the full translation process without requiring manual intervention.
• Broad language support: Supports translation across 60 different languages.
• Flexible LLM integration: Compatible with a wide range of providers including OpenAI, Google, Anthropic, and local endpoints via OpenAI-Compatible API.
• Advanced cleaning: Uses high-end generative models (FLUX) for high-quality text removal.
• Dual Interface: Offers both a user-friendly Gradio web UI and a command-line interface (CLI) for batch processing.