meangrinch/MangaTranslator

MangaTranslator – One‑click AI‑powered manga translation

What it is

• An open‑source Python app that takes manga/comic pages, detects speech bubbles and any outside‑panel text, removes the original lettering, translates the text, and re‑renders the page in the target language. It stitches together several modern ML components (YOLO for detection, OCR, LLMs for translation, and diffusion models for in‑painting) into a single, easy‑to‑use Web UI (Gradio) or CLI.

Why it matters

• Scanlation (manual manga translation) is labor‑intensive. MangaTranslator automates the whole pipeline, letting users translate 60+ languages with a single click while preserving the artwork.

Key features

• Detection – Speech‑bubble and outside‑bubble text detection via YOLOv8, RT‑DETR‑v2, and Meta’s SAM 2/3 segmenters.
• Cleaning – In‑paints removed text using FLUX diffusion models or a fast OpenCV fallback.
• OCR & translation – Uses Manga‑OCR, PaddleOCR‑VL, and any LLM provider you configure (Google, OpenAI, Anthropic, DeepSeek, or a local llama.cpp‑compatible endpoint). Supports 60+ languages.
• Rendering – Custom font packs, word‑wrap, alignment, and optional background‑art restoration.
• Upscaling – Optional 2× AnimeSharpV4 upscaler for text regions or whole pages.
• Interfaces – Graphical Web UI (Gradio) for drag‑and‑drop, plus a full‑featured CLI for scripting and batch processing (folders, ZIP/CBZ archives).
• Flexibility – Numerous config knobs (provider keys, HF token for outside‑bubble models, cleaning‑only mode, font directories, etc.).

Quick start

1. Download – Grab the portable build (Windows) or clone the repo.
2. Install – Create a Python 3.10+ venv, install PyTorch (CUDA 13.0 for GPU) and the requirements.txt.
3. Configure – In the Web UI go to Config → Translation and add an LLM API key, or set environment variables (GEMINI_API_KEY, HF_TOKEN). Enable outside‑bubble detection by providing a Hugging Face token.
4. Translate – Upload an image (or a folder/CBZ) and click Translate; or run:

   python main.py \
     --input "page.jpg" \
     --input-language Japanese \
     --output-language English \
     --font-dir fonts/KomikaHand \
     --provider Google --google-api-key <key> \
     --osb-enable --osb-hf-token <token>
   

Hardware notes – GPU with CUDA 13 (or macOS/CPU fallback) is recommended; the README links a detailed hardware‑requirements doc.

License & credits

• Apache‑2.0 license.
• Built on a collection of community models (YOLO, SAM, FLUX, Manga‑OCR, etc.) and libraries such as torch, gradio, opencv, and stable‑diffusion.cpp.

Where to learn more

• Full docs: Installation, Configuration, CLI usage, font handling, troubleshooting, and hardware requirements are all in the docs/ folder.
• Support the author via Ko‑fi if you find it useful.