hgmzhn/manga-translator-ui

What it solves

It automates the complex process of translating manga, which typically requires several manual steps: detecting text in images, performing Optical Character Recognition (OCR), translating the text, removing the original text (inpainting), and typesetting the translated text back into the image bubbles.

How it works

The tool integrates multiple AI models and services into a single pipeline:

• Detection & OCR: Uses models like PaddleOCR and MangaOCR to identify and extract text from manga pages.
• Translation: Connects to LLMs and translation engines including OpenAI (GPT-4o), Gemini, and Sakura to translate text while maintaining consistency through AI-extracted terminology.
• Image Processing: Employs models like Real-CUGAN and MangaJaNai for super-resolution and image cleaning.
• Typesetting: Features a smart bubble layout system that automatically fits text into speech bubbles, supports both horizontal and vertical text, and uses HanLP for semantic line breaking.
• Editing: Provides a visual editor for manual adjustments to text boxes, colors, fonts, and masks.

Who it’s for

• Manga translators and scanlation groups who want to speed up their workflow.
• Readers who want to translate manga in their own language for personal use.
• Developers looking for a comprehensive manga translation pipeline.

Highlights

• End-to-end pipeline: One-click flow from detection to final rendering.
• Multimodal AI: Supports GPT-4o and Gemini for high-quality, context-aware translations.
• Visual Editor: Full rich-text editing capabilities with support for Ruby characters and vertical text.
• Smart Layout: Automatic text indentation and scaling to fit bubble shapes.
• Batch Processing: Ability to process entire folders and export to PSD for further professional editing.