dmMaze/BallonsTranslator

What it solves

BallonsTranslator is a deep learning-assisted tool designed to automate the translation of comics and manga. It streamlines the tedious process of removing original text from speech bubbles, translating the content, and typesetting the translated text back into the image while maintaining the original layout.

How it works

The tool integrates several AI-driven modules to handle the translation pipeline:

• Text Detection & OCR: It identifies text blocks in images (supporting Japanese and English) and extracts the text using various OCR models (such as manga-ocr or PaddleOCR).
• Image Inpainting: It removes original text using deep learning models (AOT, LaMa) or traditional algorithms (PatchMatch) to clean the background.
• Translation: It supports a wide array of translation engines, including LLMs (OpenAI-compatible APIs, Sakura-13B-Galgame), specialized translators (DeepL, Sugoi), and general services (Google, Papago).
• Typesetting: It automatically fills translated text back into the bubbles, estimating color, alignment, and orientation based on the original layout.

Who it’s for

It is primarily for manga translators, scanlators, and readers who want to quickly translate comics or webtoons into their own language with minimal manual effort.

Highlights

• One-click translation: Automates the entire pipeline from detection to typesetting.
• Rich editing suite: Includes a WYSIWYG text editor, mask editing, and a healing brush for manual image cleanup.
• Extensive translator support: Compatible with various local and cloud-based LLMs and translation APIs.
• Flexible output: Supports importing and exporting Word documents for text-based translation workflows.
• Hardware acceleration: Supports GPU acceleration for NVIDIA and Apple Silicon, with community-driven support for AMD ROCm via ZLUDA.