dmMaze/BallonsTranslator

What it solves

BallonsTranslator is a deep learning-assisted tool designed to automate the translation of comics and manga. It streamlines the tedious process of translating visual media by combining text detection, optical character recognition (OCR), image inpainting (removing original text), and machine translation into a single workflow.

How it works

The tool operates through a pipeline of four main modules:

1. Text Detection: Identifies text areas in images (supporting Japanese and English).
2. OCR: Recognizes the text within those areas and extracts colors.
3. Image Inpainting: Erases the original text using algorithms like AOT, LaMa, or PatchMatch to restore the background.
4. Translation: Translates the text using various engines (Google, DeepL, Papago, or LLMs).

For advanced users, it integrates LLMs with visual context, translation history (to maintain consistency in names and tone), and reusable glossaries to improve translation quality. It also provides a rich text editor for manual adjustments to font styles, effects, and layout.

Who it’s for

It is primarily for manga/comic translators, scanlation groups, and readers who want to translate comics from languages like Japanese or English into their own language with minimal manual effort.

Highlights

• One-click translation: Automates the entire process from detection to typesetting.
• LLM Integration: Supports context-aware translation using history, summaries, and visual inputs to ensure narrative consistency.
• Advanced Image Editing: Includes mask editing and a specialized repair brush for cleaning up text areas.
• Flexible Translation Engines: Supports a wide array of providers including local models (Sakura-13B), API-based services (OpenAI, DeepLX), and specialized manga OCRs.
• Rich Text Tooling: Features WYSIWYG editing with support for text deformation, special effects, and batch formatting.