zyddnys/manga-image-translator

What it solves

This project provides a one-click solution for translating text within images, specifically targeting content that is rarely professionally translated, such as comics, manga, and images from group chats or image boards. It handles the entire pipeline from text detection to final typesetting, supporting Japanese, Chinese (Simplified and Traditional), English, and 20 other languages.

How it works

The tool employs a multi-stage AI pipeline to process images:

1. Detection: Identifies text areas within the image.
2. OCR: Extracts the text from those detected areas.
3. Translation: Translates the extracted text using various supported translators (including options like Sugoi, Sakura, or OpenAI).
4. Inpainting: Removes the original text from the image (image repair) using models like lama_large to create a clean background.
5. Typesetting: Renders the translated text back onto the image, with options to adjust font size, font paths, and layout to fit the original text bubbles.

Who it’s for

• Manga and comic readers who want to translate foreign language content on their own.
• Users of image boards and group chats who seeking to understand Japanese or Chinese image-based text.
• Developers who want to integrate image translation capabilities via a CLI, Web UI, or API.

Highlights

• Full Pipeline: Combines text detection, OCR, translation, and inpainting into a single workflow.
• Flexible Deployment: Available as a local batch processor, a web server with a UI, or an API service.
• Extensive Language Support: Supports over 20 languages, with a primary focus on Japanese.
• Customization: Offers deep configuration for rendering, upscaling, inpainting, and OCR to improve translation quality.
• Docker Support: Provides pre-configured Docker images for both CPU and GPU acceleration.