manga-ocr: a Japanese OCR engine optimized for manga text bubbles and complex layouts

What it solves

Manga OCR provides high-quality optical character recognition (OCR) specifically optimized for Japanese text in manga. It addresses common challenges in manga OCR, such as vertical and horizontal text, furigana (small phonetic guides), text overlaid on complex images, diverse font styles, and low-quality image sources.

How it works

The project uses a custom end-to-end model built with the Transformers' Vision Encoder Decoder framework. Unlike traditional OCR models that split text into lines, Manga OCR can recognize multi-line text in a single forward pass, allowing it to process entire text bubbles at once.

Who it’s for

It is designed for users who want to extract Japanese text from manga, novels, or video games. It is particularly useful for those using a reading and mining workflow with tools like ShareX, Flameshot, and dictionary apps like Yomitan.

Highlights

• own custom end-to-end model based on Vision Encoder Decoder
• Supports both vertical and horizontal Japanese text
• Handles furigana and text overlaid on images
• Processes multi-line text bubbles in a single pass
• Can run in the background to process images from the clipboard or a specific folder