kotaro-kinoshita/yomitoku

What it solves

YomiToku is a Document AI engine specialized for Japanese text and layouts. It solves the problem of extracting structured information from images of documents, handling complex Japanese-specific challenges such as vertical writing, handwritten text, and intricate table structures that general OCR tools often struggle with.

How it works

The project uses four custom-trained AI models to perform a full pipeline of document analysis:

1. Text Detection: Locates where text is present in the image.
2. Text Recognition: Converts the detected text areas into strings, supporting over 7,000 Japanese characters.
3. Layout Analysis: Identifies the semantic structure of the document, including paragraphs, figures, and images.
4. Table Structure Recognition: Analyzes the grid and cell structure of tables to maintain the semantic relationship between data.

It offers multiple output formats including Markdown, HTML, JSON, CSV, and searchable PDFs. It also includes a "lite" mode for faster CPU-based inference.

Who it’s for

• Developers looking to integrate high-accuracy Japanese OCR and layout analysis into their applications.
• Researchers working with Japanese document digitization.
• Businesses needing to automate the extraction of data from forms, reports, and invoices (via the Table Semantic Parser and Extractor).

Highlights

• Japanese Specialization: High accuracy for vertical writing and handwritten Japanese text.
• Structured Extraction: Capable of converting tables into structured JSON or CSV while preserving the reading order.
• Flexible Extraction: Provides both rule-based extraction (for fixed forms) and LLM-based extraction (for unstructured documents) using a YAML schema.
• Hardware Efficient: Runs on GPUs with less than 8GB VRAM or on CPUs using lightweight models.
• Privacy-First Demo: Includes a browser-based demo (YomiToku Studio) where inference is performed locally via WebAssembly/WebGPU, ensuring images are not uploaded to a server.