YomiToku – Japanese‑focused Document AI

YomiToku is an open‑source Python library that performs OCR, layout analysis, table‑structure parsing, and reading‑order estimation on Japanese (and English) document images. It ships four purpose‑trained models (text detection, text recognition, layout, and table semantics) and can run on GPU or CPU, with a lightweight “lite” mode for faster inference.

---

What it does

• Full‑text OCR – recognises >7 000 Japanese characters, supports handwritten, vertical, and mixed‑language text.
• Layout parsing – identifies paragraphs, figures, tables, and reading order, preserving the document’s visual structure.
• Table semantic parser – extracts tables as either key‑value pairs or grid data, outputting a structured JSON with cell IDs and coordinates.
• Extractor – rule‑based or LLM‑augmented extraction of fields from forms into YAML/JSON according to a user‑provided schema.
• Multiple export formats – markdown, HTML, JSON, CSV, searchable PDF, plus optional figure/image export.

---

Key features

Feature	Detail
Japanese‑centric models	Trained on Japanese document datasets; handles vertical writing and handwritten characters.
GPU‑accelerated, low‑VRAM	Works with ≤8 GB VRAM; a lightweight model (--lite) runs comfortably on CPU.
Web demo	Browser‑only inference via WebAssembly/WebGPU (no image upload).
CLI & Python API	yomitoku for batch processing, yomitoku_table for table parsing, yomitoku_extract/yomitoku_extract_with_llm for structured extraction.
Extensible extraction	Rule‑based (fast, no LLM) and LLM‑based (via vLLM, e.g., Qwen‑3‑8B‑Instruct) modes.
Commercial licensing	Free for non‑commercial use under CC BY‑NC‑SA 4.0; paid on‑prem or AWS Marketplace licences for enterprise features.

---

Installation

pip install yomitoku            # core OCR & layout
# optional extras for extraction
pip install "yomitoku[extract]"

Requires Python 3.10‑3.13, PyTorch ≥2.5, and CUDA ≥11.8 for GPU acceleration. A Dockerfile is provided for environments without compatible CUDA.

---

Quick start (CLI)

# Full OCR + layout, output markdown with figures
yomitoku path/to/image_or_dir -f md -o out_dir -v --figure

# Faster CPU‑only mode
yomitoku path/to/image -f json --lite -d cpu -o out_dir

---

Quick start (Python API)

import cv2
from yomitoku import DocumentAnalyzer

# optional lightweight config
configs = {
    "ocr": {
        "text_recognizer": {
            "model_name": "parseq-tiny-dynw-v4",
            "dynamic_width": True,
            "batch_bucketing": True,
            "device": "cpu",
        },
        "text_detector": {"device": "cpu", "infer_onnx": True},
    }
}

analyzer = DocumentAnalyzer(configs=configs, device="cpu")
img = cv2.imread("sample.jpg")
results, ocr_vis, layout_vis = analyzer(img)
results.to_json("output.json")

---

Table Semantic Parser (CLI)

yomitoku_table docs/ -o tables_out -v   # extracts key‑value & grid JSON

Options: --simple (text‑only), --raw (full schema), --lite, --grid_only, --kv_only, etc.

---

Extractor (CLI)

# Rule‑based extraction
yomitoku_extract invoice.jpg -s schema.yaml -o out_dir -v

# LLM‑based extraction (requires a running vLLM server)
yomitoku_extract_with_llm receipt.jpg -s schema.yaml -m Qwen/Qwen3-8B-Instruct

---

Documentation & Demo

• Live docs: https://kotaro-kinoshita.github.io/yomitoku/
• Browser demo (no install): https://yomitoku-studio.mlism.com
• Gallery of OCR/layout results: gallery.md and gallery_table_semantic_parser.md in the repo.

---

License

Source code and model weights are released under CC BY‑NC‑SA 4.0 (non‑commercial use allowed). Commercial deployments require a paid licence (on‑prem or AWS Marketplace).

---

Bottom line

YomiToku provides a ready‑to‑use, Japanese‑optimized document OCR and layout pipeline that can be run locally on modest hardware or accessed via a zero‑install web demo. It is suitable for developers needing high‑accuracy Japanese text extraction, table parsing, or structured data extraction from forms, with both rule‑based and LLM‑enhanced options.