Dicklesworthstone/franken_ocr

franken_ocr – Pure‑Rust CPU‑only OCR for Baidu’s Unlimited‑OCR and related models

What it is – franken_ocr (binary focr) is a Rust library + CLI that runs a handful of hand‑ported vision‑language models entirely on the CPU. The default model is Baidu Unlimited‑OCR, a strong document‑parsing model; additional models cover tables (GOT‑OCR2), image description/VQA (SmolVLM2), chart extraction (OneChart), and optical‑music‑recognition (Polyphonic‑TrOMR). No Python, CUDA, FFI, or GPU is required – the whole inference stack lives in a single, memory‑safe Rust binary.

---

Key capabilities

Feature	Why it matters
One portable binary	13‑17 MiB executables for macOS (Intel/Apple‑Silicon), Linux (x86‑64/ARM64) and Windows (x86‑64/ARM64). No external runtime, easy to ship to CI runners, edge boxes, or embedded agents.
Offline inference	After focr pull downloads the model artefacts (stored under ~/.cache/franken_ocr/models), all OCR runs are completely local – no network traffic is needed.
Rust API	The library exposes an OcrEngine with synchronous, blocking calls, so developers can embed OCR directly in Rust programs without dealing with async or FFI.
Native PDF handling	PDFs are rasterised in‑process with pure‑Rust code that respects page rotation and can split two‑page spreads.
Multi‑page & structured output	--multi-page produces a single document with <PAGE> delimiters; --json returns bounding‑box enriched JSON; --extract-figures saves charts/photos alongside Markdown.
Model zoo	focr models lists the five ready‑to‑run models; each has a custom, shape‑specialised kernel, eliminating the overhead of a generic ML framework.
Int8‑accelerated kernels	Hand‑written SIMD kernels (AVX‑512‑VNNI, AVX‑VNNI, AVX2, Apple SDOT/SMMLA, etc.) run int8‑quantised artefacts up to ~3× faster per token than a scalar reference.
Deterministic, agent‑ready output	A “robot” mode streams NDJSON events (including music‑specific staff events) with stable exit codes, making the tool suitable for autonomous agents.
Self‑test & provenance	focr robot selftest verifies that the int8 kernels produce bit‑identical results to a scalar oracle on the host CPU; extensive release‑readiness scripts and performance ledgers are part of the repo.
Memory‑safe Rust	#![forbid(unsafe_code)] everywhere except tiny audited SIMD islands; optional mmap loading for large artefacts.

---

Quick start (from the README)

# Install the binary (detects OS, verifies SHA‑256)
curl -fsSL https://raw.githubusercontent.com/Dicklesworthstone/franken_ocr/main/install.sh | bash
# Pull the default Unlimited‑OCR weights (≈4 GB)
focr pull
# OCR a single image to Markdown
focr ocr page.png
# Or get structured JSON with bounding boxes
focr ocr page.png --json -o page.json
# Use a specialised model (e.g., tables)
focr pull got-ocr2
focr ocr --model got-ocr2.int8.focrq --task tables table.png

All commands work offline after the model is cached.

---

Architecture at a glance

1. Input layer – Accepts PNG/JPG, rasterised PDF pages, image batches, or sheet‑music scans.
2. Model routing – Dispatches to the appropriate hand‑ported model based on the --task flag or explicit --model path.
3. Runtime core (OcrEngine) – Holds a single model instance, loads weights once, and records telemetry via a tiny SQLite‑like store (fsqlite).
4. CPU execution – Fixed‑shape Rust kernels written per model; SIMD dispatch selects the best ISA (AVX‑512‑VNNI, AVX‑VNNI, AVX2, scalar on x86; LLVM autovec or Apple SDOT/SMMLA on ARM). Int8 kernels use a conservative recipe that keeps most layers in high‑precision BF16 for accuracy while accelerating the feed‑forward network.
5. Output layer – Generates Markdown, JSON with layout boxes, MusicXML (for OMR), or NDJSON robot events.

---

Project health & licensing

• Version – v0.8.0 (binary releases for all major OS/arch combos). The default Unlimited‑OCR artefact is pinned to the v0.7.0 int8 checkpoint, which passes a 20‑page corpus CER budget of 0.193 (average) and a hard‑page termination test.
• Safety – unsafe is forbidden except for audited SIMD kernels; the codebase aims for full memory safety.
• License – MIT (see LICENSE).
• Release evidence – The repo contains performance ledgers, parity‑gate tests, fuzz corpora, and scripts that generate a “gauntlet” scorecard to certify each release.

---

Who might use it?

• Developers who need a lightweight, GPU‑free OCR component that can be embedded in Rust services or compiled into a single binary.
• CI / automation pipelines where installing Python + CUDA is impractical.
• Edge or offline devices (e.g., laptops, CI runners, IoT boxes) that must run document OCR without internet access.
• Agents that consume structured OCR output via the robot NDJSON stream.
• Music‑tech hobbyists looking for a pure‑Rust OMR solution.

---

Bottom line

franken_ocr delivers a production‑grade, CPU‑only OCR stack for a curated set of Baidu vision‑language models, wrapped in a memory‑safe Rust implementation. Its focus on deterministic kernels, offline operation, and extensive release‑readiness tooling makes it a solid choice for anyone needing reliable OCR without the heavyweight Python/CUDA ecosystem.