neural-maze/production-ocr-course

What it solves

This course goes beyond simple OCR tutorials that just call an API and get text back. It teaches you how to build a complete, production-grade pipeline that doesn't just read text but truly understands documents — including charts, tables, and layout. Instead of relying on a large, expensive model, it uses a small language model (Qwen 3.5) to achieve high-accuracy OCR with lower cost. The course also addresses the hard parts of running such a system in the real world: scaling to handle bursts of work, securing the pipeline, and making it usable by AI agents.

How it works

You build an event-driven OCR system on Kubernetes (AKS or GKE). Files come in through a high-concurrency Rust gateway, which stores them in Redis and queues tasks. A KEDA autoscaler monitors the queue and spins up resources as needed. A consumer worker (running on a T4 GPU) uses a layout detector (PP-DocLayoutV3) to find semantic regions like paragraphs, tables, and charts. It then sends these crops to a vLLM server (running on an A100 GPU) that uses Qwen 3.5 to generate text. The whole process uses a zero-copy shared-memory handoff (/dev/shm) to avoid disk bottlenecks. Finally, the system is wrapped in an MCP server so AI agents like Claude Code can use it natively.

Who it’s for

This course is designed for ML/AI engineers and platform engineers who already know how to call an OCR API but want to understand what it takes to run one in production. It covers GPU node pools, autoscaling economics, network security, and the systems-design tradeoffs behind serving a generative model at high throughput.

Highlights

• Real production deployment: You deploy the full pipeline on Kubernetes with separate node pools for different tasks.
• SLM-powered OCR: Uses Qwen 3.5 (4B) to understand charts, handwriting, and context — not just extract flat text.
• High throughput: Achieves 1.86 pages/second with vLLM's continuous batching and Multi-Token Prediction.
• Autoscaling: KEDA independently scales T4 and A100 node pools from zero to burst load.
• Security-first: The pipeline is locked behind an internal load balancer and an enterprise API gateway with no public exposure.
• Agent-ready: Exposed as an MCP server, allowing AI agents to use it directly.