datalab-to/surya

What it solves

Scanned documents are messy: they mix headings, tables, figures, forms, handwriting, and equations, often in different languages. Plain OCR just outputs raw text, losing the structure that makes documents usable. Surya turns a document page into machine-readable text plus a structured description of it — it detects text lines, identifies layout blocks (like headings, tables, pictures, captions), determines reading order, and recognizes table rows and columns. It also handles math inline, returning equations as LaTeX. This makes scanned PDFs and images searchable, extractable, and convertible to formats like HTML or Markdown.

How it works

Surya is a 650M-parameter vision-language model (a model that takes images as input and produces text). In version 2, one shared model handles layout analysis, full-page OCR, and table recognition; a separate, smaller model handles line-level text detection, and another detects OCR errors. On first use, the tool automatically starts an inference server — using vllm on NVIDIA GPUs or llama.cpp on CPU/Apple Silicon. You can also point it at an already-running server via an environment variable. The command-line tools (surya_ocr, surya_layout, surya_table, surya_detect) write JSON result files containing bounding boxes, layout labels, reading-order positions, confidence scores, and HTML output (tables as <table>...</table> tags, equations as <math>...</math> tags with LaTeX). A --keep_server flag keeps the inference server alive between commands so you don't pay startup costs every time. There is also a Python API and a Streamlit graphical interface.

Who it's for

Developers and teams building document-processing pipelines — for example, indexing scanned archives for search, extracting data from forms and tax documents, parsing textbooks and newspaper layouts, or converting PDFs into structured formats. It's free for research, personal use, and startups under $5M in funding or revenue; broader commercial use of the model weights requires a paid license (the code itself is Apache 2.0). The README also points heavy or difficult workloads to the hosted Datalab API.

Highlights

• 83.3% accuracy on the olmOCR-bench benchmark — the top score among models under 3B parameters
• Throughput of about 5 pages per second on an RTX 5090 GPU
• Multilingual: 87.2% on an internal benchmark covering 91 languages
• Layout analysis with reading order, plus table recognition that outputs row/column geometry and cells; full-HTML mode handles spanning cells and header rows
• Equations are recognized as part of the same OCR pass, emitted as KaTeX-compatible LaTeX inside <math> tags
• Runs on NVIDIA GPUs via vllm or on CPU / Apple Silicon via llama.cpp
• Includes an interactive Streamlit app (surya_gui) to try it on images or PDFs