Yuliang-Liu/MonkeyOCR

A lightweight LMM-based Document Parsing Model

What it solves

MonkeyOCR is designed to solve the complexities of document parsing, specifically the challenge of converting PDFs and images into structured formats like Markdown. It aims to replace cumbersome multi-tool pipelines and the inefficiency of using massive multimodal models for full-page processing by providing a more streamlined, high-performance approach to recognizing text, formulas, and tables.

How it works

The project utilizes a Structure-Recognition-Relation (SRR) triplet paradigm. This approach simplifies the parsing process by focusing on three core components: identifying the structure of the document, recognizing the content within those structures, and determining the relationship between different blocks of content. It supports both English and Chinese documents and offers different model sizes (e.g., 1.2B and 3B parameters) to balance speed and accuracy.

Who it’s for

This tool is intended for developers and researchers who need to convert complex documents (PDFs or images) into structured, machine-readable text, especially those working with multilingual (English and Chinese) documents containing mathematical formulas and tables.

Highlights

  • High Performance: Outperforms several closed-source and large open-source VLMs (like GPT-4o and Gemini 2.0-Flash) on the OmniDocBench benchmark.
  • Flexible Deployment: Supports a wide range of GPUs (from 4060 to H800) and provides Docker deployment options.
  • Comprehensive Output: Generates processed Markdown files, layout PDFs, and detailed intermediate JSON results containing block coordinates and relationships.
  • Efficient Scaling: The 1.2B model provides a significant speed increase (approx. 36%) over the 3B version with minimal performance loss.

Related

  • Project
    Yuliang-Liu/MonkeyOCRv2

    MonkeyOCRv2 is an open‑source visual‑text foundation model for Document AI. It ships a ViT‑based vision encoder plus multilingual parsing and understanding heads (≈0.6‑1.8 B parameters) that achieve state‑of‑the‑art scores on OCR, layout parsing, VQA and formula recognition across 17 languages. Models are available on Hugging Face/ModelScope, can be run with Transformers or vLLM (CPU support and DFlash acceleration optional), and are accompanied by the large MonkeyDoc v2 pre‑training dataset.

  • Project
    PaddlePaddle/PaddleOCR

    PaddleOCR is an open‑source OCR and document‑AI toolkit that converts images/PDFs into structured, LLM‑ready JSON or Markdown. It offers multilingual scene‑text models (PP‑OCRv6), a lightweight vision‑language parser (PaddleOCR‑VL‑1.6) for tables, formulas, and ancient scripts, and a structure‑aware pipeline (PP‑StructureV3). Models run on CPU, GPU, XPU, NPU or via ONNX/TensorRT, and integrate with RAG/agent platforms like Dify and RAGFlow.

  • Project
    magicrew/doc7

    A document conversion tool that turns PDFs, Office files, and images into AI-ready Markdown using multimodal vision models, supporting local inference for privacy and cost-efficiency.

  • Project
    docling-project/docling

    A document parsing tool that converts complex PDFs, Office files, and multimodal content into structured Markdown or JSON for use in generative AI and RAG pipelines.

  • Project
    datalab-to/marker

    A high-accuracy document conversion tool that transforms PDFs and other files into structured Markdown, JSON, or HTML, specializing in tables and equations.