Yuliang-Liu/MonkeyOCRv2

MonkeyOCRv2 Vision Encoder — A Document-Native Visual Backbone

What it solves

MonkeyOCRv2 is a visual-text foundation model designed for Document AI. It addresses the challenge of accurately parsing and understanding text-rich documents across multiple languages and formats, including digital-born PDFs, photographed documents, scientific papers, medical reports, and charts.

How it works

The project provides a document-native vision encoder that can serve as a visual backbone for various tasks. It offers specialized models for different needs:

  • MonkeyOCRv2-Parsing: A multilingual document parsing model that converts documents into Markdown and JSON formats.
  • MonkeyOCRv2-Und: A model for document understanding, enabling visual question answering (VQA) on documents.
  • Vision Encoder: Standalone encoders (S, B, and AS versions) that can be integrated into other OCR systems for feature extraction.

It is supported by MonkeyDoc v2, a massive multilingual corpus of 113 million document image-text pairs across 17 languages.

Who it’s for

  • AI Developers: Those building document processing pipelines or RAG (Retrieval-Augmented Generation) systems (e.g., it is already supported by RAGFlow).
  • Researchers: Those working on OCR, document understanding, and scene text recognition.
  • Enterprise Users: Organizations needing to digitize complex, multilingual documents at scale.

Highlights

  • Multilingual Support: Robust parsing across 17 languages, including Latin and non-Latin scripts.
  • DFlash Acceleration: Support for vLLM serving with DFlash for up to 2’s faster inference.
  • Versatile Task Support: Capable of text recognition, detection, formula recognition, and document tampering detection.
  • Large-scale Pre-training: Built on MonkeyDoc v2, one of the largest document image pre-training datasets.

Related

  • Project
    Yuliang-Liu/MultimodalOCR

    A collection of benchmarks (MDPBench, OCRBench, and OCRBench v2) used to evaluate the OCR and multilingual document parsing capabilities of Large Multimodal Models in real-world scenarios.

  • Project
    Yuliang-Liu/MonkeyOCR

    MonkeyOCR is a document parsing model that uses a Structure-Recognition-Relation triplet paradigm to convert PDFs and images into structured Markdown, supporting English and Chinese.

  • Dispatch
    Hugging Face Accelerating Document AI

    Hugging Face provides a comprehensive guide to using open-source multimodal models to automate document classification, parsing, and visual question answering for enterprise workflows.

  • Dispatch
    Hugging Face releases vdr-2b-multi-v1 multilingual visual document retrieval model

    Hugging Face has introduced vdr-2b-multi-v1, a multilingual embedding model for visual document retrieval that enables searching complex documents without OCR by encoding page screenshots into dense vectors.

  • Project
    PaddlePaddle/PaddleOCR

    PaddleOCR is an open‑source OCR and document‑AI toolkit that converts images or PDFs into structured, LLM‑ready text, tables and markdown/JSON. It offers multilingual scene‑text models (PP‑OCRv6) and lightweight vision‑language models (PaddleOCR‑VL, PP‑StructureV3) with state‑of‑the‑art accuracy, fast CPU/GPU inference, and deployment options ranging from Docker to browser SDKs. Integrated with RAG platforms like Dify and RAGFlow, it serves as a data‑engine for building AI agents and retrieval‑augmented generation pipelines.