Tencent/WeVisDoc

WeVisDoc is an end-to-end document parser for page images. Fine-tuned from Qwen3-VL-2B-Instruct and Qwen3-VL-4B-Instruct, it turns a page into structured Markdown, with LaTeX formulas and HTML tables.

What it solves

WeVisDoc addresses the challenge of end-to-end document parsing, converting page images into structured, machine-readable formats. It specifically solves the problem of accurately extracting text, complex LaTeX formulas, and HTML tables from images of documents while maintaining the original structure.

How it works

WeVisDoc is a vision-language model fine-tuned from Qwen3-VL (in 2B and 4B parameter versions). It processes page images as input and directly generates structured Markdown output. It can be deployed as a service using vLLM for high-throughput inference or run locally using the Transformers library.

Who it’s for

This tool is designed for developers and researchers who need to digitize documents, automate data extraction from PDFs or images, or convert visual document layouts into structured Markdown for further processing by other AI systems.

Highlights

  • High Accuracy: Outperforms other end-to-end parsers on OmniDocBench v1.6 and PureDocBench benchmarks.
  • Structured Output: Supports the conversion of page images into Markdown, including LaTeX for formulas and HTML for tables.
  • Flexible Deployment: Offers multiple ways to run, including a vLLM-based server for API access or local Transformers-based inference.
  • Scalable Options: Available in two model sizes (2B and 4B parameters) to balance performance and accuracy.

Related

  • Project
    OpenBMB/MiniCPM-V-Apps

    OpenBMB’s MiniCPM‑V‑Apps repo provides three on‑device demo apps (iOS, Android, HarmonyOS) that run MiniCPM‑V multimodal LLMs locally via a fork of llama.cpp. It includes build scripts, model‑download instructions, and hardware requirements for models ranging from 1 B to a 35 B MoE that streams experts from SSD.

  • Project
    shallowdream204/BitDance

    BitDance is a 14B parameter autoregressive multimodal model that uses binary tokens and a next-patch diffusion paradigm to generate high-resolution images up to 30x faster than standard AR models.

  • Project
    StanfordMIMI/Merlin

    Merlin is a 3D vision-language foundation model and dataset for computed tomography that integrates CT scans with electronic health records and radiology reports for medical analysis.

  • Project
    TaoLiveAIGC/TaoMate

    TaoMate is a research‑grade, real‑time audio‑video digital‑human generation system. The repo provides inference code, multi‑GPU launchers, and a four‑GPU interactive browser demo. It stitches together a 22 B LTX‑2.3 transformer, a Gemma‑3 text encoder, and optional Gemma‑4 dialogue model, requiring high‑memory NVIDIA GPUs and separate model checkpoints from Hugging Face.