Tencent-Hunyuan/HunyuanOCR

HunyuanOCR‑1.5 – Lightweight OCR‑specialised Vision‑Language Model

What it is

• An end‑to‑end vision‑language model (VLM) that reads text in images and can also do document parsing, information extraction, table parsing and text‑image translation. It builds on the earlier HunyuanOCR‑1.0 but adds two major upgrades: faster inference via DFlash speculative decoding and easy PC‑side deployment through llama.cpp.

Key ideas & features

Feature	Why it matters
DFlash speculative decoding	A lightweight “draft” model predicts several candidate tokens in parallel; the main model verifies them in one pass, cutting the long‑autoregressive decoding bottleneck for dense documents, tables or formulas while keeping the same output distribution.
Unified inference environment	One uv‑based environment (CUDA 13) lets you run three configurations – vanilla vLLM autoregressive, DFlash‑accelerated, or native HuggingFace Transformers – with the same weights and prompts, making benchmarking straightforward.
PC‑side deployment via llama.cpp	The model (and its DFlash draft) can be converted to GGUF and served on CPU, consumer‑GPU or laptop hardware with an OpenAI‑compatible llama-server.
Agentic Data Flow	An internal pipeline where agents turn model weaknesses into targeted data‑construction tasks (e.g., low‑resource scripts, ancient Chinese characters, multi‑image QA). This data is fed back into the training recipe.
Large context & resolution	Supports up to 4K image resolution and a 128K token context window, enabling long documents and multi‑page reasoning.
Extensive benchmarks	The repo ships with several community‑level benchmarks (CHAOS‑Bench, Chronicles‑OCR, ChartArena, Wild‑OmniDocBench, MMTIT‑Bench) for evaluating hallucination, ancient‑script OCR, chart parsing, etc.

Typical use cases

• Digitising scanned documents, academic papers, contracts, or historical manuscripts.
• Extracting structured information (tables, forms) directly from images.
• Performing OCR‑driven question answering or translation across languages.
• Running on modest hardware (laptop/CPU) thanks to the llama.cpp path.

Getting started (high‑level)

1. Set up the unified inference env (requires Python 3.12 and CUDA 13):

   pip install uv
   uv venv --python 3.12 && source .venv/bin/activate
   uv pip install "vllm>=0.25.1" "flash-attn==2.8.3"
   

2. Download the model from HuggingFace:

   pip install -U "huggingface_hub[cli]"
   huggingface-cli download tencent/HunyuanOCR --local-dir ./HunyuanOCR --exclude "v1.0/*"
   

3. Run a quick server (vLLM autoregressive, single GPU):

   MODEL_PATH=./HunyuanOCR GPU=0 PORT=8000 bash inference/vLLM/serve.sh
   

4. Send an image (example for document parsing):

   python inference/vLLM/infer_vllm_client.py \
       --image /path/to/doc.png \
       --task-type doc_parse \
       --model tencent/HunyuanOCR \
       --port 8000 --max-tokens 32768
   

   Replace doc_parse with any of the 12 supported task types (see --list-tasks).
5. Optional DFlash acceleration – use the inference/DFlash/serve_DFlash.sh script; the same client works unchanged.
6. PC‑side deployment – convert the checkpoint to GGUF with the provided convert_hf_to_gguf.py scripts and serve with llama.cpp as described in docs/llama_cpp.md.

Training (if you want to fine‑tune or extend)

• Pack raw OCR JSONL files into long sequences (up to 20 480 tokens) with the scripts/pack_data.sh pipeline.
• Run supervised‑fine‑tuning on the base model (scripts/sft_base.sh).
• Train a new DFlash draft from scratch (scripts/sft_dflash.sh) or continue‑fine‑tune an existing draft (scripts/sft_dflash_finetune.sh).
• All scripts support multi‑node distributed training via standard NNODES, MASTER_ADDR, etc., and rely on PyTorch 2.1+, DeepSpeed, and Flash‑Attention.

Documentation & further reading

• docs/training.md – detailed hyper‑parameters and distributed setup.
• docs/inference/inference.md – full inference recipes, task‑type table, and multi‑GPU guidance.
• docs/llama_cpp.md – step‑by‑step conversion and serving on consumer hardware.
• Benchmark descriptions are in benchmarks/CHAOS-Bench and the linked arXiv papers.

Citation If you use HunyuanOCR‑1.5 in research, please cite the arXiv paper (2026) provided in the README.

---

All information above is taken directly from the repository’s README; no external assumptions have been added.