run-llama/liteparse

A fast, helpful, and open-source document parser

LiteParse – Fast, local PDF parsing for LLM‑friendly pipelines

What it is – LiteParse is an open‑source library that extracts text, layout, images and vector graphics from PDF‑like documents entirely on‑device. It is built around a Rust core that uses PDFium for text extraction and Tesseract (or any HTTP‑based OCR) for image‑based text. The project ships language bindings for Rust, Python, Node/TypeScript and WebAssembly, plus a small CLI (lit) that works the same regardless of how you install it.

Why it matters for AI – LLM‑based Retrieval‑Augmented Generation (RAG) and agent workflows need clean, structured document data. LiteParse produces:

  • Markdown with reconstructed headings, tables, lists and image placeholders – ready to feed directly to an LLM.
  • JSON that includes the raw text, precise bounding boxes, optional image metadata, vector‑graphics paths, PDF structure trees and form fields.
  • Screenshots (PNG) that capture the visual appearance of pages, useful for vision‑augmented agents. All of this runs locally, so you avoid cloud‑service costs, latency and data‑privacy concerns.

Key capabilities

  • Speed & light footprint – PDFium‑based text extraction is fast; OCR is optional and can be swapped for any HTTP OCR service.
  • Selective OCR – A cheap “complexity check” (lit is‑complex) tells you whether a document needs OCR before doing the heavy work.
  • Multi‑format input – PDFs, DOCX, XLSX, PPTX and images are first converted (LibreOffice + Rust image crates) into a PDF‑like representation for uniform processing.
  • Rich output options – Choose plain text, Markdown, or structured JSON; opt‑in to image extraction, vector graphics, PDF‑structure tree, layout blocks, annotations, form fields, document metadata, content bounds, XFA packets, etc.
  • Cross‑platform – Works on Linux, macOS (Intel/ARM) and Windows; also available as a WASM package for browsers.
  • Agent skill integration – Can be installed as an LLM‑agent skill via the skills CLI, enabling agents to call liteparse on the fly.

Installation (pick your language/runtime)

Language Command Docs
Rust (CLI) cargo install liteparse crates.io README
Python pip install liteparse Python README
Node/TypeScript npm i -g @llamaindex/liteparse Node README
Browser (WASM) npm i @llamaindex/liteparse-wasm WASM README

Typical CLI workflow

# Basic text extraction
lit parse report.pdf

# Markdown output (great for LLM prompts)
lit parse report.pdf --format markdown -o report.md

# JSON with bounding boxes and images
lit parse report.pdf --format json --extract-images -o report.json

# Quick OCR‑need check
lit is-complex report.pdf && lit parse report.pdf --no-ocr

# Batch processing a folder
lit batch-parse ./in ./out --format markdown

# Generate page screenshots for visual agents
lit screenshot report.pdf -o ./screenshots --dpi 300

How to use as a library (example in Python)

from liteparse import LiteParse
parser = LiteParse(ocr=False)          # disable OCR for speed
result = parser.parse_path('report.pdf')
print(result.json())                  # structured output with bboxes

(Equivalent APIs exist in Rust, Node and WASM.)

When to reach for the cloud version – For very complex PDFs (dense tables, multi‑column layouts, scanned documents, handwritten notes) the authors recommend their cloud service LlamaParse, which adds heavyweight AI‑driven layout reconstruction. LiteParse remains the go‑to choice when you need speed, privacy, or offline operation.

Who might benefit

  • Developers building RAG pipelines that ingest PDFs and need fast, deterministic preprocessing.
  • LLM‑agent platforms that require page screenshots or precise text coordinates.
  • Enterprises with data‑privacy constraints that cannot send documents to external services.
  • Anyone needing a lightweight, cross‑language PDF parser without a heavy ML stack.

TL;DR – LiteParse is a fast, locally‑run PDF (and office‑doc) parser that outputs Markdown, JSON with bounding boxes, images and vector graphics, and can be called from Rust, Python, Node or the browser. It’s designed for AI/RAG workflows that need structured document data without cloud dependencies.

Related

  • Project
    firecrawl/pdf-inspector

    pdf‑inspector is a Rust library (with Python, Node, and WebAssembly bindings) that quickly classifies PDFs (text‑based, scanned, mixed) and extracts their content into position‑aware Markdown. It runs in ~10‑50 ms for classification and under 200 ms for extraction, optionally adding a lightweight OCR step only on pages that need it. Ideal for developers building fast document‑ingestion pipelines, LLM data prep, or browser‑based PDF viewers.

  • Project
    drmingler/smart-llm-loader

    A Python package that converts documents and scanned images into clean, structured markdown chunks using OCR and LLM-powered semantic chunking for RAG systems.

  • Project
    PaddlePaddle/PaddleOCR

    PaddleOCR is an open‑source OCR and document‑AI toolkit that converts images/PDFs into structured, LLM‑ready JSON or Markdown. It offers multilingual scene‑text models (PP‑OCRv6), a lightweight vision‑language parser (PaddleOCR‑VL‑1.6) for tables, formulas, and ancient scripts, and a structure‑aware pipeline (PP‑StructureV3). Models run on CPU, GPU, XPU, NPU or via ONNX/TensorRT, and integrate with RAG/agent platforms like Dify and RAGFlow.

  • Project
    pymupdf/PyMuPDF

    A high-performance Python library for extracting and manipulating PDF and Office documents, featuring specialized tools for converting documents into LLM-ready Markdown for RAG pipelines.

  • Project
    Dicklesworthstone/llm_aided_ocr

    An advanced OCR pipeline that uses LLMs to correct errors in raw Tesseract OCR output and transform it into high-quality, formatted Markdown documents.