xberg-io/xberg

The fast, precise document-intelligence engine for every language: text, tables and metadata from 100+ formats, with OCR, transcription and code intelligence.

What it solves

Xberg is a comprehensive document-intelligence engine designed to eliminate the need for assembling complex pipelines of multiple libraries to extract structured data from diverse file formats. It provides a single, unified interface for format detection, reading, OCR, and content extraction across a vast array of documents, images, audio, and web content.

How it works

The engine handles the entire ingestion process—from identifying the MIME type of a file to extracting its content. It supports 107 document formats (including PDFs, Office files, and e-books), OCR via multiple backends (Tesseract, PaddleOCR, Candle, or VLM), and speech-to-text transcription using Whisper ONNX. It can also crawl URLs, recursively extract nested archives, and perform syntax-aware chunking for code in 371 programming languages. The extracted content can be output in six different formats, including Markdown and JSON.

Who it’s for

Developers building RAG (Retrieval-Augmented Generation) pipelines, AI agents, or any application requiring high-precision document parsing and structured data extraction from a wide variety of sources.

Highlights

  • Massive Format Support: Handles 107 formats across 141 file extensions, including academic citations, emails, and databases.
  • Multi-Language Bindings: Available for 15 languages, including Rust, Python, Node.js, Go, Java, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WASM, and Kotlin.
  • AI-Ready Features: Includes built-in support for embeddings, reranking, NER, summarization, and schema-driven JSON extraction via local or hosted LLMs.
  • Flexible Deployment: Can be run as a library, CLI tool, REST API, or MCP server, and is CPU-optimized by default.
  • Advanced Layout Analysis: Uses ML models to reconstruct reading order and table grids for clean Markdown output.

Related

  • Project
    firecrawl/anydoc

    A high-performance Rust library that converts various office document formats into clean, consistent Markdown for LLM ingestion and agentic workflows.

  • Dispatch
    Hugging Face Accelerating Document AI

    Hugging Face provides a comprehensive guide to using open-source multimodal models to automate document classification, parsing, and visual question answering for enterprise workflows.

  • Project
    docling-project/docling

    Docling is an open‑source Python library/CLI that parses many document types (PDF, Word, slides, spreadsheets, HTML, e‑books, audio, video, email, etc.) into a unified, structured representation. It extracts layout, tables, code, formulas, images, and speech‑to‑text, then exports to Markdown, HTML, JSON, DocLang, DocTags, etc. Designed for generative‑AI workflows, it offers plug‑and‑play integrations with LangChain, LlamaIndex, Crew AI, Haystack, and supports local execution for privacy‑sensitive data.

  • Project
    PaddlePaddle/PaddleOCR

    PaddleOCR is an open‑source OCR and document‑AI toolkit that converts images or PDFs into structured, LLM‑ready text, tables and markdown/JSON. It offers multilingual scene‑text models (PP‑OCRv6) and lightweight vision‑language models (PaddleOCR‑VL, PP‑StructureV3) with state‑of‑the‑art accuracy, fast CPU/GPU inference, and deployment options ranging from Docker to browser SDKs. Integrated with RAG platforms like Dify and RAGFlow, it serves as a data‑engine for building AI agents and retrieval‑augmented generation pipelines.

  • Project
    DocAILab/XRAG

    XRAG is an open‑source benchmarking framework for Retrieval‑Augmented Generation (RAG). It lets users evaluate individual RAG components—retrievers, embeddings, LLMs, orchestrators—using classic IR metrics and LLM‑based quality scores. Features include a modular plug‑in architecture, support for OpenAI, Ollama, and Hugging Face models, a CLI, a web UI for interactive testing, and a REST API service. Install via pip, configure with a TOML file, and run benchmarks or generate synthetic QA data. Ideal for researchers and engineers building or comparing RAG systems.