Unstructured-IO/unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

What it solves

Unstructured is designed to solve the challenge of ingesting and pre-processing unstructured data—such as PDFs, HTML, Word documents, and images—into a structured format. This process is essential for streamlining data pipelines for Large Language Models (LLMs), making it easier to transform messy, diverse file types into clean, structured outputs suitable for vector databases and other downstream AI applications.

How it works

The library uses a modular system of functions and connectors to partition, enrich, chunk, and embed files. It provides a primary partition function that automatically detects the file type and routes it to the appropriate specialized partitioning function. It can handle over 60 different file formats, including scanned files and emails.

Who it’s for

It is intended for developers and data engineers building LLM-based applications, specifically those who need to prepare diverse document sets for RAG (Retrieval-Augmented Generation) or other AI agents.

Highlights

  • Broad Format Support: Supports over 60 file types, including PDFs, emails, Word docs, and images.
  • MCP Server Integration: Offers an Unstructured Transform MCP server that allows AI agents to parse and chunk documents directly within their session.
  • Flexible Deployment: Can be run as a Python library, via Docker containers, or through a SaaS API.
  • Automated Partitioning: Automatically detects file types to simplify the ingestion process.

Related

  • Project
    drmingler/smart-llm-loader

    A Python package that converts documents and scanned images into clean, structured markdown chunks using OCR and LLM-powered semantic chunking for RAG systems.

  • Project
    docling-project/docling

    Docling is an open‑source Python library that parses a huge variety of document types (PDF, Word, slides, spreadsheets, e‑books, audio, video, emails, images, and many XML schemas) and enriches them with AI‑driven layout analysis, OCR, visual‑language‑model understanding, and speech‑to‑text. It outputs a unified `DoclingDocument` that can be exported to Markdown, HTML, JSON, DocLang, DocTags, etc., and integrates directly with LangChain, LlamaIndex, Crew AI, Haystack, or custom agents via an MCP or REST API. The tool runs locally for privacy‑sensitive data and also offers a CLI and service mode, making it ideal for building RAG pipelines, enterprise knowledge bases, financial or legal document processing, and multimodal AI agents.

  • Project
    virgiliojr94/book-to-skill

    A tool that converts technical books and documents into structured, on-demand AI agent skills, reducing token costs and hallucinations by avoiding full-context dumping.

  • Project
    docling-project/docling-mcp

    An MCP server that provides AI agents with tools for converting PDFs into structured formats and generating new documents using the Docling library.

  • Project
    deepdoctection/deepdoctection

    A Python library for document understanding that orchestrates layout analysis, OCR, and token classification to extract structured data from PDFs and scans.