Unstructured-IO/unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

What it solves

Unstructured is designed to solve the challenge of ingesting and pre-processing unstructured data—such as PDFs, HTML, Word documents, and images—into a format that is usable for Large Language Models (LLMs). It streamlines the data processing workflow by transforming messy, unstructured documents into structured outputs.

How it works

The library uses a modular system of functions and connectors. The primary entry point is the partition function, which automatically detects the file type and routes the document to the appropriate file-specific partitioning function. It can handle a wide variety of formats including PDFs, emails, and text files, often leveraging system dependencies like tesseract-ocr for images and poppler-utils for PDFs to extract text and layout elements.

Who it’s for

It is intended for developers and data engineers building LLM-powered applications who need to efficiently clean and prepare diverse document types for downstream AI tasks.

Highlights

  • Broad Format Support: Handles PDFs, HTML, Word docs, images, and many other document types.
  • Automated Partitioning: The partition function simplifies ingestion by automatically detecting file types.
  • Conda and Docker Support: Provides official Docker images and Conda installation paths for easier environment setup.
  • Flexible Installation: Allows users to install only the dependencies required for specific document types (e.g., unstructured[docx,pptx]).

Related

  • Project
    lumina-ai-inc/chunkr

    An open-source document intelligence API that converts PDFs, PPTs, Word docs, and images into structured, RAG-ready chunks using layout analysis and OCR.

  • Project
    emcf/thepipe

    thepipe is a Python library that extracts clean markdown, tables, and images from many file types (PDF, DOCX, PPTX, notebooks, audio/video, etc.) using vision‑language models. It returns “chunks” that can be fed to LLMs, converted to LlamaIndex documents, or used for structured extraction with Pydantic schemas. Install via `pip install thepipe-api` and add optional extras for Whisper, sentence‑transformers, GPU support, or LlamaIndex integration.

  • Project
    drmingler/smart-llm-loader

    A Python package that converts documents and scanned images into clean, structured markdown chunks using OCR and LLM-powered semantic chunking for RAG systems.

  • Project
    data-prep-kit/data-prep-kit

    A scalable toolkit for cleansing, transforming, and enriching unstructured data to prepare it for LLM pre-training, fine-tuning, and RAG applications.

  • Project
    docling-project/docling

    Docling is an open‑source Python library/CLI that parses many document types (PDF, Word, slides, spreadsheets, HTML, e‑books, audio, video, email, etc.) into a unified, structured representation. It extracts layout, tables, code, formulas, images, and speech‑to‑text, then exports to Markdown, HTML, JSON, DocLang, DocTags, etc. Designed for generative‑AI workflows, it offers plug‑and‑play integrations with LangChain, LlamaIndex, Crew AI, Haystack, and supports local execution for privacy‑sensitive data.