langextract: a structured information extraction library that maps data back to source text for precise verification

What it solves

LangExtract is a Python library designed to extract structured information from unstructured text documents. It addresses the challenge of turning messy, free-form text (like clinical notes or reports) into organized data while ensuring that every piece of extracted information is grounded in the source text for easy verification.

How it works

The library uses Large Language Models (LLMs) and a few-shot prompting strategy. Users define an extraction task by providing a prompt and a few high-quality examples of how text should be mapped to structured entities. To handle long documents, it employs a strategy of text chunking, parallel processing, and multiple extraction passes to improve recall.

Who it’s for

It is built for developers and researchers who need to convert unstructured text into structured datasets without fine-tuning a model. It is particularly useful for those working with domain-specific documents, such as medical reports or literary analysis, where traceability and precision are required.

Highlights

• Precise Source Grounding: Maps extractions to exact character intervals in the source text for visual traceability.
• Reliable Structured Outputs: Uses controlled generation (in supported models) and few-shot examples to enforce consistent schemas.
• Long Document Optimization: Uses parallel processing and multiple passes to overcome the "needle-in-a-haystack" problem.
• Interactive Visualization: Generates self-contained HTML files to visualize extracted entities within their original context.
• Flexible LLM Support: Compatible with cloud models (Gemini, OpenAI) and local models via Ollama.
• Extensible Provider System: Allows users to add custom LLM providers via a plugin system.