explosion/spacy-llm

What it solves

It allows developers to integrate Large Language Models (LLMs) into spaCy pipelines to perform structured NLP tasks without needing training data. It bridges the gap between the fast prototyping capabilities of LLM prompting and the production-ready reliability of supervised learning or rule-based systems.

How it works

spacy-llm provides a modular system where users can define a task (which handles prompting and parsing the response) and a model (the LLM being used). It integrates as a serializable component within a spaCy pipeline, turning unstructured LLM responses into robust, structured outputs. It also supports a map-reduce approach to handle texts that exceed an LLM's context window by splitting prompts and fusing the results.

Who it’s for

NLP practitioners and developers who want to quickly prototype NLP tasks using LLMs but maintain the flexibility to mix them with traditional spaCy components or eventually replace them with smaller, supervised models for production.

Highlights

• Broad Model Support: Interfaces with OpenAI, Cohere, Anthropic, Google PaLM, Microsoft Azure AI, and open-source models via Hugging Face (e.g., Llama 2, Mistral).
• LangChain Integration: Supports all LangChain models and features.
• Pre-built Tasks: Includes out-of-the-box support for NER, text classification, lemmatization, relationship extraction, sentiment analysis, and more.
• Customizable: Allows the implementation of custom prompting, parsing, and model integrations via spaCy's registry.