explosion/spacy-transformers

What it solves

spaCy is a popular Python library for natural language processing (NLP), but by itself it doesn't include transformer models like BERT, GPT-2, or XLNet. These pretrained models capture deep language understanding and have become state-of-the-art for many tasks. spacy-transformers bridges that gap by letting you plug any Hugging Face transformer into your spaCy pipeline, so you can build powerful NLP applications without training everything from scratch.

How it works

This package provides spaCy components and architectures that wrap Hugging Face's transformers library. You add a transformer component to your spaCy pipeline, which loads a pretrained model (e.g., BERT, RoBERTa, XLNet) and runs it on your text. The component automatically aligns the transformer's output with spaCy's tokenization, saving contextual embeddings into the Doc object. You can then use those embeddings as features for other spaCy components, like named entity recognition (NER) or text classification. Training uses spaCy v3's config system, which makes it easy to set up and run multi-task learning—meaning you can backpropagate gradients from several downstream tasks into the same transformer model. The package also handles serialization, so models can be saved and packaged like any other spaCy pipeline.

Who it’s for

Developers and NLP practitioners who already use spaCy and want to incorporate modern transformer models into their pipelines without juggling separate frameworks. It’s especially useful for tasks that benefit from contextualized word representations, like NER, text classification, or any custom spaCy component that needs rich features. Because it integrates with spaCy’s training workflow, it’s also suitable for teams that want to fine-tune a transformer on their own data.

Highlights

• Use a wide range of pretrained transformers (BERT, RoBERTa, XLNet, etc.) directly inside spaCy.
• Seamless integration with spaCy v3’s config system for training and evaluation.
• Multi-task learning: one transformer can feed features to multiple downstream components.
• Automatic tokenization alignment between transformer subwords and spaCy tokens.
• Customizable what transformer output is stored in the Doc object.
• Built-in support for serialization and model packaging.
• Requires Python 3.6+, PyTorch 1.5+, and spaCy v3+.