huggingface/sentence-transformers

State-of-the-Art Embeddings, Retrieval, and Reranking

What is Sentence‑Transformers?

Sentence‑Transformers is a Python library that makes it easy to work with text embedding models built on top of Hugging Face Transformers. It provides a thin, user‑friendly API for:

  • Computing dense or sparse sentence embeddings (so each sentence becomes a fixed‑size vector you can store, compare, or index).
  • Reranking with Cross‑Encoders – models that score a query‑document pair directly, useful for the second stage of a retrieve‑and‑rerank pipeline.
  • Late‑interaction / Multi‑Vector encoders (ColBERT‑style) that keep token‑level vectors for more accurate similarity.
  • Training / fine‑tuning any of the above model types on your own data.

All of this is wrapped in a single package, with over 15 000 pre‑trained models hosted on the Hugging Face Hub, covering many languages and tasks.

Quick start (code you can copy‑paste)

# Install the library (Python 3.10+, PyTorch 2.2+, transformers 5.0+ recommended)
# pip install -U sentence-transformers

# 1️⃣ Load a ready‑made dense embedding model
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

sentences = [
    "The weather is lovely today.",
    "It's so sunny outside!",
    "He drove to the stadium."
]
emb = model.encode(sentences)          # shape → (3, 384)
print(emb.shape)

# 2️⃣ Compute pair‑wise similarities (cosine‑like scores)
print(model.similarity(emb, emb))

Reranking example (Cross‑Encoder):

from sentence_transformers import CrossEncoder
ranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2")
query = "How many people live in Berlin?"
passages = [
    "Berlin had a population of 3,520,031 registered inhabitants…",
    "Berlin has a yearly total of about 135 million day visitors…",
    "In 2013 around 600,000 Berliners were registered in sport clubs."
]
# Score each (query, passage) pair
scores = ranker.predict([(query, p) for p in passages])
print(scores)
# Or let the library rank for you
print(ranker.rank(query, passages, return_documents=True))

Sparse‑embedding example:

from sentence_transformers import SparseEncoder
sparse = SparseEncoder("naver/splade-cocondenser-ensembledistil")
emb = sparse.encode(["The weather is lovely today."])
print(emb.shape)          # e.g. (1, 30522) – one dimension per vocab token
print(SparseEncoder.sparsity(emb))

Late‑interaction (Multi‑Vector) example:

from sentence_transformers import MultiVectorEncoder
mv = MultiVectorEncoder("lightonai/GTE-ModernColBERT-v1")
q_emb = mv.encode_query(["What is the capital of France?"])
 d_emb = mv.encode_document(["Paris is the capital of France.", "Berlin is the capital of Germany."])
print(mv.similarity(q_emb, d_emb))

When would you use it?

Scenario Recommended model type Why use Sentence‑Transformers?
Semantic search / retrieval Dense or Sparse Encoder One‑line model.encode → vectors you can index with FAISS, Annoy, etc.
Rerank top‑k results Cross‑Encoder (reranker) Directly scores query‑document pairs for higher precision
Hybrid search (dense + sparse) Combine a dense encoder with a sparse encoder Leverages lexical matching and semantic similarity together
Multilingual similarity Multilingual Sentence‑BERT models Works out‑of‑the‑box for >100 languages
Very large corpora with limited RAM Sparse or Matryoshka embeddings Sparse vectors are tiny; Matryoshka can be truncated on‑the‑fly
Fine‑tune on a custom dataset Any of the four model families Library ships >20 loss functions and training utilities

How does it work under the hood?

  1. Base transformer – a BERT/RoBERTa/etc. model from 🤗 Transformers.
  2. Pooling layer – for dense models it collapses token embeddings into a single sentence vector (mean, CLS, max‑pool, etc.).
  3. Cross‑Encoder head – a classification head that takes the concatenated pair [CLS] query [SEP] passage and outputs a relevance score.
  4. Sparse head – predicts a distribution over the vocabulary, yielding a high‑dimensional sparse vector (like SPLADE).
  5. Multi‑Vector head – keeps token‑level embeddings, enabling late‑interaction scoring without collapsing to a single vector.

All of these components are wired together in the sentence_transformers package, so you never have to write the boilerplate yourself.

Getting the library

# Recommended Python version
pip install -U sentence-transformers

Optional extras (install with pip install "sentence-transformers[train]" etc.) add support for:

  • Image / audio / video encoders ([image], [audio], [video])
  • ONNX / OpenVINO export for faster inference ([onnx], [openvino])
  • Training utilities ([train])
  • Development tools ([dev])

Documentation & community

  • Full docs: www.sbert.net – includes installation guides, API reference, tutorials, and a model zoo.
  • Model hub: 🤗 Hub – library=sentence-transformers – browse 15 k+ pre‑trained models.
  • Blog posts: training guides, multimodal extensions, efficiency tricks (Matryoshka, static embeddings, quantization).
  • Issues & contributions: see CONTRIBUTING.md; the maintainer is Tom Aarsen (🤗 Hugging Face).

Cite the work

If you use the library in research, please cite the original Sentence‑BERT paper (Reimers & Gurevych 2019) and, for multilingual models, the 2020 distillation paper. Bibliographic entries are provided in the README.

Related