MinishLab/model2vec

Fast State-of-the-Art Static Embeddings

Model2Vec – Fast, Tiny Static Embedding Models

What it is – Model2Vec is a Python library that turns any sentence‑transformer (a neural model that produces a vector for a whole sentence) into a static‑embedding model. A static model stores a fixed vector for each token (wordpiece/sub‑word) and can therefore be tiny (8‑30 MB) and very fast (up to 500× faster on CPU) while still delivering state‑of‑the‑art quality on standard benchmark suites.

Key capabilities (as described in the README)

Feature What it means for you
Size reduction Up to 50× smaller than the original sentence‑transformer (e.g., a 32 M‑parameter model becomes ~30 MB on disk).
Speed Inference can be hundreds of times faster on a CPU because the model only looks up token vectors instead of running a full transformer.
Zero‑data distillation You can create a Model2Vec model from any existing sentence‑transformer in ~30 seconds on a CPU, without needing a training corpus.
Fine‑tuning for classification After distillation you can further train a lightweight classifier on top of the static vectors for custom tasks.
Lightweight dependencies The core package only depends on numpy; optional extras add distillation (model2vec[distill]) and training (model2vec[train]).
Integration Ready‑to‑use with popular ecosystems such as Sentence‑Transformers and LangChain, and models are hosted on the Hugging Face hub for easy from_pretrained loading.
Open‑source & MIT‑licensed Free to use, modify, and embed in commercial projects.

Typical use‑cases

  • Retrieval / RAG – fast token‑level embeddings for building similarity search indexes.
  • Text classification – train a lightweight classifier on top of static vectors.
  • Clustering / visualization – cheap to compute embeddings for large corpora.
  • Low‑resource deployment – embed on edge devices or servers where memory/CPU is limited.

Quick start (from the README)

# Install the core library (only numpy is required)
pip install model2vec

from model2vec import StaticModel

# Load a pre‑distilled model from Hugging Face
model = StaticModel.from_pretrained("minishlab/potion-base-32M")

# Get sentence embeddings
emb = model.encode(["It's dangerous to go alone!", "It's a secret to everybody."])

# Or get token‑level embeddings
token_emb = model.encode_as_sequence(["It's dangerous to go alone!"])

Distill your own model (optional, needs the distill extra):

pip install model2vec[distill]

from model2vec.distill import distill
m2v = distill(model_name="BAAI/bge-base-en-v1.5")   # ~30 s on CPU
m2v.save_pretrained("my_m2v_model")

Fine‑tune for classification (needs the train extra):

pip install model2vec[train]

from model2vec.train import StaticModelForClassification
from datasets import load_dataset

clf = StaticModelForClassification.from_pretrained("minishlab/potion-base-32M")
ds = load_dataset("setfit/subj")
clf.fit(ds["train"]["text"], ds["train"]["label"])
report = clf.evaluate(ds["test"]["text"], ds["test"]["label"])

Model catalogue (excerpt)

Model Language Base sentence‑transformer Params Typical task
potion-base-32M English bge-base-en-v1.5 32 M General purpose
potion-multilingual-128M Multilingual bge-m3 128 M General purpose
potion-retrieval-32M English bge-base-en-v1.5 32 M Retrieval
potion-base-8M English bge-base-en-v1.5 7.5 M General purpose (smallest)
potion-code-16M-v2 Code CodeRankEmbed 16 M Code embeddings
All are hosted on the Hugging Face hub and can be loaded with StaticModel.from_pretrained.

Documentation & community

Licensing & citation

  • License – MIT (permissive, commercial‑friendly).
  • Citation – Provided BibTeX entry for academic use.

Bottom line

Model2Vec gives you the speed and size advantages of classic static embeddings (like GloVe) while retaining the quality of modern transformer‑based sentence encoders. It’s a practical tool for anyone needing fast, low‑memory text representations—especially in retrieval‑augmented generation, on‑device NLP, or large‑scale indexing scenarios.

Related