qdrant/fastembed

Fast, Accurate, Lightweight Python library to make State of the Art Embedding

What it solves

FastEmbed 是一个轻量级的 Python 库，用于快速生成文本、图像和多模态嵌入。它使用 ONNX Runtime，避免了沉重的 PyTorch 依赖和 GPU 要求，非常适合服务器无状态环境（如 AWS Lambda）以及对速度和低资源消耗有严格要求的应用。

How it works

FastEmbed 采用 ONNX Runtime 替代 PyTorch 来执行模型，显著降低内存占用，且无需下载数 GB 的依赖。通过数据并行加速大规模数据集的编码。库支持多种嵌入类型，包括 dense、sparse（SPLADE++）、late interaction（ColBERT）以及 multimodal（ColPali）嵌入，还支持 rerankers（Cross Encoders）。

Who it’s for

适用于需要生成嵌入但不想引入沉重机器学习框架的 AI 开发者，尤其是部署到无服务器运行时或追求高性能 CPU 推理的场景。

Highlights

  • Lightweight Architecture: Uses ONNX Runtime to avoid PyTorch dependencies and GPU requirements by default.
  • Versatile Model Support: Supports dense text, sparse text, and image embeddings, as well ...
  • Broad Compatibility: Includes built-in support for popular models and the ability to add custom models from Hugging Face.
  • Qdrant Integration: Seamlessly integrates with the Qdrant vector database for easy collection creation and data upload.
  • GPU Acceleration: Optional GPU support via the fastembed-gpu package for increased performance。