Anush008/fastembed-rs

Rust library for generating vector embeddings, reranking locally!

What it solves

FastEmbed-rs is a Rust library designed for generating vector embeddings and reranking locally. It removes the need for external API calls by allowing developers to run high-performance embedding models directly on their own hardware, supporting text, images, and sparse embeddings.

How it works

The library leverages the ONNX Runtime (via the ort crate) for performant inference and the Hugging Face tokenizers library for fast text encoding. It supports a wide variety of pre-trained models from Hugging Face, which are automatically downloaded and cached locally. For specific advanced models like Qwen3 and Nomic v2 MoE, it uses a candle backend.

Who it’s for

Rust developers who need to integrate embedding generation or reranking into their applications without relying on cloud services, particularly those building RAG (Retrieval-Augmented Generation) pipelines or similarity search tools.

Highlights

  • Multimodal Support: Generates embeddings for text, images, and sparse text representations.
  • Local Reranking: Includes built-in support for reranking candidates to improve search precision.
  • BGE-M3 Integration: Can produce dense, sparse, and ColBERT embeddings in a single forward pass.
  • Hardware Acceleration: Supports CPU inference and GPU acceleration via DirectML on Windows.
  • Zero-Dependency Async: Supports synchronous usage and does not depend on Tokio.
  • Built-in Similarity Tools: Provides helper functions for cosine similarity and top-k ranking without requiring additional crates.