Anush008/fastembed-rs

What it solves

FastEmbed-rs provides a lightweight, local-first approach to generating vector embeddings and reranking documents. It eliminates the need for external API calls or heavy dependencies, allowing developers to integrate text, image, and sparse embeddings directly into Rust applications with minimal setup.

How it works

The library leverages the ONNX Runtime (via ort) for performant inference and Hugging Face's tokenizers for fast text encoding. It supports a wide variety of pre-trained models from Hugging Face, which are automatically downloaded and cached locally. For specific high-performance models like Qwen3 or Nomic MoE, it utilizes a candle backend.

Who it’s for

Rust developers building RAG (Retrieval-Augmented Generation) pipelines, search engines, or similarity search applications who want to run embedding models locally on CPU or GPU (via DirectML on Windows).

Highlights

• Multimodal Support: Generates embeddings for text, images, and sparse text representations.
• Local Inference: No network required at runtime once models are cached.
• Extensive Model Library: Supports a vast array of BGE, sentence-transformers, Nomic, and Qwen3 models, including quantized versions.
• Reranking Capabilities: Includes built-in support for reranking candidates to improve search precision.
• Zero-Tokio Dependency: Supports synchronous usage without requiring an async runtime.
• Similarity Helpers: Includes built-in functions for cosine similarity and top-k ranking.