anush008/fastembed-rs

What it solves

FastEmbed-rs provides a lightweight, high-performance Rust library for generating vector embeddings and performing reranking locally. It eliminates the need for external API calls or heavy dependencies like Tokio, allowing developers to integrate text, image, and sparse embeddings directly into their Rust applications with minimal overhead.

How it works

The library leverages the ONNX Runtime via the ort crate for performant inference and uses Hugging Face's tokenizers for fast text encoding. It supports a wide variety of pre-trained models from Hugging Face, which are automatically downloaded and cached locally. For specific advanced models like Qwen3 and Nomic MoE, it utilizes a candle backend.

Who it’s for

Rust developers building RAG (Retrieval-Augmented Generation) pipelines, search engines, or similarity-based applications who require local, fast, and synchronous embedding generation without relying on cloud APIs.

Highlights

• Multi-modal Support: Generates dense text embeddings, sparse text embeddings, and image embeddings.
• Reranking Capabilities: Includes built-in support for reranking candidates to improve search precision.
• Extensive Model Library: Supports a vast array of BGE, sentence-transformers, Nomic, and Qwen3 models, including quantized versions.
• Local-First: Models are cached locally after the first download, enabling offline runtime operation.
• Hardware Acceleration: Supports GPU acceleration on Windows via DirectML.
• Built-in Similarity Tools: Provides helper functions for cosine similarity and top-k ranking without requiring additional crates.