StarlightSearch/EmbedAnything

What it solves

EmbedAnything is a high-performance, local embedding pipeline designed to simplify the process of converting various data sources into vector embeddings. It removes the heavy dependency on PyTorch, reducing memory footprints and making deployment to the cloud easier. It specifically addresses the bottleneck of sequential processing by introducing "vector streaming," which separates document preprocessing from model inference to increase throughput and reduce latency.

How it works

Built in Rust for speed and memory safety, the project provides Python bindings (via PyO3) for ease of use. It supports multiple backends including Candle, ONNX, and cloud-based models. The pipeline handles the entire flow from ingestion (PDFs, text, images, audio) to chunking (semantic or late-chunking) and finally streaming the resulting embeddings directly into a vector database using modular adapters.

Who it’s for

Developers building RAG (Retrieval-Augmented Generation) applications who need a lightweight, fast, and memory-efficient way to generate embeddings locally or in the cloud without the overhead of full ML frameworks.

Highlights

• Multi-modal Support: Processes text (PDF, TXT, MD), images (JPG), and audio (.WAV).
• Vector Streaming: Concurrent workflow that separates file processing, indexing, and inferencing on different threads.
• Modular Adapters: Easy integration with various vector databases with minimal code.
• Diverse Model Support: Compatible with dense, sparse (SPLADE), late-interaction (ColBERT), and ReRanker models.
• Advanced Chunking: Includes built-in semantic and late-chunking strategies to preserve context.
• Hardware Acceleration: Supports GPU acceleration and is available as a pre-built Docker image.