StarlightSearch/EmbedAnything

What it solves

EmbedAnything provides a high-performance, local embedding pipeline that simplifies the process of converting various data sources (text, images, audio, PDFs, websites) into vector embeddings and streaming them into vector databases. It eliminates the heavy dependency on PyTorch, reducing memory footprints and making cloud deployment easier.

How it works

Built in Rust for speed and memory safety, the project uses a modular architecture with multiple backends (Candle, ONNX, and cloud models). It employs "Vector Streaming," which separates document preprocessing from model inference using multi-threading (MPSC) to reduce latency and prevent memory leaks by saving embeddings directly to the database.

Who it’s for

AI engineers who need to build RAG (Retrieval-Augmented Generation) pipelines and want a lightweight, fast, and local way to generate and index embeddings across multiple modalities without the overhead of large ML frameworks.

Highlights

• Multi-modal Support: Handles text (PDF, MD, TXT), images (JPG), audio (.WAV), and websites.
• Flexible Backends: Supports Candle (for seamless Hugging Face model integration), ONNX (for faster inference), and cloud APIs (e.g., Cohere).
• Advanced Chunking: Includes built-in semantic chunking and late-chunking to better preserve context.
• Vector DB Integration: Provides adapters for popular databases including Elastic, Weaviate, Pinecone, Qdrant, Milvus, and Chroma.
• Hardware Acceleration: Supports GPU acceleration for increased performance.