RyanCodrai/turbovec

A vector index built on TurboQuant, written in Rust with Python bindings

What it solves

Vector databases often require massive amounts of RAM to store high-dimensional embeddings (e.g., 31 GB for 10 million documents). Turbovec solves this by using advanced quantization to compress these vectors—reducing memory usage by up to 16x—while maintaining high search speed and recall.

How it works

Built in Rust with Python bindings, the engine uses the TurboQuant algorithm to compress vectors without a separate training phase. It works through several steps:

  1. Normalization: Strips vector length and stores it separately.
  2. Random Rotation: Applies a random orthogonal matrix so coordinates follow a predictable distribution.
  3. Calibration (TQ+): Fits scalars to match the empirical distribution of the data, allowing for quantization without retraining.
  4. Lloyd-Max Quantization: Uses precomputed optimal buckets to compress coordinates into 2-bit or 4-bit integers.
  5. SIMD Search: Uses hand-written SIMD kernels (AVX-512/AVX2 for x86 and NEON for ARM) to score compressed vectors directly without full decompression.

Who it’s for

It is designed for developers building Retrieval-Augmented Generation (RAG) applications where memory efficiency, low latency, and data privacy (local/air-gapped execution) are critical.

Highlights

  • Extreme Compression: Reduces 1536-dim vectors from 6,144 bytes down to 384 bytes using 2-bit quantization.
  • No Training Required: Supports online ingestion where vectors are indexed immediately without a separate training or rebuild step.
  • High Performance: Outperforms FAISS on ARM and matches/beats it on x86 in various configurations.
  • Framework Ready: Provides drop-in replacements for LangChain, LlamaIndex, Haystack, and Agno.