RyanCodrai/turbovec

A vector index built on TurboQuant, written in Rust with Python bindings

What it solves

turbovec addresses the high memory and latency costs associated with large-scale vector search. It allows massive corpora (e.g., 10 million documents) to be stored in a fraction of the RAM (4 GB vs 31 GB for float32) while maintaining high search speeds and recall, eliminating the need for expensive training phases or frequent index rebuilds.

How it works

It implements the TurboQuant algorithm, a data-oblivious quantizer that compresses vectors without needing a separate training phase. The process involves:

  1. Normalization: Vectors are converted to unit directions on a hypersphere.
  2. Random Rotation: Vectors are multiplied by a random orthogonal matrix to make coordinate distributions predictable (Beta distribution).
  3. Calibration (TQ+): Optional per-coordinate shift and scale are applied to map empirical quantiles to the codebook's centroids for better recall.
  4. Lloyd-Max Quantization: Coordinates are bucketed into small integers (e.g., 4 buckets for 2-bit, 16 for 4-bit) based on precomputed optimal boundaries.
  5. Bit-packing: These integers are packed tightly into bytes for extreme compression.
  6. SIMD Acceleration: Hand-written kernels (NEON for ARM, AVX-512 for x86) perform fast search and filtering directly on the compressed data.

Who it’s for

It is designed for developers building Retrieval-Augmented Generation (RAG) systems where privacy, memory efficiency, and low latency are critical, particularly those operating in air-gapped or local environments.

Highlights

  • No Training Phase: Online ingest allows adding vectors without parameter tuning or index rebuilds.
  • Extreme Compression: Up to 16x reduction in memory usage compared to float32.
  • High Performance: Outperforms FAISS IndexPQFastScan in search speed on both ARM and x86 architectures.
  • Incremental Persistence: The sync(path) method saves only changes since the last sync, ensuring crash-safe and fast updates.
  • Search-time Filtering: Supports an id allowlist that is honored directly within the SIMD kernel to avoid over-fetching.
  • Framework Ready: Provides drop-in replacements for vector stores in LangChain, LlamaIndex, Haystack, and Agno.

Related

  • Dispatch
    Turbovec: High-Performance Vector Search using Google's TurboQuant

    Turbovec is a Rust-based vector index implementing the TurboQuant algorithm, offering up to 16x memory compression and faster search speeds than FAISS IndexPQFastScan.

  • Project
    sqliteai/sqlite-vector

    A production-grade SQLite extension that brings efficient vector search and quantization to embedded databases for Edge AI and RAG applications.

  • Project
    SeekStorm/SeekStorm

    A high-performance Rust-based search engine providing native hybrid vector and lexical search with sub-millisecond latency for billion-scale indices.

  • Project
    facebookresearch/faiss

    A library for efficient similarity search and clustering of dense vectors, capable of handling billion-scale datasets using CPU and GPU acceleration.

  • Project
    marcelroed/gigatoken

    A high-performance Rust-based tokenizer for language modeling that provides up to 1000x speedup over HuggingFace tokenizers by using SIMD-optimized pretokenization.