timescale/pg_vectorsearch

What it solves

This project provides a high-performance vector search index for PostgreSQL, designed to handle large-scale datasets (100M+ vectors) more efficiently than existing alternatives. It addresses the performance bottlenecks of traditional vector indexing methods by using advanced quantization and hierarchical partitioning.

How it works

It introduces the PRISM (Partitioned Routing Index for Similarity Matching) index, a native PostgreSQL access method. It uses a hierarchical centroid tree to cluster vectors and RaBitQ quantization to compress data while maintaining theoretical error bounds. During a search, the system traverses the centroid tree to identify relevant clusters, filters candidates using the quantized data, and performs a final high-precision reranking. It leverages SIMD instructions (AVX2, AVX512, NEON) for rapid distance calculations and integrates directly with the PostgreSQL buffer cache.

Who it’s for

This is for database administrators and developers building AI-driven applications on PostgreSQL who need to perform fast, scalable similarity searches on large vector embeddings and want to optimize memory usage and query latency.

Highlights

• High Performance: Significantly faster query and index build times compared to IVFFlat and HNSW for large datasets.
• Memory Efficient: Achieves high recall on 100M+ vectors using significantly less memory than graph-based alternatives.
• pgvector Compatible: Can be used as a drop-in replacement for existing vector or halfvec columns without requiring table rewrites.
• Advanced Quantization: Uses RaBitQ to reduce the need for full-precision reranking, speeding up search operations.
• SIMD Optimized: Utilizes modern CPU instruction sets for accelerated distance computations.