1yefuwang1/vectorlite

Fast, SQL powered, in-process vector search for any language with an SQLite driver

What it solves

Vectorlite is a runtime-loadable extension for SQLite that adds fast vector search capabilities. It allows developers to perform approximate nearest neighbor (ANN) searches directly within a SQL interface, eliminating the need for a separate standalone vector database while maintaining high performance across Windows, MacOS, and Linux.

How it works

The project implements a SQLite virtual table module backed by the hnswlib library for efficient indexing and searching. It uses Google's highway library for SIMD-accelerated vector distance calculations, which significantly speeds up computations for higher-dimensional vectors. Users can create virtual tables for vectors, insert data using standard SQL, and query them using specific functions like knn_search and knn_param to find the closest vectors based on L2, cosine, or inner product distances.

Who it’s for

It is designed for developers who use SQLite and want to add vector search to their local databases without the overhead of managing a separate vector database server.

Highlights

  • High Performance: Uses HNSW for ANN search and SIMD acceleration via Google Highway, claiming significantly faster query speeds than similar SQLite extensions.
  • Cross-Platform: Pre-compiled for Windows, Linux, and MacOS (x64 and ARM).
  • Tunable: Provides full control over HNSW parameters (like ef_construction and M) to balance speed and accuracy.
  • Metadata Filtering: Supports predicate pushdown for rowid filters, allowing users to combine vector search with metadata filtering in a single query.
  • Persistence: Supports saving and loading vector indices to and from disk via SQL commands.
  • Flexible API: Includes helper functions for JSON serialization/deserialization of vectors.

Related

  • Project
    sqliteai/sqlite-vector

    A cross-platform SQLite extension that adds production-grade vector search and quantization to SQLite, enabling efficient semantic search and RAG on the edge and mobile devices.

  • Project
    VexDB-THU/VexDB-Lite

    A high-performance vector similarity search engine for PostgreSQL, DuckDB, and SQLite that uses a custom graph index and quantization to enable efficient ANN search.

  • Project
    pgvector/pgvector

    An open-source PostgreSQL extension that enables vector similarity search, allowing users to store embeddings and perform nearest neighbor queries within their relational database.

  • Project
    alibaba/AliSQL

    An open-source MySQL branch that integrates DuckDB for high-performance analytics and native HNSW vector search for AI applications.

  • Project
    datastax/jvector

    A graph-based approximate nearest neighbor search index that combines HNSW and DiskANN to enable fast, memory-efficient vector search over massive datasets.