pg_vectorsearch: Timescale's PRISM Index Claims Faster Vector Search Than pgvector IVFFlat and HNSW

Timescale has released pg_vectorsearch, an open-source PostgreSQL extension that ships a new approximate nearest neighbor index called PRISM, which the project claims is significantly faster than pgvector's IVFFlat and HNSW for both querying and index building. The extension is explicitly labeled alpha and is meant for testing rather than production, but it introduces a distinct architectural bet: partitioning and compression instead of HNSW's graph traversal.

What pg_vectorsearch is

pg_vectorsearch is a PostgreSQL extension for high-performance approximate nearest neighbor (ANN) search, published by Timescale under the PostgreSQL License. Its headline component is PRISM (Partitioned Routing Index for Similarity Matching), a PostgreSQL-native index access method implemented in C.

The extension's stated feature set includes:

  • PRISM index for 100M+ vectors — IVF-style, with hierarchical routing
  • RaBitQ quantization with theoretical error bounds for two-stage search
  • SIMD-optimized distance computation (AVX2, AVX512, NEON)
  • Dynamic updates — inserts, updates and deletes, with posting-list splits via prism_rebalance
  • pgvector compatibility — it can index existing vector and halfvec columns as-is

The repository carries a prominent status warning: "Status: alpha. Meant for testing, not production. Upgrading between prereleases may not be possible; plan to drop and recreate the extension and its indexes."

How the PRISM index works

PRISM is conceptually similar to pgvector's IVFFlat, but replaces IVFFlat's flat list of centroids with a hierarchical centroid tree and adds quantization-based filtering. Timescale describes the result as "an index that is significantly faster than pgvector's IVFFlat and HNSW, for both querying and index building."

The index partitions vectors into clusters using a hierarchical centroid tree, and relies on the PostgreSQL buffer cache to keep that tree warm in memory. Vectors are RaBitQ-quantized and stored in posting lists with a page layout optimized for SIMD distance computation. The RaBitQ encoding carries error bounds that let a search rule out vectors that cannot possibly be nearest neighbors, so far fewer vectors need reranking against the full-precision values stored in the table.

The documented query flow is:

Query Flow:
1. Traverse centroid tree to find nearest clusters (buffer cache)
2. Scan posting lists for candidates (buffer cache, async I/O)
3. RaBitQ two-stage filtering: estimate → error bound → rerank
4. Full-precision reranking from heap
5. Return k nearest neighbors

The project cites RaBitQ (SIGMOD 2024), SPFresh's LIRE protocol (SOSP 2023), SPANN (NeurIPS 2021), and Google's ScaNN for AlloyDB whitepaper as its research foundations.

The performance claims

Timescale's accompanying announcement blog post makes specific benchmark claims. The extension's author, jascha_eng, quoted the post's summary on Hacker News:

TLDR: We are launching pg_vectorsearch, a new open-source PostgreSQL extension that makes vector search dramatically faster and more cost-efficient. Its new PRISM index uses partitioning and compression instead of HNSW's graph-based approach, delivering faster queries with a fraction of the memory and significantly faster index builds. At 100 million vectors, PRISM achieved 6 ms query latency on a machine with 4× less memory than HNSW, with index builds over 20× faster.

The GitHub README states that "compared to graph-based HNSW, PRISM is faster at equivalent recall on a 100-million-vector dataset, on an instance with a quarter of the memory."

Hacker News commenter esafak pushed back on how those numbers are presented, arguing the benchmarks should lead: "Shouldn't you lead with your benchmarks if you're fast? qps and precision/recall, please." The repository itself does not reproduce a full QPS-versus-recall table in the scraped content; it links to the announcement post and to a tuning guide.

Installation and usage

pg_vectorsearch requires PostgreSQL 18 or above with development headers, a GNU C11 compiler (GCC 11+ or Clang 14+), and the Meson build system. CBLAS is optional but speeds up RaBitQ encoding.

# Release build
meson setup builddir --buildtype=release

# Build and install to PostgreSQL
meson install -C builddir

There are two usage paths, and both build the same index and use the same operators (<->, <#>, <=>) and tuning knobs.

Native types

If you are starting fresh or not using pgvector, you store vectors in pg_vectorsearch's own vec32 (float32) or vec16 (float16) columns. No other extension is needed.

CREATE EXTENSION pg_vectorsearch;

-- PLAIN keeps the vector inline. EXTERNAL (the default) toasts a value
-- over ~2 kB, and rerank then pays an extra fetch the planner does not cost.
CREATE TABLE items (
    id serial PRIMARY KEY,
    embedding vec32(3) STORAGE PLAIN
);

-- Load first; the index is sized from the row count at CREATE INDEX.
-- 500 rows is enough for the planner to use the index.
INSERT INTO items (embedding)
SELECT ARRAY[i::real, (i % 7)::real, (i % 5)::real]::vec32
FROM generate_series(1, 1000) i;

-- For inner product or cosine, see Operator classes below.
CREATE INDEX ON items USING prism (embedding vec32_l2_ops);

-- 0 = auto (~0.95 recall). Raise for recall, lower for speed.
SET prism.nprobe = 40;

SELECT * FROM items
ORDER BY embedding <-> '[3,1,2]'
LIMIT 5;

Existing pgvector tables

If you already have a table with pgvector's vector or halfvec columns, you can build a PRISM index on it directly without changing the table or your queries. vec32 and vec16 are binary-compatible with pgvector's vector and halfvec but have different names, so both extensions can be installed side by side, even in the same schema.

CREATE EXTENSION vector;

CREATE TABLE documents (
    id        bigserial PRIMARY KEY,
    body      text,
    embedding vector(384)
);
CREATE INDEX documents_embedding_hnsw
    ON documents USING hnsw (embedding vector_l2_ops);

-- Install pg_vectorsearch next to pgvector and build the index on the
-- existing column. The column is not rewritten or copied.
CREATE EXTENSION pg_vectorsearch;

CREATE INDEX documents_embedding_prism
    ON documents USING prism (embedding vec32_l2_ops);

The README notes that PRISM is normally faster, so the planner should prefer it over the HNSW index while both exist, and recommends verifying with EXPLAIN before dropping the old index.

Operator classes

Pick the operator class for your distance metric and column type. L2 is the default.

Distance Operator vec32 / vector vec16 / halfvec
L2 <-> vec32_l2_ops vec16_l2_ops
Inner product <#> vec32_ip_ops vec16_ip_ops
Cosine <=> vec32_cosine_ops vec16_cosine_ops

When replacing an HNSW index, pgvector's vector_<metric>_ops maps to vec32_<metric>_ops and halfvec_<metric>_ops to vec16_<metric>_ops.

Tuning and maintenance

Defaults are tuned from large-scale benchmarks, and the documentation states that most deployments only ever adjust prism.nprobe. Setting prism.nprobe = 0 selects an automatic value targeting roughly 0.95 recall.

On maintenance, the README explains that an insert appends to whichever posting list its vector routes to, so lists grow as rows arrive and never split on their own. Besides REINDEX, the prism_rebalance procedure splits oversized lists in place.

Adoption constraints raised by the community

Hacker News commenter devy catalogued three practical blockers for anyone considering replacing an existing pgvector HNSW index:

  • pg_vectorsearch uses a quantized IVF index called PRISM. PRISM caps at 1968 dimensions for vectors. If your previous HNSW index has more dimensions than that, you're SOL (at least for now until they fix that).
  • pg_vectorsearch requires Postgres version 18 or above. If you run PG versions below that you need to migrate your database first. Many legacy PG production environment are way dated on PG 14 or 15, migration is prohibitively high cost to just try this out.
  • pg_vectorsearch shipped yesterday and it's obviously alpha. Hasn't been battle tested in production environments outside of Timescale's. Also since it's so new, it isn't on Azure's allowed-extension list or any other Hyperscale's managed PG instances' allowed extension list.

That third point was echoed by swe_dima: "Sadly if cloud providers don't offer it on managed postgres then not many people can use it..."

Filtering, quantization, and open questions

Several commenters raised capabilities the README does not advertise. Rapzid asked whether the index supports filtering and concluded from the documentation that it is post-filtering only: "Does it support filtering though? Edit: post filtering only I don't understand how people are efficiently implementing tenant/user/permission scopes with these indexes."

potomushto noted the absence of binary quantization and Hamming distance support: "Interesting, I see no mention of support for bit precision and hammer distance operator which supposed to be much faster and storage efficient."

Others asked for comparisons the project has not published. thomasahle asked how PRISM compares to the Roar graph, dmix asked how it compares to ParadeDB's pg_search, and jakobedding asked how it compares to pgvectorscale. arunmu went further, questioning the strategy: "Didn't these guys also write pgvectorscale which seems to be unmaintained now? Why a new index instead of adding onto or improving pgvectorscale?"

Competitive context

Commenter EgregiousCube reported running VectorChord in production and framed the release as part of a broader trend toward memory-efficient indexes: "I'm currently running VectorChord and I love it - with a fast backing disk, it's cut my postgres cost in half. Excited to see competition in low-RAM vector index tools. RAMageddon seems to be creating incentives to find more ways to take advantage of how fast modern nvme drives are!"

ilumanty expressed the opposite wish, that the same class of index existed outside PostgreSQL: "It would be so awesome if SQLite supported anything along these lines."

One lighter observation came from alexpadula, who noted the repository's PRISM logo image: "Dark side of the PRISM! The image on the repo 100% inspired by Dark Side of The Moon by Pink Floyd."

What to take away

pg_vectorsearch is a PostgreSQL 18+ extension that trades HNSW's graph structure for a hierarchical IVF-style partition tree plus RaBitQ quantization with error-bounded filtering. Timescale's own numbers claim 6 ms query latency at 100 million vectors on a machine with 4× less memory than HNSW, and index builds over 20× faster. Against that, the extension is alpha, requires PostgreSQL 18, is not yet on managed cloud allow-lists, and per community reports caps at 1968 dimensions and supports post-filtering rather than filtered search. The project is pgvector-compatible at the column level, so it can be evaluated alongside an existing HNSW index without rewriting tables or queries.

Sources