lightonai/next-plaid

What this project is

Two related tools built by LightOn AI:

• NextPlaid — a search database that stores multi-vector representations of text and serves search queries over a REST API. It runs locally on your machine.
• ColGREP — a code-search tool built on NextPlaid. You type a question like "database connection pooling" and it finds relevant code in your project, combining old-school regex matching with semantic ranking. Everything runs on your machine; your code never leaves it.

Both are open source (Apache-2.0).

ColGREP: semantic code search

ColGREP is a single Rust binary — no server, no API, no dependencies. Install with Homebrew or a shell installer, then:

colgrep init /path/to/project   # build the index once
colgrep "database connection pooling"   # search

The index is built automatically on first use and updated incrementally as files change.

Two search modes:

• Direct search — semantic ranking over your codebase. You can also combine it with regex: colgrep -e "async.*await" "error handling" narrows results by pattern and meaning.
• Agent mode — colgrep --agent "where is the database connection pool configured" downloads a small local model (one-time, 2.5 GB) that acts like a sub-agent: it searches, reads candidate files, and returns the exact line numbers that answer the question. A large coding model can delegate to it, saving its own context window.

How ColGREP indexes code

1. Parse — Tree-sitter parses your code into functions, methods, classes.
2. Structure — each code unit is turned into structured text: signature, parameters, calls, docstring, code body.
3. Embed — a small model (lightonai/LateOn-Code-edge, 17M parameters) converts that structured text into multiple vectors per code unit (multi-vector embedding). Runs on CPU.
4. Index — NextPlaid stores these vectors in a quantized, memory-mapped binary index.

This structured representation gives the model richer signal than raw code alone.

Model switching

You can swap the embedding model (colgrep set-model, --model flag, colgrep status to see what an index was built with). Each (project, model) pair has its own index, so switching never corrupts existing indexes. Private HuggingFace models work with an HF_TOKEN.

Agent integrations

Installs hooks for Claude Code (--install-claude-code), OpenCode, Codex, and Hermes. Claude has full hooks support; the other three are basic for now.

Benchmark

On the Semble public code-search benchmark (1,251 queries × 63 repos × 19 languages), ColGREP scores NDCG@10 = 0.859 with the LateOn-Code model, 0.846 with the smaller edge model — edging out a prior model (semble potion-code-16M at 0.853).

Why multi-vector?

Traditional vector search squeezes a whole document into one embedding — a lossy summary. That's poor for code, where a single function has a name, parameters, docstring, control flow, and dependencies.

Multi-vector search keeps ~300 embeddings of dimension 128 per document. At query time, each query token finds its best match across all document tokens (a technique called MaxSim). The tradeoff — more storage — is what NextPlaid solves with quantization and memory-mapped indexing.

NextPlaid: the underlying engine

NextPlaid is a general-purpose, local-first multi-vector database with a REST API. Features:

• Built-in encoding — send raw text, get results; no separate inference server. It ships with ONNX Runtime for ColBERT models.
• Memory-mapped indices — indices live on disk and are paged in on demand, so RAM footprint stays low.
• Product quantization — 2-bit or 4-bit compression; a million documents fit in memory.
• Incremental updates — add/delete documents without rebuilding.
• Metadata pre-filtering — SQL WHERE clauses on a built-in SQLite store run before scoring, so only matching documents get scored.
• CPU-optimized with optional CUDA support.

Relationship to FastPlaid: FastPlaid is a GPU batch indexer for large, single-pass workloads. NextPlaid wraps the same algorithm in a production API geared to streaming ingestion and serving.

Quick start: run the Docker container (CPU or CUDA image), then use the Python client (pip install next-plaid-client) to create indexes, add documents, search, filter by metadata, and delete. A Swagger UI is available at http://localhost:8080/swagger-ui.

API benchmarks

Measured end-to-end, including encoding time (the model runs inside the API). Setup: GTE-ModernColBERT-v1 on an NVIDIA H100 80GB, top_k=100. Sample results on BEIR datasets:

• quora (522,931 docs): NDCG@10 = 0.8519, 105 docs/sec indexing, ~21 QPS on GPU, ~18 QPS on CPU.
• nfcorpus (3,633 docs): NDCG@10 = 0.3828, ~7 QPS GPU.
• scifact (5,183 docs): NDCG@10 = 0.7593.

The full table covers six datasets with MAP, NDCG@10/100, Recall@10/100, throughput, and latency percentiles.

Models

Any HuggingFace ColBERT-style model can be exported to ONNX format using pylate-onnx-export (FP32 and INT8 variants by default; INT8 is ~4× smaller and faster with minimal quality loss). Ready-to-use models:

Model	Use case
lightonai/LateOn-Code-edge	Code search, lightweight
lightonai/LateOn-Code	Code search, accurate
lightonai/mxbai-edge-colbert-v0-32m-onnx	Text retrieval, lightweight
lightonai/answerai-colbert-small-v1-onnx	Text retrieval, lightweight
lightonai/GTE-ModernColBERT-v1	Text retrieval, accurate

Bottom line

ColGREP puts semantic code search on your own machine with a single binary; NextPlaid is the reusable multi-vector database underneath it. If you need bulk GPU indexing instead of streaming serving, the sibling project FastPlaid is the alternative.