StarTrail-org/LEANN

LEANN – A Tiny, Privacy‑First Vector Database for RAG

What it is – LEANN (Lightweight Embedding‑Aware Nearest‑Neighbor) is an open‑source vector‑search engine that stores almost no embedding vectors. It builds a graph of document chunks, recomputes embeddings on‑the‑fly when a query arrives, and prunes the graph so that the index size stays tiny (the authors claim ~97 % less storage than traditional vector DBs) while keeping search quality.

Why it matters – Traditional vector databases keep every high‑dimensional embedding on disk, which quickly balloons to hundreds of gigabytes for millions of text chunks. LEANN’s on‑demand recomputation lets you run a full‑text‑scale Retrieval‑Augmented Generation (RAG) pipeline on a laptop with zero cloud cost and full privacy.

Key features

• Graph‑based selective recomputation – only the most‑needed embeddings are recomputed at query time.
• High‑degree preserving pruning – keeps the navigation graph small without hurting recall.
• Multiple back‑ends – HNSW (default) and DiskANN for fast approximate nearest‑neighbor search.
• Zero‑telemetry, local‑only – all data stays on your machine; no OpenAI or cloud calls are required unless you explicitly use an LLM provider.
• MCP integration – can expose the index as a Model Context Protocol service, letting other tools (Claude Code, Slack bots, etc.) query it.
• Broad data support – PDFs, txt/md, Apple Mail, browser history, chat logs (WeChat, iMessage, ChatGPT, Claude), live data streams, and codebases.
• Plug‑and‑play LLM back‑ends – OpenAI‑compatible APIs, Ollama, HuggingFace, Anthropic, etc., for generation and embeddings.

Typical workflow

1. Install – uv pip install leann (or build from source for custom back‑ends).
2. Build an index – point LEANN at a folder of documents; it chunks the text, builds a graph, and writes a .leann file.
3. Search – call LeannSearcher or the CLI to retrieve the most relevant chunks.
4. Chat – wrap the searcher with LeannChat and an LLM to ask natural‑language questions over your data.

from leann import LeannBuilder, LeannSearcher, LeannChat

# 1️⃣ Build
builder = LeannBuilder(backend_name="hnsw")
for txt in ["…", "…"]:
    builder.add_text(txt)
builder.build_index("mydata.leann")

# 2️⃣ Retrieve
searcher = LeannSearcher("mydata.leann")
print(searcher.search("What does LEANN save?", top_k=3))

# 3️⃣ Chat
chat = LeannChat("mydata.leann", llm_config={"type": "hf", "model": "Qwen/Qwen3-0.6B"})
print(chat.ask("How much storage does LEANN save?"))

Performance claim – The README cites a benchmark where indexing 60 M text chunks occupies ~6 GB (vs. >200 GB for a conventional vector DB) and a RAG‑enabled coding agent achieved 2.1× higher relevant‑code recall than a BM25 baseline on the ContextBench suite.

Who should try it –

• Developers who want a local, privacy‑preserving RAG store for personal files, emails, or code.
• Researchers experimenting with retrieval strategies without paying for cloud vector services.
• Teams building AI assistants that must run offline or on‑device (e.g., on a laptop or edge device).

Getting started – Clone the repo, create a uv virtual environment, install the leann package, and run the provided demo.ipynb or the CLI examples listed in the README. Detailed OS‑specific build instructions are included for Ubuntu, Arch, macOS, and Windows.

---

All information above is taken directly from the repository’s README; no external claims have been added.