milvus-io/milvus-lite

What it solves

Milvus Lite provides a lightweight, local version of the Milvus vector database, eliminating the need for complex infrastructure setup for developers who need vector search capabilities during prototyping, testing, or for small-scale AI applications. It allows users to run a fully functional vector store as a local file or an embedded server, ensuring that code written for local development can be easily migrated to production-grade Milvus deployments (Standalone or Distributed) or Zilliz Cloud with minimal changes.

How it works

Rebuilt from scratch in pure Python, Milvus Lite uses a local LSM-style storage engine. It manages data through a Write-Ahead Log (WAL), in-memory memtables, and immutable Parquet segments. For search, it utilizes FAISS-backed vector indexes and INVERTED scalar indexes. It can be operated in three ways: as a local .db file via the pymilvus client, as a standalone local gRPC server for multi-client access, or directly through its internal Embedded Engine API.

Who it’s for

• AI Developers: Those building RAG applications or AI agents that need a local vector store for rapid prototyping.
• Data Scientists: Users working in notebooks (like Jupyter) who want to integrate vector search without managing a separate database server.
• QA Engineers: Developers needing a lightweight, isolated environment for running automated tests against a vector database.

Highlights

• Drop-in Local Usage: Start a database with a single line of code using MilvusClient("./demo.db").
• Hybrid Search: Supports dense vector search, sparse BM25 full-text search, and combined hybrid search.
• Pure Python: Implemented in Python for easier inspection and debugging with standard Python stack traces.
• Rich Filtering: Implements Milvus-style scalar, JSON path, array, and geometry (WKT) filters.
• Seamless Migration: Compatible with the pymilvus API, allowing easy transition to full Milvus clusters.
• Data Export: Includes a dump command to export collections to JSON files for bulk insertion into production Milvus environments.