SeekStorm/SeekStorm

What it solves

SeekStorm is a high-performance search engine designed to provide sub-millisecond latency for both vector and lexical (keyword) search. It addresses the need for a unified search system that can handle billion-scale indices on a single commodity server without requiring expensive clusters or proprietary hardware accelerators, while maintaining stable performance during heavy indexing.

How it works

SeekStorm employs a "twin-core" architecture using two separate, native index architectures: an inverted index for lexical relevance and an ANN (Approximate Nearest Neighbor) index for vector similarity. A query planner then decides how to combine these results using various fusion types, such as Reciprocal Rank Fusion (RRF), to provide hybrid search capabilities. It is written in Rust and utilizes SIMD hardware acceleration for both x86-64 and AArch64 architectures.

Who it’s for

It is intended for developers building search-intensive applications, RAG (Retrieval-Augmented Generation) systems, or services requiring real-time, low-latency search across massive datasets across various platforms (Windows, Linux, MacOS).

Highlights

• Hybrid Search: Native integration of vector and lexical search engines under a single query planner.
• Performance: Sub-millisecond query latency and high-throughput sharded indexing (up to 35K docs/sec on a laptop).
• Integrated Inference: Ability to generate and index embeddings directly from text fields using Model2Vec.
• Versatility: Available as both an in-process library and a multi-tenancy REST server with SDKs for Rust, Python, C#, Java, and TypeScript.
• Advanced Search Features: Supports faceted search, geo-proximity search, typo-tolerance, and real-time indexing where documents are searchable immediately.
• Scalability: Supports billion-scale indices with low RAM consumption and no need for resource-intensive segment merges.