usemoss/moss

The retrieval layer for production AI systems. Lightning-fast (<10ms) search without vector databases. Built for browser, edge, on-device, and cloud.

What it solves

Most retrieval stacks rely on remote vector databases, introducing network latency (200–500 ms) that can make real-time conversational AI agents feel laggy. Moss eliminates this "network hop on the hot path" by running search and embedding directly inside the application process, achieving sub-10 ms query latency.

How it works

Moss operates as a search runtime rather than a traditional database. It consists of three components:

  1. Moss Cloud: Manages document ingestion, embedding, storage, and distribution.
  2. Index: A packaged artifact of documents and vectors stored on Moss Cloud.
  3. Runtime: An SDK embedded in the application that pulls indexes over HTTPS, holds them in memory, and executes queries locally.

It supports server-side execution (Python, Node.js, Elixir, C) and client-side execution via a WebAssembly build for browsers.

Who it’s for

Developers building real-time AI agents, such as voice bots, copilots, or any conversational interface where low latency is critical to the user experience.

Highlights

  • Ultra-low latency: Sub-10 ms end-to-end query time (embedding + search).
  • Hybrid search: Combines semantic and keyword search in a single query.
  • Built-in embeddings: Includes embedding models out-of-the-box, removing the need for external API keys.
  • Flexible deployment: Runs in the backend or entirely in the browser via WASM.
  • Broad integration: Compatible with frameworks like LangChain, LlamaIndex, DSPy, and various voice AI platforms (Vapi, ElevenLabs, LiveKit).