usemoss/moss

The retrieval layer for production AI systems. Lightning-fast (<10ms) search without vector databases. Built for browser, edge, on-device, and cloud.

What it solves

Most retrieval stacks for AI agents rely on remote vector databases, creating network round trips that add 200–500 ms of latency. This lag can break the flow of real-time conversational AI, such as voice bots or copilots, making them feel unresponsive.

How it works

Moss is a search runtime rather than a traditional database. It uses a hybrid architecture consisting of three parts:

  1. Moss Cloud: Manages document ingestion, embedding, storage, and distribution.
  2. Index: Packages documents and vectors into a single artifact stored on the cloud.
  3. Runtime: An SDK embedded directly into the application (available for Python, TypeScript, Elixir, and C). The runtime pulls the index over HTTPS and holds it in memory, allowing queries to be processed locally within the application process to eliminate network hops on the hot path.

It also offers a WebAssembly build for running semantic search entirely within the browser.

Who it’s for

Developers building real-time Conversational AI agents, voice bots, copilots, and offline-first applications that require ultra-low latency retrieval.

Highlights

  • Ultra-low latency: Achieves sub-10 ms query times (including embeddings) by running search in-process.
  • Hybrid Search: Combines semantic and keyword search in a single query.
  • Built-in Embeddings: Includes embedding models out-of-the-box, removing the need for external API keys.
  • Browser Support: A WASM-based SDK allows for client-side semantic search without a server.
  • Broad Integration: Compatible with major frameworks like LangChain, LlamaIndex, DSPy, and various voice AI platforms (Pipecat, Vapi, ElevenLabs).

Related

  • Project
    softmata/horus

    HORUS is a real‑time, lock‑free middleware for robotics that lets Rust, Python and C++ processes share data via shared‑memory ring buffers. It provides deterministic scheduling, safety watchdogs, zero‑copy AI pipelines, and claims sub‑200 ns cross‑process latency, positioning itself as a high‑performance alternative to ROS 2.

  • Project
    benchen4395/alpha_agentic_search

    A stable, low-latency multi-agent search framework featuring a 5-layer hierarchical memory RAG stack that improves over time as it archives successful answers.

  • Project
    lithos-ai/motus

    An open-source agent serving project that provides the infrastructure to build, serve, and deploy AI agents using plain Python, featuring a task-graph runtime for automatic parallelism.

  • Project
    infiniflow/infinity

    An AI-native database designed for LLM applications that provides high-speed hybrid search across dense and sparse embeddings, tensors, and full-text data.

  • Project
    OpenMOSS/MOSS-TTS

    A family of open-source speech and sound generation models providing high-fidelity TTS, multi-speaker dialogue, voice design, real-time streaming, and sound effect generation.