spiceai/spiceai

Add a real-time analytics node to your operational database. Spice is a portable, accelerated SQL query, search, and LLM-inference engine in Rust for data-grounded AI apps and agents.

What it solves

Spice solves the complexity of building data-grounded AI applications and agents by eliminating the need for complex data pipelines and glue code. It addresses the latency issues of querying large datasets by providing a high-performance engine that can run as a local sidecar or a distributed cluster, providing millisecond-latency access to data and AI inference.

How it works

Spice uses a cluster-sidecar architecture where a lightweight runtime runs on localhost next to an application to serve data and inference with minimal latency. It uses a tiered latency model: local results cache, a local working set, and transparent delegation to a distributed cluster for larger queries. The engine is built in Rust and leverages foundations like Apache DataFusion, Arrow, and Iceberg. It also features a real-time CDC (Change Data Capture) mechanism to create sandboxed, analytics-ready replicas of operational databases like PostgreSQL, MySQL, and MongoDB without impacting production load.

Who it’s for

  • AI Application Developers building agents that require fast, data-grounded retrieval and inference.
  • Data Engineers looking to implement real-time analytics on operational databases without traditional ETL processes.
  • Enterprise Teams needing secure, multi-tenant AI runtimes with built-in observability and secret management.

Highlights

  • AI-Native Runtime: Provides OpenAI-compatible APIs, text-to-SQL (NSQL), and integrated vector/text search primitives.
  • Hybrid Search: Supports petabyte-scale hybrid search using Amazon S3 Vectors, Tantivy BM25, and DuckDB HNSW.
  • Real-time CDC: Maintains analytics-ready replicas of databases with ~2-second freshness and zero production load.
  • High Performance: Features the Cayenne data accelerator for significantly faster queries and lower memory usage compared to DuckDB.
  • Federated Data Access: Connects to over 30 data sources including Snowflake, Databricks, S3, and Kafka with query push-down.