The-Vibe-Company/quivr

What it solves

Quivr V2 is an open-source engine designed to transform continuous streams of content into a searchable and monitorable knowledge base. It solves the problem of managing large volumes of unstructured data by ensuring that ingested content is durably stored, indexed for both keyword and semantic search, and monitored for specific topics or alerts in real-time.

How it works

Quivr V2 acts as a modular backend that orchestrates data ingestion, enrichment, and retrieval. It uses PostgreSQL for transactional storage, S3-compatible storage for raw artifacts, and Weaviate for hybrid search (lexical and vector). The system is built on a Go-based modular monolith and utilizes Temporal for durable task orchestration. When content is ingested, it is first made searchable via keyword indexing, while semantic enrichment (embeddings) happens in the background. The system supports a plugin architecture, allowing users to add custom connectors, normalizers, and delivery channels without modifying the core engine.

Who it’s for

This project is for developers and organizations who need to build custom RAG (Retrieval-Augmented Generation) applications, content monitoring systems, or internal search engines that require high reliability, data durability, and the ability to process large-scale, evolving datasets.

Highlights

• Durable Ingestion: Ensures content is committed to storage before processing, preventing data loss during outages.
• Hybrid Search: Combines fast keyword search with semantic vector search for comprehensive retrieval.
• Extensible Plugin Architecture: Core functionality is separated from domain-specific logic, allowing for custom connectors and enrichers.
• Idempotent Operations: Every write is tracked with an idempotency key, ensuring consistent results even if processes are replayed.
• Real-time Monitoring: Supports saved queries and subscriptions that trigger alerts when new content matches specific criteria.
• Observability: Includes built-in metrics, structured logging, and support for OpenTelemetry to track document lifecycles and system performance.