dragonflydb/dragonfly

A modern replacement for Redis and Memcached

What it solves

Dragonfly is a high-performance in-memory data store designed to replace legacy systems like Redis and Memcached. It addresses the limitations of single-threaded architectures by scaling vertically to utilize modern multi-core CPUs, significantly increasing throughput and reducing the resource footprint required for the same workload.

How it works

Dragonfly employs a "shared-nothing" architecture, partitioning the keyspace into shards managed by individual threads to eliminate contention. It uses a transactional framework based on the VLL lock manager redesign to ensure atomicity for multi-key operations without relying on mutexes or spinlocks. For data storage, it implements a hashtable based on the "Dash" scalable hashing design, which enables efficient record expiry, a zero-overhead cache eviction algorithm, and a fork-less snapshotting process.

Who it’s for

It is built for developers and organizations running modern cloud application workloads that require an ultra-fast, cost-efficient in-memory store with full compatibility for Redis and Memcached APIs.

Highlights

  • High Throughput: Capable of reaching up to 25X the throughput of single-process Redis, with benchmarks showing up to 10M-15M QPS in pipeline mode.
  • API Compatibility: Fully compatible with Redis and Memcached APIs, requiring no code changes for adoption.
  • Resource Efficiency: Can run on up to 80% fewer resources for equivalent workloads and is more memory efficient during snapshotting.
  • Built-in Tooling: Includes a native HTTP console and Prometheus-compatible metrics for monitoring and management.

Related

  • Project
    redis/redis

    Redis is a high-performance in-memory data structure server used as a cache, NoSQL database, and vector store to enable sub-millisecond latency for real-time applications.

  • Project
    NVIDIA/aistore

    A high-performance distributed storage stack tailored for AI workloads, providing linear scalability and optimized data retrieval for ML pipelines across multi-cloud environments.

  • Project
    hosseinmoein/DataFrame

    A high-performance C++ library for in-memory tabular data analysis, providing efficient data manipulation and statistical tools similar to Pandas but without Python overhead.

  • Project
    memvid/memvid

    A portable, single-file memory layer for AI agents that provides persistent long-term memory and instant retrieval without requiring a database.

  • Project
    milvus-io/milvus

    Milvus is a high-performance, distributed vector database built for scaling AI applications that require efficient search and organization of massive amounts of unstructured data.