dragonflydb/dragonfly

What it solves

Dragonfly is a high-performance, in-memory data store designed to overcome the throughput and resource limitations of legacy systems like Redis and Memcached. It addresses the bottleneck of single-threaded architectures by scaling vertically to utilize modern multi-core CPUs and high-memory cloud instances, reducing the resources needed for the same workload by up to 80%.

How it works

Dragonfly employs a "shared-nothing" architecture, partitioning the keyspace into shards managed by individual threads to maximize CPU and I/O utilization. To ensure atomicity for multi-key operations without relying on mutexes or spinlocks, it uses a transactional framework based on the VLL lock manager redesign. Its core hashtable is based on the Dash scalable hashing design, which enables incremental hashing, stateless scan operations, and a fork-less snapshotting algorithm.

Who it’s for

It is built for developers and organizations running modern cloud application workloads that require an ultra-fast, cost-efficient in-memory store with full compatibility for Redis and Memcached APIs.

Highlights

• High Throughput: Capable of reaching up to 25X the throughput of a single Redis process, crossing 3.8M QPS.
• API Compatibility: Fully compatible with Redis and Memcached APIs, requiring no code changes for adoption.
• Memory Efficiency: Up to 30% more memory efficient than Redis in idle states and utilizes a fork-less snapshotting process to avoid memory spikes.
• Adaptive Caching: Features a unified, zero-overhead caching algorithm that evicts items based on future access likelihood when near memory limits.