Dragonfly

Profile

Dragonfly represents a modern in-memory data store engineered as a high-performance, drop-in replacement for Redis and Memcached. Built from the ground up with a multi-threaded, shared-nothing architecture, it addresses fundamental limitations of single-threaded database designs that struggle with contemporary cloud infrastructure. Dragonfly achieves approximately 25 times higher throughput than Redis while reducing memory consumption by up to 80 percent on equivalent hardware. The project operates under Business Source License 1.1, which automatically converts to Apache 2.0 after four years, providing a transparent path to full open source status. Developed by DragonflyDB Inc. and backed by institutional investors including Redpoint Ventures and Quiet Capital, Dragonfly maintains active development with regular releases addressing performance optimization, stability improvements, and expanded feature support for production workloads.

Focus

Dragonfly solves the performance ceiling and infrastructure cost challenges inherent in single-threaded in-memory data stores. Organizations running Redis at scale encounter throughput limitations requiring expensive horizontal scaling through complex cluster configurations, while Dragonfly's architecture enables vertical scaling on multi-core processors to handle millions of operations per second on single instances. The platform targets engineering teams building high-performance applications requiring sub-millisecond latency for caching layers, feature stores for machine learning operations, real-time session management, and vector search applications. Platform engineers benefit from seamless Redis and Memcached API compatibility, enabling migration with only connection endpoint changes and no code modifications. The architecture eliminates operational complexity of managing distributed clusters while delivering superior memory efficiency through optimized data structures and custom allocators.

Background

DragonflyDB was founded in March 2022 by Oded Poncz and Roman Gershman, former Google engineers who experienced Redis scaling challenges firsthand. The project achieved general availability with version 1.0 in March 2023, coinciding with $21 million in funding from Redpoint Ventures and Quiet Capital. The technology has demonstrated production viability through enterprise deployments, including India's largest social media company migrating over 150 services to achieve measurable cost reductions and performance improvements. DragonflyDB Inc. maintains concentrated governance with strategic direction controlled by company leadership rather than distributed foundation structures. The project demonstrates sustained engineering investment through regular version releases addressing diverse functional areas including JSON optimization, vector search capabilities, and stability improvements across core subsystems.

Main features

Multi-threaded shared-nothing architecture

Dragonfly implements a fundamentally different execution model than Redis by fully utilizing multi-core processors through a shared-nothing architecture where each thread owns a distinct subset of the keyspace. Keys are internally sharded across threads using deterministic hash functions, eliminating lock contention for most operations while maintaining consistency through specialized cross-shard transaction mechanisms. This design enables approximately four million operations per second on single instances compared to Redis's 500,000 operations per second, representing an eight-fold throughput improvement. The architecture achieves P99 latency under four milliseconds while handling concurrent workloads, making it suitable for latency-sensitive applications including payment processing, real-time auctions, and machine learning inference pipelines requiring sub-millisecond response times.

Memory-efficient data structures and storage optimization

The platform employs custom memory allocators and compressed data structures called Dashtables that achieve two to four times better memory utilization than Redis for equivalent datasets. Compact string encoding provides superior handling of small strings, while reduced metadata structures minimize overhead across all data types. Real-world deployments demonstrate 55 to 56 percent memory usage reduction, enabling a dataset requiring 40 gigabytes in Redis to consume only 18 gigabytes in Dragonfly. This memory efficiency translates directly to infrastructure cost savings, allowing organizations to handle identical workloads on significantly smaller instances or fit substantially more data on equivalent hardware, reducing total cost of ownership while maintaining performance characteristics.

Complete Redis and Memcached protocol compatibility

Dragonfly maintains full API compatibility with both Redis and Memcached protocols, enabling organizations to migrate existing applications without code modifications. Developers continue using established client libraries, command-line interfaces, and operational tooling by simply changing connection endpoints to point to Dragonfly infrastructure. The compatibility layer supports core Redis commands, data structures including strings, lists, sets, sorted sets, hashes, and streams, plus advanced capabilities like transactions, pub/sub messaging, and Lua scripting. Organizations can leverage existing operational knowledge and monitoring infrastructure while gaining performance benefits, though certain specialized Redis modules may require compatibility assessment before migration for workloads depending on advanced extensions.