How We Built a General Purpose Key Value Store for Facebook With ZippyDB
ZippyDB is the largest strongly consistent, geographically distributed key-value store at Facebook.
Since we first deployed ZippyDB in 2013, this key-value store has expanded rapidly, and today, ZippyDB serves a number of use cases, ranging from metadata for a distributed filesystem, counting events for both internal and external purposes, to product data that’s used for various app features.
ZippyDB uses RocksDB as the underlying storage engine.
Before ZippyDB, various teams across Facebook used RocksDB directly to manage their data. This resulted, however, in a duplication of efforts in terms of each team solving similar challenges such as consistency, fault tolerance, failure recovery, replication, and capacity management.
One of the significant design decisions we made early during the development of ZippyDB was to reuse as much of the existing infrastructure as possible. Consequently, most of our initial efforts were focused on building a reusable and flexible data replication library called Data Shuttle.