ZippyDB Design

Learn the behavior and working of the ZippyDB key-value store, in detail.

In Tectonic design, metadata management is critical in achieving our goals of scalability, availability, and durability. A special-purpose key-value store (ZippyDB) is the cornerstone of our Metadata Store. In this lesson, we’ll focus on ZippyDB’s design. Following is the overall architecture of the Tectonic.

ZippyDBhttps://www.youtube.com/watch?v=ZRP7z0HnClc is a general-purpose, consistent, and geographically distributed key-value store created by Facebook on top of RocksDBRocksDB is a consistent key—value store–a fork of Google’s LevelDB. It uses log-structured database engines for maximum performance. It is optimized for fast and low-latency storage devices. It also provides basic and advanced database operations. Source: RocksDB’s official website.. ZippyDB is an end-to-end managed service that provides usual key-value operations (get, put, del, etc.) because of the underlying RocksDB storage engine, ZippyDB enables the system to perform a large amount of write operations efficiently while providing good performance for the read workload.

Note: RocksDB is primarily an efficient storage engine that can be embedded in other applications as a library. It frees the programmers from the messy details of efficient storage and lets them concentrate on their specific problems. RocksDB is optimized for write-heavy workload by using log-structured storage. The read performance is good because of the use of Bloom filtersThis is is a probabilistic data structure that saves space and can be used to determine whether an element is a part of a set or not..

ZippyDB uses basic key-value operations, snapshot readsThe process of creating a copy of the content which is going to be read., and partial read and writeThe process of reading or writing partial data from/to a storage node. The partial append, if not handled carefully, can leave different sizes of replica chunks which can cause inconsistency. operations. ZippyDB uses partial read and writes for low latency and durability (we’ll learn about it more in an upcoming ...