System Design Interview Prep Crash Course/

...

/

Design a Distributed Cache

Design a Distributed Cache

Learn to design a distributed caching system.

We'll cover the following...

Introduction

A typical system consists of service hosts, data storage, and clients that are making requests to the service hosts. Under normal circumstances, this abstraction performs fine. However, as the number of users increases, the database queries also increase, resulting in slow performance. In such cases, a cache is added to the system to deal with performance deterioration.

A cache is a temporary data storage that can serve data faster by keeping data entries in memory. Caches store only the most frequently accessed data. When a request reaches the serving host, it retrieves data from the cache (cache hitWhen the requested data is found in the cache, the server responds with the data immediately.) and serves the user. However, if the data is unavailable in the cache (cache missWhen the requested data isn’t found in the cache, it’s called a cache miss.), the data will be queried from the database.

A cache is a nonpersistent storage area used to keep repeatedly read and written data, which provides the end user with lower latency. Therefore, a cache must serve data from a storage component that is fast, has enough storage, and is affordable in terms of dollar cost as we scale the caching service. The following illustration highlights the suitability of RAM as the raw building block for caching:

We understand the need for a cache and suitable storage hardware, but what is distributed cache? Let’s discuss this next.

What is a distributed cache?

A distributed cache is a caching system where multiple cache servers coordinate to store frequently accessed data. Distributed caches are needed in environments where a single cache server isn’t enough to store all the data. At the same time, it’s scalable and guarantees a higher degree of availability.

Caches are generally small, frequently accessed short-term storage with fast read time. Caches use the locality of referenceLocality of reference is the notion that a program will access a particular set of data over a short period of time. Locality can be spatial or temporal. principle.

Generally, distributed caches are beneficial in the following ways:

  • They minimize user-perceived latency by precalculating results and storing frequently accessed data.
  • They pre-generate expensive queries from the database.
  • They store user session data temporarily.
  • They serve data from temporary storage even if the data store is down temporarily.
  • Finally, they reduce network costs by serving data from local resources.

Requirements

Let’s start by going over the requirements of our solution.

Functional requirements

The following are the functional requirements:

  • Insert data: The user of a distributed cache system must be able to insert an entry to the cache.
  • Retrieve data: The user should be able to retrieve data corresponding to a specific key.

Non-functional requirements

We’ll consider the following non-functional requirements:

  • High performance: The primary reason for the cache is to enable fast retrieval of data. Therefore, both the insert and retrieve operations must be fast. ...