Caching at Different Layers

Introduction

When designing the API, we tend to focus on optimizing it with respect to performance. Let’s assume that multiple users want to access a Twitter trend and want to see the Tweets related to it. In response to these multiple network calls, the server retrieves the Tweets related to the trend from different databases and delivers them to each client. Next time, the client will experience the same delay for each request, including those made multiple times. The delay occurs due to HTTP's stateless nature, network calls, server computations, and so on.

This is illustrated in the slides below:

Therefore, we require a mechanism that efficiently handles the issues mentioned above and reduces the client latency. For this purpose, we can use a cache.

What is a cache?

A cache is a temporary memory that frequently stores reusable API responses. For instance, the cache stores the trends object when it’s retrieved for the first time by the client. If there isn’t any new trend and the same client refreshes or revisits the page, it’s returned from the cache instead of the server. Therefore, the cache significantly reduces the delay and recomputation on the server's side. The flow of the HTTP request-response is given below using the cache:

1. The client sends the HTTP request to the server, which is served by the intermediate cache.

2. If the cache does not have the requested object, the request is forwarded to the back-end server.

3. The server sends the response to the browser and saves the copy of the requested object in the cache for subsequent requests.

Since the cache is essentially a copy of the data in the origin of truth (database), an outdated/stale version of data in the cache is a known issue. In order to achieve consistency between the cache and the server, we need to update the cache at regular intervals. To evict stale entries and replace outdated entries with fresh ones, various eviction strategies, such as LRUThis stands for the Least Recently Used. The least recently accessed entry is evicted from the cache., LFUThis stands for Least ...