Design of a Rate Limiter

High-level design

A rate limiter can be deployed as a separate service that will interact with a web server, as shown in the following figure. When a request is received, the rate limiter suggests whether the request should be forwarded to the server or not. The rate limiter consists of rules which should be followed by each incoming request. These rules define the throttling limit for each operation. Let’s go through a rate-limiter rule from Lyft, which has open-sourced its rate-limiting component.

In the above rate-limiting rule, the unit is set to day and the request_per_unit is set to 5. These parameters define that the system can allow 5 marketing messages per day.

Detailed design

The high-level design given above does not answer the following questions:

• Where are the rules stored?
• How to handle requests that are rate-limited?

In this section, we will first expand the high-level architecture into several other essential components. Further, we will explain each component in detail, as shown in the following figure.

Let’s discuss each component that is present in the detailed design of a rate limiter.

Rule database: This is the storage consisting of rules defined by the service owner. Each rule specifies the number of requests allowed for a particular client per unit time.

Rules retriever: Rules retriever is a background process that periodically checks for any modifications to the rules in the database. Rule cache is updated if there are any modifications made to the existing rules.

Throttle rules cache ...