Design of a Rate Limiter

High-level design

A rate limiter can be deployed as a separate service that will interact with a web server, as shown in the figure below. When a request is received, the rate limiter suggests whether the request should be forwarded to the server or not. The rate limiter consists of rules that should be followed by each incoming request. These rules define the throttling limit for each operation. Let’s go through a rate limiter rule from Lyft, which has open-sourced its rate limiting component.

Press + to interact
domain: messaging
descriptors:
   -key: message_type
    value: marketing
    rate_limit:
               unit: day
               request_per_unit: 5

In the above rate-limiting rule, the unit is set to day and the request_per_unit is set to 5. These parameters define that the system can allow five marketing messages per day.

1 / 6
A request with ID 101, received by one of the web servers

Detailed design

The high-level design given above does not answer the following questions:

  • Where are the rules stored?
  • How do we handle requests that are rate limited?

In this section, we’ll first expand the high-level architecture into several other essential components. We’ll also explain each component in detail, as shown in the following figure.

The rate limiter accepts or rejects requests based on throttle rules

Let’s discuss each component that is present in the detailed design of a rate limiter.

  • Rule database: This is the database, consisting of rules defined by the service owner. Each rule specifies the number of requests allowed for a particular client per unit of time.

  • Rules retriever: This is ...

Level up your interview prep. Join Educative to access 70+ hands-on prep courses.