Requirements of a Rate Limiter’s Design

Understand the requirements and important concepts of a rate limiter.

We'll cover the following

Requirements
- Functional requirements
- Non-functional requirements
Types of throttling
Where to place the rate limiter
Two models for implementing a rate limiter
Building blocks we will use

Requirements

Our focus in this lesson is to design a rate limiter with the following functional and non-functional requirements.

Functional requirements

To limit the number of requests a client can send to an API within a time window.
To make the limit of requests per window configurable.
To make sure that the client gets a message (error or notification) whenever the defined threshold is crossed within a single server or combination of servers.

Non-functional requirements

Availability: Essentially, the rate limiter protects our system. Therefore, it should be highly available.
Low latency: Because all API requests pass through the rate limiter, it should work with a minimum latency without affecting the user experience.
Scalability: Our design should be highly scalable. It should be able to rate limit an increasing number of clients’ requests over time.