Throttling Pattern

Intent

This pattern controls the throughput of a processing flow to meet service-level agreements and prevents the overloading of other systems. The Throttling pattern is also known as the Back-Pressure Mechanism (in reactive systems).

Context and problem

Typically, the nonfunctional requirements of the service are concerned with scalability and performance. An application needs to satisfy certain throughput and latency requirements and be able to scale further when the load increases. Less often, an application will need to handle an increasing load not by increasing capacity but by throttling it. For example, an application may have an instant load burst at certain hours of the day, certain days of the week, or perhaps at the end of the month or the holiday season. Some downstream systems may have a fixed capacity that cannot handle such a load burst as activity. Or it may be that our application is dependent on external services, and there are agreed-upon SLAs about a maximum rate of calls. In these and similar circumstances, we can use the Load Leveling pattern or scale up our service to handle the increased incoming message load, but this will not help us control the outgoing message rate. To limit the message processing rate in a service, we need to apply the Throttling pattern.