Unbalanced Capacities

Mismatched ratios

Whether our resources take months, weeks, or seconds to provision, we can end up with mismatched ratios between different layers. That makes it possible for one tier or service to flood another with requests beyond its capacity. This especially holds when we deal with calls to rate-limited or throttled APIs.

In the illustration below, the front-end service has 3,000 request-handling threads available. During peak usage, the majority of these will be serving product catalog pages or search results. Some smaller numbers will be in various corporate “telling” pages and a few will be involved in a checkout process.