Create Back Pressure

Queues in performance

Every performance problem starts with a queue backing up somewhere. Maybe it’s a socket’s listening queue. Maybe it’s the OS’s run queue or the database’s I/O queue. If a queue is unbounded, it can consume all available memory. As the queue grows, the time it takes for a piece of work to get all the way through it grows too. (See Little’s law.$^{3}$) So as a queue’s length reaches toward infinity, response time also heads toward infinity. We really don’t want unbounded queues in our systems.

On the other hand, ...