Timeouts Side Effects And Latency problems

Circuit Breaker pattern

Use a generic gateway to provide the template for connection handling, error handling, query execution, and result processing. That way we only need to get it right in one place, and calling code can provide just the essential logic. Collecting this common interaction pattern into a single class also makes it easier to apply the Circuit Breaker pattern. Make full use of the platform. Infrastructure services like Amazon API Gateway can handle many of the dirty details for us. Language runtimes that use callbacks or reactive programming styles also let us specify timeouts more easily.

Timeouts are often found in the company of retries. Under the philosophy of “best effort,” the software attempts to repeat an operation that timed out. Immediately retrying an operation after a failure has a number of consequences, but only some of them are beneficial. If the operation failed because of any significant problem, it’s likely to fail again if we immediately retry it. Some kinds of transient failures might be overcome with a retry (for example, dropped packets over a WAN). Within the walls of a data center, however, the failure is probably because of something wrong with the other end of a connection.

Author’s Note: My experience has been that problems on the network, or with other servers, tend to last for a while. Thus, fast retries are very likely to fail again.