Background Jobs

At a certain scale, the benefits of background jobs outweigh their complexity, and we’d be wise to use them as much as possible. We likely aren’t at that scale now and might never be; thus, we want to be judicious when we use background jobs.

The two main problems that happen when we do all processing in the request are: overuse of resources and failures due to network timeouts. Therefore, background jobs should only be used when we cannot tolerate these failures at whatever level we are seeing them. This can be hard to judge. A guideline that we adopt is to always communicate with third parties in a background job because, even at a tiny scale, those communications will fail.

For all other code, it’s best to monitor its performance, set a limit on how poor the performance is allowed to get, and use background jobs when performance gets bad (keeping in mind that background jobs aren’t the only solution to poor performance). For example, we might decide that the 90th percentile of controller action response times should always be under 500ms. When we are going to use background jobs, we need to understand how the underlying system actually works to avoid surprises.