Celery: Handling Failures

Task executions might fail, and in this case, it is crucial to handle that properly. It is common for tasks to depend on external services, such as a remote database or a REST API. Connection failure might be transient; it is therefore better to deal with defeat and retry later.

Note: The example in this lesson assumes that you have a Redis server. We already run a Redis server as a daemon when Run is clicked on a widget so you do not need to worry about starting the server yourself.

Retry backoff

In the following application, after running python celery-task-retry.py, open another terminal and enter command cd examples/queue-based-distribution/ to move to the directory where celery-task-retry.py is located. Then, enter the command: celery -A celery-task-retry worker. The celery-task-retry.py will keep waiting until the worker is started. Once it does, the result is calculated by the celery-task-retry.py.