Daemon Processes

Background

Now that we have become aware of the difference between multithreading and multiprocessing in Python, it becomes more clear that using multiple processes to schedule different jobs is efficient.

A widespread use case is to run long-running, background processes (often called daemons) that are responsible for scheduling some tasks regularly or processing jobs from a queue.

It could be possible to leverage concurrent.futures and a ProcessPoolExecutor to do that, as discussed in Using Futures. However, the pool does not provide any control regarding how it dispatches jobs. The same goes for using the multiprocessing module. They both make it hard to efficiently control the running of background tasks. Think of it as the “pets vs. cattle” analogy for processes.

cotyledon

This section introduces you to cotyledon, a Python library designed to build long-running processes.

To run the following code snippet, press Run and enter the command python cotyledon-simple.py. ...