Summary

Recall

We’ve looked closely at a variety of topics related to concurrent processing in Python:

• Threads have an advantage of simplicity for many cases. This has to be balanced against the GIL interfering with compute-intensive multi-threading.
• Multiprocessing has an advantage of making full use of all cores of a processor. This has to be balanced against interprocess communication costs. If shared memory is used, there is the complication of encoding and accessing the shared objects.
• The concurrent.futures module defines an abstraction – the future – that can minimize the differences in application programming used for accessing threads or processes. This makes it easy to switch and see which approach is fastest.
• The async/await features of the Python language are supported by the AsyncIO package. Because these are coroutines, there isn’t true parallel processing; control switches among the coroutines allow a single thread to interleave between waiting for I/O and computing.
• The dining philosophers benchmark can be helpful for comparing different kinds of concurrency language features and libraries. It’s a relatively simple problem with some interesting complexities.
• Perhaps the most important observation is the lack of a trivial one-size-fits- all solution to concurrent processing. It’s essential to create—and measure—a variety of solutions to determine a design that makes best use of the computing hardware.

Synopsis

This chapter ends our exploration of object-oriented programming with a topic that isn’t very object-oriented. Concurrency is a difficult problem, and we’ve only scratched the surface. While the underlying OS abstractions of processes and threads do not provide an API that is remotely object-oriented, Python offers some really good object-oriented abstractions around them. The threading and multiprocessing packages both provide an object-oriented interface to the underlying mechanics. Futures are able to encapsulate a lot of the messy details into a single object. AsyncIO uses coroutine objects to make our code read as though it runs synchronously, while hiding ugly and complicated implementation details behind a very simple loop abstraction.