The ultimate guide to senior engineering interviews. Strategies developed by FAANG engineers on Python concurrency, multithreading, monitors, and event loops. Prep faster for top company questions.

Concurrency in Python is one of the most complex and advanced topics brought up during senior engineering interviews. Knowledge of concurrency and multithreading can put interviewees at a considerable advantage. This course lays the foundations of advanced concurrency and multithreading and explains concepts such as Monitors, Event Loops and Deferred Callbacks in depth. It also builds simple and complete solutions to popular concurrency problems that can be asked about in interviews like the Reader-Writer Problem and the Dining Philosopher Problem. While prior knowledge of concurrency is not required to follow through with this course, familiarity with the very basic Python would be helpful.

Python Concurrency for Senior Engineering Interviews

<div class="card">
 # Pool Executors 
 <div class="ans">
In the previous sections, we studied how to create and manage threads and processes. However, managing these entities can be taxing on the developer so Python alleviates this burden by providing an interface which abstracts away the subtleties of starting and tearing down threads or processes. The <code class="abtc">concurrent.futures</code> package provides the <code class="clazz">Executor</code> interface which can be used to submit tasks to either threads or processes. The two subclasses are:

<ul>
<li>ThreadPoolExecutor</li>
<li>ProcessPoolExecutor</li>
</ul>

Tasks can be submitted synchronously or asynchronously to the pools for execution.

## ThreadPoolExecutor
The <code class="clazz">ThreadPoolExecutor</code> uses threads for executing submitted tasks. Let's look at a very simple example.
### Using ThreadPoolExecutor


```python
from concurrent.futures import ThreadPoolExecutor
from threading import current_thread


def say_hi(item):

 print("\nhi " + str(item) + " executed in thread id " + current_thread().name, flush=True)


if __name__ == '__main__':
 executor = ThreadPoolExecutor(max_workers=10)
 lst = list()
 for i in range(1, 10):
 lst.append(executor.submit(say_hi, "guest" + str(i)))

 for future in lst:
 future.result()

 executor.shutdown()

```



We create a thread pool with a maximum of ten threads. Next, we run in a loop and submit tasks to be executed. The first argument to the <code class="mthd">submit()</code> is a callable which gets invoked with the arguments that follow. If you examine the output you'll see that tasks
are executed by threads with different names. The submit calls return what we call a future. The <code class="clazz">Future</code> class represents the
execution of the callable. Note that the invocation <code class="mthd">future.result()</code> is blocking. Interestingly, if we change the code within the first
for loop as follows, the execution becomes serial.


```python
 for i in range(1, 10):
 future = executor.submit(say_hi, "guest" + str(i))
 future.result()
```


Generally this is not the pattern we use for submitting tasks. The idea is to be able to process multiple tasks in parallel and query the progress of the tasks using the future object.
For instance, say your program is trying to download multiple mp3 files from the internet, then the download tasks can be carried out in parallel independent of each other. Or you are writing a script that copies several files from a source directory to a destination directory then again the task is amenable for parallel processing.

At the end of the program we invoke <code class="mthd">shutdown()</code> method on the executor. By default, the executor will wait for the futures to complete and any associated
resources to be freed. Any task submission after invoking shutdown will result in an exception. If the call is invoked with wait=False then the call becomes nonblocking. However,
the entire Python program will not shut down until all the futures are done.

</div>
</div>

<div>
 # Pool Executors 
 <div>
In the previous sections, we studied how to create and manage threads and processes. However, managing these entities can be taxing on the developer so Python alleviates this burden by providing an interface which abstracts away the subtleties of starting and tearing down threads or processes. The <code>concurrent.futures</code> package provides the <code>Executor</code> interface which can be used to submit tasks to either threads or processes. The two subclasses are:

<ul>
<li>ThreadPoolExecutor</li>
<li>ProcessPoolExecutor</li>
</ul>

Tasks can be submitted synchronously or asynchronously to the pools for execution.

## ThreadPoolExecutor
The <code>ThreadPoolExecutor</code> uses threads for executing submitted tasks. Let's look at a very simple example.
### Using ThreadPoolExecutor


```python
from concurrent.futures import ThreadPoolExecutor
from threading import current_thread


def say_hi(item):

 print("\nhi " + str(item) + " executed in thread id " + current_thread().name, flush=True)


if __name__ == '__main__':
 executor = ThreadPoolExecutor(max_workers=10)
 lst = list()
 for i in range(1, 10):
 lst.append(executor.submit(say_hi, "guest" + str(i)))

 for future in lst:
 future.result()

 executor.shutdown()

```



We create a thread pool with a maximum of ten threads. Next, we run in a loop and submit tasks to be executed. The first argument to the <code>submit()</code> is a callable which gets invoked with the arguments that follow. If you examine the output you'll see that tasks
are executed by threads with different names. The submit calls return what we call a future. The <code>Future</code> class represents the
execution of the callable. Note that the invocation <code>future.result()</code> is blocking. Interestingly, if we change the code within the first
for loop as follows, the execution becomes serial.


```python
 for i in range(1, 10):
 future = executor.submit(say_hi, "guest" + str(i))
 future.result()
```


Generally this is not the pattern we use for submitting tasks. The idea is to be able to process multiple tasks in parallel and query the progress of the tasks using the future object.
For instance, say your program is trying to download multiple mp3 files from the internet, then the download tasks can be carried out in parallel independent of each other. Or you are writing a script that copies several files from a source directory to a destination directory then again the task is amenable for parallel processing.

At the end of the program we invoke <code>shutdown()</code> method on the executor. By default, the executor will wait for the futures to complete and any associated
resources to be freed. Any task submission after invoking shutdown will result in an exception. If the call is invoked with wait=False then the call becomes nonblocking. However,
the entire Python program will not shut down until all the futures are done.

</div>
</div>

This lesson discusses the various APIs for thread and pool executors.

The Basics

Threading Module

Multiprocessing

Concurrent Package

Async.io

Global Interpreter Lock

Interview Practise Problems

The End

Pool Executors

Pool Executors

ThreadPoolExecutor

Using ThreadPoolExecutor