The ultimate guide to senior engineering interviews. Strategies developed by FAANG engineers on Python concurrency, multithreading, monitors, and event loops. Prep faster for top company questions.

Concurrency in Python is one of the most complex and advanced topics brought up during senior engineering interviews. Knowledge of concurrency and multithreading can put interviewees at a considerable advantage. This course lays the foundations of advanced concurrency and multithreading and explains concepts such as Monitors, Event Loops and Deferred Callbacks in depth. It also builds simple and complete solutions to popular concurrency problems that can be asked about in interviews like the Reader-Writer Problem and the Dining Philosopher Problem. While prior knowledge of concurrency is not required to follow through with this course, familiarity with the very basic Python would be helpful.

Python Concurrency for Senior Engineering Interviews

<div class="card">
# Web Crawler Example
<div class="ans">
<p>From what we have learnt so far, asyncio is an excellent choice for blocking operations. Usually, there are two kinds of blocking operations:</p> 

<ul>
<li><p>network I/O</p></li>
<li><p>disk I/O</p></li>
</ul>    

<p>Let's start with implementing a simple web crawler. <span class="hi-yl">A web crawler is a program that systematically browses the world wide web, typically with the intent to index it.</span> For our purposes, we'll dumb down our crawler and limit its capability to fetch the HTML for a list of URLs. The downloaded HTML is passed onto a consumer which then performs the indexing but we'll not implement that part.</p>


<p>The meat of the problem lies in asynchronously downloading the given URLs. We'll be using the <code class="abtc">aiohttp</code> module for asynchronous REST GET calls. If we were to serially download the URLs, we'll unnecessarily be wasting CPU cycles as a network request is usually the slowest form of I/O. Let's start with the serial version of the code.</p>

## Serial Download
<p>In our example, we'll pass three URLs to be downloaded. The serial version of the code uses the following snippet to make the network requests:</p>
<b>

```python
    # url is a string such as www.cnn.com    
    html = urlopen(url)
    text = html.read()
```
</b>

<p>The complete code appears in the code widget below. Run the code and observe the total time taken to download the URLs in the output. </p>
</div>
</div>

<div>
# Web Crawler Example
<div>
<p>From what we have learnt so far, asyncio is an excellent choice for blocking operations. Usually, there are two kinds of blocking operations:</p> 

<ul>
<li><p>network I/O</p></li>
<li><p>disk I/O</p></li>
</ul>    

<p>Let's start with implementing a simple web crawler. <span>A web crawler is a program that systematically browses the world wide web, typically with the intent to index it.</span> For our purposes, we'll dumb down our crawler and limit its capability to fetch the HTML for a list of URLs. The downloaded HTML is passed onto a consumer which then performs the indexing but we'll not implement that part.</p>


<p>The meat of the problem lies in asynchronously downloading the given URLs. We'll be using the <code>aiohttp</code> module for asynchronous REST GET calls. If we were to serially download the URLs, we'll unnecessarily be wasting CPU cycles as a network request is usually the slowest form of I/O. Let's start with the serial version of the code.</p>

## Serial Download
<p>In our example, we'll pass three URLs to be downloaded. The serial version of the code uses the following snippet to make the network requests:</p>
<b>

```python
    # url is a string such as www.cnn.com    
    html = urlopen(url)
    text = html.read()
```
</b>

<p>The complete code appears in the code widget below. Run the code and observe the total time taken to download the URLs in the output. </p>
</div>
</div>

This lesson discusses a text-book example of a concurrent program - the web crawler.

The Basics

Threading Module

Multiprocessing

Concurrent Package

Async.io

Global Interpreter Lock

Interview Practise Problems

The End

Web Crawler Example

Web Crawler Example