The ultimate guide to Ruby concurrency interviews. Developed by FAANG engineers, this course covers Monitors, Deferred Callbacks and problem-solving strategies for Reader-Writer and Dining Philosopher Problems.

Concurrency in Ruby is one of the most complex and advanced topics brought up during senior engineering interviews. Knowledge of concurrency and multithreading can put interviewees at a considerable advantage. This course lays the foundations of advanced concurrency and multithreading and explains concepts such as Monitors and Deferred Callbacks in depth. It also builds simple and complete solutions to popular concurrency problems that can be asked about in interviews like the Reader-Writer Problem and the Dining Philosopher Problem. While prior knowledge of concurrency is not required to follow through with this course, familiarity with the basics of Ruby would be helpful.

Ruby Concurrency for Senior Engineering Interviews

<div class="ans">
# WebServers
A common application of the asynchronous paradigm is in web servers, which spend most of their time waiting for network I/O. A more recent approach to implementing web servers uses event loops. An event loop is a programming construct that waits for events to happen and then dispatches them to an event handler. Languages such as JavaScript, Ruby (MRI implementation) and Python (standard C implementation) enable the asynchronous programming model using an event loop. The idea is that a single thread runs in a loop waiting for an event to occur. Once an event arrives, it is appropriately dispatched to an event handler. The event loop's thread immediately goes back to listening for another event. The event handler may run on another thread if the language supports multiple threads. This design achieves very high concurrency if an application is frequently involved in either of the following:

<ul>
<li>Network I/O</li>
<li>Disk I/O</li>
</ul> 

If an application spends most of its time waiting for I/O, it can benefit from an asynchronous design. One of the most common use cases you'll find in the wild is of web servers implemented using asynchronous design. A web server waits for an HTTP request to arrive and returns the matching resource. Folks familiar with JavaScript would recall NodeJS works on the same principle. It is a web server that runs an event loop to receive web requests in a single thread. Contrast that to web servers, which create a new thread, or worse, fork a new process, to handle each web request. In some benchmarks, the asynchronous event loop based web servers outperformed multithreaded ones, which may seem counterintuitive.

To truly appreciate asynchronous programming model, we'll present <a href="https://en.wikipedia.org/wiki/Ryan_Dahl" target="_blank">Ryan Dahl's</a> motivation for creating NodeJS which also runs an event loop. Ryan classifies disk and network I/O operations as blocking operations and <a href="https://www.youtube.com/watch?v=ztspvPYybIY" target="_blank">presents</a> the following table to put the latency for various operations in perspective.

<table style="color:#820C02; border:1px solid #DDD">
<tr style="color:#820C02;"><th style="background-color:#ffe5e5;">Device</th><th style="background-color:#ffe5e5;">CPU Cycles</th><th style="background-color:#ffe5e5;">Humanified</th></tr>
<tr><td>L1 cache</td><td>3</td><td>1 seconds</td></tr>
<tr><td>L2 cache</td><td>14</td><td>4.6 seconds</td></tr>
<tr><td>RAM</td><td>250</td><td>83 seconds</td></tr>
<tr><td>Disk</td><td>41000000</td><td>158 days</td></tr>
<tr><td>Network</td><td>240000000</td><td>2.5 years</td></tr>
</table>

The third column humanizes the CPU cycles by assuming if 3 CPU cycles equaled one second then a network I/O would feel equivalent of 2.5 years. This may dawn upon you how slow can disk and network I/O be in comparison to access times for other devices. The long waiting times suggest that if these blocking calls are synchronous then we end up wasting a lot of CPU cycles. The CPU can be better utilized especially in environments with a lot of I/O.

As a thought exercise, if we implement a chat server using the asynchronous model, a single thread can serve several hundred clients since the thread spends most of its time waiting for clients to send messages. In such a scenario, if we created one thread per client, we would quickly use up system resources since threads aren't free. Creating, maintaining and tearing down threads takes CPU cycles in addition to memory. In fact, this difference becomes more visible in web servers that use threads to handle HTTP web requests vs. which use an event loop. Apache is an example of the former and NGINX of the latter. NGINX outshines Apache in memory usage under high load.

If your application is CPU bound then shifting to an asynchronous model may not display significant performance gains. In fact, you'll see none for a hundred percent compute bound program. In such a case a hardware upgrade is more likely to improve performance.


# Asynchronous Programming in Ruby
Various Ruby frameworks/gems exist that enable asynchronous programming. We'll not be discussing anyone in particular but list them below for reference:

<ul>
<li><a href="https://github.com/socketry/async" target="_blank">async gem</a></li>

<li><a href="https://github.com/igrigorik/em-synchrony" target="_blank">EM Synchrony</a></li>

<li><a href="https://github.com/eventmachine/eventmachine" target="_blank">Event Machine</a></li>

<li><a href="http://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/Async.html" target="_blank">Concurrent Rub's Async Mixin</a></li>

</ul> 

</div>

<div>
# WebServers
A common application of the asynchronous paradigm is in web servers, which spend most of their time waiting for network I/O. A more recent approach to implementing web servers uses event loops. An event loop is a programming construct that waits for events to happen and then dispatches them to an event handler. Languages such as JavaScript, Ruby (MRI implementation) and Python (standard C implementation) enable the asynchronous programming model using an event loop. The idea is that a single thread runs in a loop waiting for an event to occur. Once an event arrives, it is appropriately dispatched to an event handler. The event loop's thread immediately goes back to listening for another event. The event handler may run on another thread if the language supports multiple threads. This design achieves very high concurrency if an application is frequently involved in either of the following:

<ul>
<li>Network I/O</li>
<li>Disk I/O</li>
</ul> 

If an application spends most of its time waiting for I/O, it can benefit from an asynchronous design. One of the most common use cases you'll find in the wild is of web servers implemented using asynchronous design. A web server waits for an HTTP request to arrive and returns the matching resource. Folks familiar with JavaScript would recall NodeJS works on the same principle. It is a web server that runs an event loop to receive web requests in a single thread. Contrast that to web servers, which create a new thread, or worse, fork a new process, to handle each web request. In some benchmarks, the asynchronous event loop based web servers outperformed multithreaded ones, which may seem counterintuitive.

To truly appreciate asynchronous programming model, we'll present <a href="https://en.wikipedia.org/wiki/Ryan_Dahl" target="_blank">Ryan Dahl's</a> motivation for creating NodeJS which also runs an event loop. Ryan classifies disk and network I/O operations as blocking operations and <a href="https://www.youtube.com/watch?v=ztspvPYybIY" target="_blank">presents</a> the following table to put the latency for various operations in perspective.

<table>
<tr><th>Device</th><th>CPU Cycles</th><th>Humanified</th></tr>
<tr><td>L1 cache</td><td>3</td><td>1 seconds</td></tr>
<tr><td>L2 cache</td><td>14</td><td>4.6 seconds</td></tr>
<tr><td>RAM</td><td>250</td><td>83 seconds</td></tr>
<tr><td>Disk</td><td>41000000</td><td>158 days</td></tr>
<tr><td>Network</td><td>240000000</td><td>2.5 years</td></tr>
</table>

The third column humanizes the CPU cycles by assuming if 3 CPU cycles equaled one second then a network I/O would feel equivalent of 2.5 years. This may dawn upon you how slow can disk and network I/O be in comparison to access times for other devices. The long waiting times suggest that if these blocking calls are synchronous then we end up wasting a lot of CPU cycles. The CPU can be better utilized especially in environments with a lot of I/O.

As a thought exercise, if we implement a chat server using the asynchronous model, a single thread can serve several hundred clients since the thread spends most of its time waiting for clients to send messages. In such a scenario, if we created one thread per client, we would quickly use up system resources since threads aren't free. Creating, maintaining and tearing down threads takes CPU cycles in addition to memory. In fact, this difference becomes more visible in web servers that use threads to handle HTTP web requests vs. which use an event loop. Apache is an example of the former and NGINX of the latter. NGINX outshines Apache in memory usage under high load.

If your application is CPU bound then shifting to an asynchronous model may not display significant performance gains. In fact, you'll see none for a hundred percent compute bound program. In such a case a hardware upgrade is more likely to improve performance.


# Asynchronous Programming in Ruby
Various Ruby frameworks/gems exist that enable asynchronous programming. We'll not be discussing anyone in particular but list them below for reference:

<ul>
<li><a href="https://github.com/socketry/async" target="_blank">async gem</a></li>

<li><a href="https://github.com/igrigorik/em-synchrony" target="_blank">EM Synchrony</a></li>

<li><a href="https://github.com/eventmachine/eventmachine" target="_blank">Event Machine</a></li>

<li><a href="http://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/Async.html" target="_blank">Concurrent Rub's Async Mixin</a></li>

</ul> 

</div>

This lesson continues the discussion on asynchronous programming.

The Basics

API Overview

Brief Intro to Asynchronous Programming

Interview Practice Problems

Bonus Questions

Epilogue

... continued

WebServers