Let’s say we need to read 3 files and then count the occurrence of the words starting with “t”. In a single-threaded program, we can go with either of the following approaches:
With both these approaches, the time taken to execute the task will be the same. But what happens if there are too many files or the files are too large and an error occurs while reading the files? Our program breaks down, and we go back, fix the error, and then we re-run the program. This is tedious and unproductive, so what if there was a way to read all files in parallel? This would enable our program to count words as soon as a file is done being read, while other files are also being read in the background.
The event loop in JS allows us to solve the problem discussed above. Since JS is a single-threaded language, the tasks are executed sequentially. However, the event loop allows the program to delegate a task (such as file I/O) to another process (the OS in this case). Once the delegated task is completed, the result is queued into the program. The program executes all tasks that are on its thread and then looks at a pending task in the queue. If a task is found, the program performs the computation associated with that task and repeats this process. The following diagram clarifies the event loop:
As seen in the above diagram, once “All Tasks on Thread (are) Executed”, then the JS program checks the queue for other pending tasks. If no pending task is found, the JS program keeps on checking the queue and the program is in a suspended state until a new task is found on the queue.
The event loop construct is exposed to the programmer via the following 3 constructs:
A callback in JS is a function that is passed as an argument to another function. Consider the following code snippet:
function sum(a, b) {return a + b;}function mult(a, b) {return a * b;}function callAndPrint(a, b, func) {console.log(func(a, b));}callAndPrint(2, 3, sum);callAndPrint(2, 3, mult);
In the example above, the callAndPrint
function takes a function argument func
. This allows the program to print the sum and product of 2 numbers, a
and b
, using the same callAndPrint
function.
Now, if we want to read 3 files and then count the occurrence of the words starting with “t” using callbacks, we’ll do it as shown below:
// const fs = require('fs');function countTs(files, doneFunc) {let filesLeft = files.length;let errorFiles = 0;let ts = 0;for (let i = 0; i < files.length; i++) {fs.readFile(files[i],(readError, fileContents) => {if (readError) {errorFiles++;} else {let words = fileContents.toString().split(' ');for (let w = 0; w < words.length; w++) {if (words[w][0] == 't') {ts++;}}}filesLeft--;if (filesLeft == 0) {doneFunc(ts, errorFiles);}});}}countTs(['a.txt', 'b.txt', 'c.txt'],(ts, errorFiles) => {console.log('Count "t":', ts);console.log('Error reading files:', errorFiles);});
In the example above, the callback functions are highlighted. The first one is a callback function with readErr
and fileContents
as parameters. This callback function is passed to the fs.readFile
function, which executes this function after the given file name (file[i]
in this case) has been read, either successfully or unsuccessfully. The callback function also calls the second callback function doneFunc
with the result of the computation.
If we wanted to write the words starting with “t” to a file, we would have to nest a fs.writeFile
call inside the callback function for fs.readFile
. This would cause our program to go through callback hell or pyramid of doom as shown in the diagram below:
To solve the callback hell problem, promises were introduced in ES2015 (JS standardization).
Promises in JS allow you to chain callback functions in a semantically pleasing way. A promise has 3 states:
A promise is in the pending state upon creation. The passed executor function can either call resolve
or reject
, which changes the state of the promise to fulfilled or rejected respectively. We can rewrite the file reading example with promises as shown below:
// const fs = require('fs');const { promisify } = require('util');let readFile = promisify(fs.readFile);function countTs(files, doneFunc) {let filesLeft = files.length;let errorFiles = 0;let ts = 0;for (let i = 0; i < files.length; i++) {let readPromise = readFile(files[i]);readPromise.then(fileContents => {let words = fileContents.toString().split(' ');for (let w = 0; w < words.length; w++) {if (words[w][0] == 't') {ts++;}}filesLeft--;if (filesLeft == 0) {doneFunc(ts, errorFiles);}}).catch(readError => {errorFiles++;filesLeft--;if (filesLeft == 0) {doneFunc(ts, errorFiles);}})}}countTs(['a.txt', 'b.txt', 'c.txt'],(ts, errorFiles) => {console.log('Count "t":', ts);console.log('Error reading files:', errorFiles);});
In the example above, the function fs.readFile
is converted to a function that does not take a callback function and returns a promise instead. This is done using the util.promisify
function in line 4, and the resulting function is stored in the readFile
variable. The readFile
function is called with only one argument, which is the file name, and the resulting promise is stored in the readPromise
variable. In line 11, the then
function is called on readPromise
, which attaches the function defined in lines 13 to 23 with the promise. This function is executed only if the promise is set to the fulfilled state. Moreover, the catch
function is also called on readPromise
, which attaches the function defined in lines 25 to 29. This function is executed only if the promise is set to the rejected state.
In this example, not only has our nesting decreased, but the error handling code that was part of the callback function is now a part of the catch
handler as well. Therefore, promises allow programmers to linearize callback-based code.
Async functions in JS allow callbacks and promises to be converted into seemingly synchronous code that performs asynchronously. The async functions have a preceding async
keyword before the function definition. The await
keyword is used to denote an asynchronous operation. The count_ts
function can be rewritten as shown below:
async function countTs(files, doneFunc) {let errorFiles = 0;let ts = 0;let readPromises = [];for (let i = 0; i < files.length; i++) {readPromises.push(readFile(files[i]));}await Promise.all(readPromises);for (let i = 0; i < files.length; i++) {try {console.log(readPromises[i].toString());let words = readPromises[i].toString().split(' ');for (let w = 0; w < words.length; w++) {if (words[w][0] == 't') {ts++;}}} catch(readError) {errorFiles++;}}doneFunc(ts, errorFiles);}
In the code above, the code is written in a synchronous way and there is no chaining, as was the case with callbacks and promises. The await Promise.all
line blocks the code execution until all files are read. Afterwards, the code that was previously inside the callback is now part of the count_ts
function itself in lines 12 to 22.
Async functions not only allow the programmer to escape from callback hell and promise chaining in asynchronous code, but they also make the code seemingly synchronous.
Free Resources