Introduction

Streams are one of the most important components and patterns of Node.js. There’s a motto in the community that goes, “Stream all the things!” This alone should be enough to describe the role of streams in Node.js. Dominic Tarr, a top contributor to the Node.js community, defines streams as “Node’s best and most misunderstood idea.” There are different reasons that make Node.js streams so attractive; again, it’s not just related to technical properties, such as performance or efficiency, but it’s more about their elegance and the way they fit perfectly into the Node.js philosophy.

This chapter aims to provide a complete understanding of Node.js streams. The first half of this chapter serves as an introduction to the main ideas, the terminology, and the libraries behind Node.js streams. In the second half, we’ll cover more advanced topics and, most importantly, we’ll explore useful streaming patterns that can make our code more elegant and effective in many circumstances.

In this chapter, we’ll learn about the following topics:

• Why streams are so important in Node.js

• Understanding, using, and creating streams

• Streams as a programming paradigm: leveraging their power in many different contexts and not just for I/O

• Streaming patterns and connecting streams together in different configurations

Let’s discover why streams are one of the cornerstones of Node.js.

Discovering the importance of streams

In an event-based platform such as Node.js, the most efficient way to handle I/O is in real time, consuming the input as soon as it’s available and sending the output as soon as the application produces it.

Here’s an initial introduction to Node.js streams and their strengths.

Buffering vs. streaming

Almost all the asynchronous APIs that we’ve seen so far in this book work using buffer mode. For an input operation, buffer mode causes all the data coming from a resource to be collected into a buffer until the operation is completed; it’s then passed back to the caller as one single blob of data. The following illustration shows a visual example of this paradigm: