List-based Dataflow Programming

Dataflow programming paradigm

Dataflow programming is a programming paradigm in which we model programs as directed graphs consisting of nodes and directed edges connecting the nodes. A node represents an operation that accepts inputs and produces output. A directed edge from node A to node B sends A’s output as B’s input. To illustrate, let’s use an example of arithmetic calculation from Bert Sutherland’s Ph.D. thesis titled “The on-line graphical specification of computer procedures,” which pioneered dataflow programming.

Compared to the textual representation, the graphical one follows the dataflow programming style by emphasizing how the data flows from one operation to the other. This has several advantages. First, the dataflow-oriented program describes how the program is made up of smaller building blocks and, therefore, relatively easy to understand. Second, the dataflow version enables parallel execution without any extra effort from the programmer. The executor of the dataflow program can automatically determine which operations can be run in parallel based on the data dependencies between them. In our example, W and Y can be calculated simultaneously because they don’t depend on each other.

This paradigm generally encourages the creation of a library of general-purpose components. This library can be used to construct programs, usually in the form of visual programming. When we mix and match existing components to build programs, we save tons of development time and effort compared to writing and implementing them from scratch. Moreover, the building blocks can be combined to solve problems not even envisioned by their creators.

Dataflow programming is commonly employed by embedded software engineers who develop control systems, such as cruise control for cars. A functional programming language allows us to program in the dataflow-oriented style and reap all its benefits.

Functions as dataflow components

In functional programming languages, functions are pure. This means that they always return the same output for the same input, and their application has no side effects. Due to these properties, functions in the functional paradigm behave similarly to dataflow components—they also accept input and produce output without causing any side effects.

Note: If a set of functions agree on a common input/output data format, we can compose them in the dataflow style.

Let’s consider three functions—map, filter, and fold. We can view those functions as dataflow components. In particular, map acts as a dataflow component that maps each element of the input signal to a new value in the output signal. It is a ...