Stream-based Dataflow Programming

Understand how streams allow us to formulate dataflow programs that work with signals of infinite elements.

Limitations of list-based dataflow programming

The functional paradigm allows us to formulate programs in the dataflow style by composing function lists, such as map, filter, and fold. Unfortunately, list-based dataflow programming in OCaml has a severe limitation—lists are finite.

To illustrate, suppose we implement a function called first_prime_greater_equal. This function returns the first prime number greater than or equal to a given n. We can attempt to formulate a dataflow program to solve this problem with the following steps:

  • Enumerate integers greater than or equal n
  • Filter the list, keeping only prime numbers
  • Take the head of the resulting list

In other words, we attempt to formulate the following dataflow diagram:

Get hands-on with 1400+ tech skills courses.