Fan Out and Fan In Design Patterns

Fan Out

The goal of the Fan Out design pattern is to distribute work between multiple concurrent processors, also known as workers. To understand it better, let’s recall the Pipeline design pattern but consider the following problem:

What if the amount of work at the different steps in our pipeline is very different?

For example, it takes a lot more time to fetch the HTML content than to parse it. In such a case, we may want to distribute that heavy work between multiple coroutines. A single coroutine can read from each channel, but multiple coroutines can consume from a ...