Fan-Out

Fan-Out pattern overview

Fan-out is a term that describes the process of diverging a single task to multiple processes or a goroutine. Fan-out is a method of demultiplexing in golang. Fan-out divides the data into numerous smaller chunks and distributes the work among a group of workers to parallelize CPU and I/O consumption.

Suppose we work in a factory where the only job of one of the workers is to package the items they receive from the conveyor belt. Now let’s suppose that the speed at which they receive the item is much more than the speed at which they package them. Since they are the only person with this job, the items will eventually pile up and this will increase the execution time.

Now let’s assume a different scenario. Instead of one person, we have ten people doing the packaging of the items received. This time the items won’t pile up.

The above is an example of a real-life scenario where demultiplexing works. Similarly, many times in our program, we distribute the work among the workers (like in the worker pool pattern). After the work is distributed, each goroutine either can act as a separate entity and perform the fan-out pattern again or directly pass the output.