Asynchronous Control Flow Patterns: Ordered Parallel Execution

The parallel streams that we created previously may shuffle the order of the emitted data, but there are situations where this is not acceptable. Sometimes, in fact, it’s necessary to have each chunk emitted in the same order in which it was received. However, we can still run the transform function in parallel; all we have to do is sort the data emitted by each task so that it follows the same order in which the data was received.

This technique involves the use of a buffer to reorder the chunks while they’re emitted by each running task. For brevity, we’re not going to provide an implementation of such a stream. What we’re going to do instead is reuse one of the available packages on npm built for this specific purpose, that is, the parallel-transform package.

Implementing ordered parallel stream

We can quickly check the behavior of an ordered parallel execution by modifying our existing check-urls module. Let’s say that we want our results to be written in the same order as the URLs in the input file while executing our checks in parallel. We can do this using the parallel-transform package.