Unordered Limited Parallel Execution

If we try to run the check-urls.js application against a file that contains thousands or millions of URLs, we’ll surely run into issues. Our application will create an uncontrolled number of connections all at once, sending a considerable amount of data in parallel, and potentially undermining the stability of the application and the availability of the entire system. As we already know, the solution to keep the load and resource usage under control is to limit the concurrency of the parallel tasks.

Limited execution

Let’s see how this works with streams by creating a limited-parallel-stream.js module, which is an adaptation of the parallel-stream.js module.

Let’s see what it looks like, starting from its constructor (the changed parts are highlighted):