The Importance of Parallelism

From a programmer’s perspective, it would be very convenient if today’s computer hardware was a 100 GHz single-core CPU rather than a 3 GHz multi-core CPU; we wouldn’t need to care about parallelism. Unfortunately, making single-core CPUs faster and faster has hit a physical limit. So, as the evolution of computer hardware is going in the direction of multi-core CPUs and programmable GPUs, programmers have to use efficient parallel patterns to make the most of the hardware.

Parallel algorithms allow us to optimize our programs by executing multiple individual tasks or subtasks at the exact same time on a multi-core CPU or GPU.