What is the point of concurrency?

Software systems are essential tools that help perform simple and complex tasks. Creating effective and reliable software systems requires carefully considering how the system will function under different circumstances. In this answer, we will discuss the importance of concurrency in software systems and its benefits.

Historically, hardware advancements have been the primary ways to increase the efficiency and effectiveness of software; however, the rate of hardware innovation has slowed down in recent years. As a result, software developers have turned to software solutions to increase system performance, with concurrency being one of the most effective approaches.

What is Concurrency?

Concurrency is the ability of a system to handle multiple tasks or processes simultaneously by dividing them into smaller units of work and executing them at the same time. Concurrency arose due to a combination of hardware and software advancements. In the early days of computing, computers were simple and had only one processor, which could execute one instruction at a time. As processors became faster and more powerful, it became possible to perform multiple tasks simultaneously.

Operating systems allow multiple applications to run concurrently by dividing the available resources among them and reducing idle time. In application programming, concurrency has become more widespread and popular with the advent of multi-core processors. Applications can provide a responsive user interface while performing computationally intensive tasks  in the background.

We can utilize concurrent execution when there is a delay between one task's action and another task's response to that action. We can reduce or mask this delay with the execution of another task during the delay period. Concurrency enables a task to take full advantage of the available processing resources, improving software performance.

Analogy

When developing an online shopping website, we may need to fetch the product details, price, and availability from multiple databases. Without concurrency, our website would have to retrieve information from each database one at a time, which could take a long time and reduce the responsiveness of the website. With concurrency, our website can fetch data simultaneously, resulting in faster overall performance and better user response.

Benefits of concurrency

Concurrency is beneficial for software performance in the following ways:

Faster processing

Concurrency helps manage system resources by allowing multiple tasks to run simultaneously, thus reducing the CPU's idle time. Rather than waiting for one task to finish before the next is run, the CPU can switch between tasks; the CPU saves the current state of the task (context) into memory and loads the saved state of the next task to be executed. This can lead to faster execution time and maximized CPU usage.

Improved memory utilization

Concurrency helps manage memory by allowing multiple threads to access the same memory region (address space) simultaneously. This can reduce the amount of memory needed to store redundant copies of the same data. This allows for more fine-tuned control over memory usage, as different tasks (threads) can also be assigned to specific memory regions.

Improved scalability

Scalability refers to a system's ability to maintain or improve performance as its workload or demand increases. Scalability can be achieved by distributing workloads across multiple processing units. In a scalable system, adding more resources should increase its capacity to handle more requests or users without degrading performance. If a system is not designed for concurrency, adding more resources may not improve performance because the system does not take advantage of those extra resources.

Improved fault tolerance

Concurrency can allow a software system to continue functioning even if some components fail. By breaking down a task into smaller parts that can run independently and concurrently, a fault in one component does not necessarily bring down the entire system. If one thread or process fails, other threads or processes can continue to execute, ensuring the system remains available and responsive. This can help ensure that critical systems remain operational even in the face of hardware or software failure.

Here' is an illustration showing a concurrently designed program that incorporates fault-tolerant mechanisms, such as checkpointing, which periodically saves the tasks' current states, and rollback recovery, which loads the saved states in the event of a failure:

Here is a sequential version that incorporates the rollback recovery mechanism in the event of a task failure:

Conclusion

Concurrency is useful because it allows multiple tasks to run simultaneously, leading to faster processing, improved memory utilization, improved scalability, and improved fault tolerance. By taking advantage of concurrency, software developers can create more efficient, effective, and reliable systems that can handle a wide range of tasks and demands.