Microbenchmarking

Optimizing small code blocks

Profiling can help us find the bottlenecks in our code. If these bottlenecks are caused by inefficient data structures, the wrong choice of algorithm, or unnecessary contention, these bigger issues should be addressed first. But sometimes we find a small function or a small block of code that we need to optimize, and in those cases, we can use a method called microbenchmarking. With this process, we create a microbenchmark—a program that runs a small piece of code in isolation from the rest of the program. The process of microbenchmarking consists of the following steps:

1. Find a hot spot that needs tuning, preferably using a profiler.

2. Separate it from the rest of the code and create an isolated microbenchmark.

3. Optimize the microbenchmark. Use a benchmarking framework to test and evaluate the code during optimization.

4. Integrate the newly optimized code into the program and measure again to see if the optimizations are relevant when the code runs in a bigger context with more relevant input.

The four steps of the process are illustrated in the following figure:

Microbenchmarking is fun. However, before diving into the process of trying to make a particular function faster, we should first make sure that:

• The time spent inside the function when running the program significantly affects the overall performance of the program we want to speed up. Profiling and Amdahl's law will help us understand this. Amdahl's law will be explained below.

• We cannot easily decrease the number of times the function is being called. Eliminating calls to expensive functions is usually the most effective way of optimizing the overall performance of the program.

Optimizing code using microbenchmarking should usually be seen as the last resort. The expected overall performance gains are usually small. However, sometimes we cannot avoid the fact that we need to make a relatively small piece of code run faster by tuning its implementation, and in those cases, microbenchmarks can be very effective.

Next, we will learn how the speedup of a microbenchmark can affect the overall speedup of a program.