Memory and CPU Profiling

Profiling a Python program means doing a dynamic analysis that measures the execution time of the program and everything that involves. That means measuring the time spent in each of its functions. This data gives you info about where your program is spending time, and what area might be worth optimizing.

This is a very interesting exercise. Many people focus on local optimization, such as determining, for example, which of the Python 2 functions range or xrange is going to be faster. It turns out that knowing which one is faster may never be an issue in your program, and that the time gained by using one of the functions above might not be worth the time you spend researching it or arguing about it with your colleague.

Trying to blindly optimize a program without measuring where it is actually spending its time is a useless exercise. Following your gut alone is not always sufficient.

cProfile

There are many types of profiling, as there are many things that you can measure. Here we focus on CPU and memory utilization profiling, meaning the time spent by each function executing instructions or allocating memory.

Since its 2.5 version, Python provides a few built-in tools to help you in achieving that task. The standard one is cProfile and it is easy enough to use.