Debugging Memory Usage and Timing Information

Memory usage

Because pandas requires that we load our data into RAM, we need to be aware of the size of our data. Because pandas doesn’t mutate data (in general), we’ll need some overhead to be able to work with data. It’s recommended that we have 3–10 times more memory than the size of the data we’re analyzing.

One way to explore the data is to look at the info method. We just need to remember to use the memory_usage='deep' option so we take into account any Python objects the DataFrame might use (strings ...