Customizing Pickles

Overview

With most common Python objects, pickling just works. Basic primitive types such as integers, floats, and strings can be pickled, as can any container objects, such as lists or dictionaries, provided the contents of those containers are also picklable. Further, and importantly, any object can be pickled, so long as all of its attributes are also picklable.

So, what makes an attribute unpicklable? Usually, it has something to do with dynamic attribute values subject to change. For example, if we have an open network socket, open file, running thread, subprocess, processing pool, or database connection stored as an attribute on an object, it will not make sense to pickle these objects. Device and operating system state will be meaningless when we attempt to reload the object later. We can’t just pretend the original thread or socket connection exists when we reload. No, we need to somehow customize how such transient and dynamic data is dumped and loaded.

Example

Here’s a class that loads the contents of a web page every hour to ensure that they stay up to date. It uses the threading. Timer class to schedule the next update: