Python and pickle

So you’ve got a complex object in Python, and you want to write it out to a file. For example, a dictionary of dictionaries. You can’t just write it out as you would plain-text.

This article from DataCamp gives a great explanation of the pickle module, which not only solves that problem; it also goes the other way, enabling you to un-pickle the saved data and use it in its original complex form.

This writing out of a data object is especially important for journalists, because you’ll want to be able to check and re-check your results. If you re-scrape data, the new dataset might not be the same as the one you used in the first place.

Note that there is also something faster (for very large datasets) called cPickle, but for Python 3 it has changed to _pickle (see this post).

pickle is a built-in module (see the docs).