Python – Page 2 – Code Notes

Eval in Python 3; or, how to restore a string to a dictionary or list

One of my students scraped a website and built a CSV from the data. Inadvertently, he had some columns that appeared to contain a list of Python dictionaries. To his surprise, he found he could not read these as lists or as dictionaries directly from his CSV.

I knew that they were stored as strings, and I knew there must be a way to restore the data to its original data types. Google led me to Stack Overflow (of course), and the solution could hardly be simpler:

from ast import literal_eval
result = literal_eval(my_string)

We were then able to pluck out the value of a given dictionary key-value pair with ease.

ast is a built-in module (see docs).

Python and pickle

So you’ve got a complex object in Python, and you want to write it out to a file. For example, a dictionary of dictionaries. You can’t just write it out as you would plain-text.

This article from DataCamp gives a great explanation of the pickle module, which not only solves that problem; it also goes the other way, enabling you to un-pickle the saved data and use it in its original complex form.

This writing out of a data object is especially important for journalists, because you’ll want to be able to check and re-check your results. If you re-scrape data, the new dataset might not be the same as the one you used in the first place.

Note that there is also something faster (for very large datasets) called cPickle, but for Python 3 it has changed to _pickle (see this post).

pickle is a built-in module (see the docs).

A new Flask tutorial for journalism students

This semester I’ve been gradually building out a single, comprehensive python-beginners repo at GitHub, and the latest segment is all about getting started with Flask — a popular Python framework for building web apps (and easier than Django).

I have tried to use various books and online tutorials to teach Flask for the past two years, but I’ve finally given up on that because there’s just so much extra stuff (confusing for my students), or they are outdated and largely wrong now, or both.

Miguel Grinberg’s new, fully updated mega-tutorial for Flask is comprehensive and thorough — but it’s just too thorough, really, for journalism and communications students who only learned Python about four weeks ago.

Some handy Python tricks, and a Python3 cheat sheet

This blog post demonstrates 10 cool Python tricks that are not covered in introductory lessons, involving tuples, lists, join, etc. I have documented some of these already here in Code Notes, but the format of the blog post is really excellent for a quick skim.

There are many Python3 cheat sheets available out there, but this one is actually clear and easy to read — unlike many others!

Fixing Jupyter Notebook startup error

I previously wrote about Jupyter Notebooks here.

Before the MacOS Sierra 10.12.5 update, to launch Jupyter in your browser, you only needed to do this:

Open Terminal.
Enter the desired directory.
Activate your virtualenv (if using one).
At the command line, type:

jupyter notebook

A new window would open in your default web browser, and there were all your Jupyter files.

But after the MacOS update, instead we got an execution error: localhost doesn’t understand the “open location” message. And the browser window did not open.

The way to fix it (do it once) is here.

Note: In the config file, I had to search for

#c.NotebookApp.browser =

instead of the string the author provided.

Combining lists in Python

Another cool shortcut method in Python! So, you have two lists. You want to pair them up, in order. That is, you want to associate the first item in list A with the first item in list B, the second item in list A with the second item in list B, and so on.

grades = ["B", "D", "A", "B"]
students = ["Ken", "Ben", "Wendy", "Sandy"]
for student, grade in zip(students, grades):
   print( student + ": " + grade )

This would be nice for creating a list of dictionaries, which is essentially JSON format:

student_records = []
for student, grade in zip(students, grades):
   student_records.append( {"student":student, "grade":grade} )

for record in student_records:
   print(record)

{'student': 'Ken', 'grade': 'B'}
{'student': 'Ben', 'grade': 'D'}
{'student': 'Wendy', 'grade': 'A'}
{'student': 'Sandy', 'grade': 'B'}

So easy! zip() does it all.

Tuple unpacking in Python

With the typical list-based for-loop in Python, we don’t use indexes:

foods = ["bread", "milk", "eggs"]
for food in foods:
   print(food)

But if we also need the index number for each list item (which does, of course, exist):

foods = ["bread", "milk", "eggs"]
for i, food in enumerate(foods):
   print( str(i) + ": " + food )

I learned that today from this 25-minute video about loops in Python (also covers while-loops):

Tuple unpacking is explained at 11:58 in the video. It’s what makes enumerate() possible.

(PyCon 2017)

Python 3.x for the win

Good news, I think.

“Today, there are very few libraries that do not support Python 3. Python 3 Readiness shows that 342 of the 360 top packages for Python support 3.x.” (Read more. A link-filled resource.)

I learned Python from Zed Shaw’s Learn Python the Hard Way, which predisposed me to use Python 2.x. Also, as a Mac OS user, I already have Python 2.x without downloading or installing anything. Little by little, though, I’ve started to do all new projects in Python 3.x.

SMS and Google Sheets with Python

Student presentations in my advanced class are usually very interesting, and we all learn about new libraries and tools.

This week, Ryan S. showed an app that a soon-to-be-married fellow built to manage invitations and RSVPs for his wedding (read about the app).

“Sending an SMS or MMS is one of the most common tasks performed on the Twilio Platform” (see example).

The Twilio API is quite a Swiss Army Knife for making phone calls and sending/receiving SMS messages! Read more about SMS and Twilio.

The Python-happy groom also used gspread, a Python client library for the Google Sheets API (docs and GitHub).

Just another demonstration that you can do anything with Python!

Jupyter Notebooks

To install: Install (instructions at jupyter.org). Note that pip or pip3 install works. It’s on that page, after the Anaconda part.

You do not have to use Anaconda, which installs a lot of extra things.

You can install into a virtualenv.

After installing (with virtualenv activated, if you installed it that way), in Terminal, at bash prompt:

jupyter notebook

On Mac OS and Chrome, a new browser tab opens automatically, and you’re in the same folder where you were in the Terminal. If you have someone else’s notebook or a folder full of notebooks, you can toss them into that folder using Finder, just like any files.

Above: Two folders and one notebook file. Below: A notebook, open for work.

Create a new notebook file: The New button is on the far right side.

The thing I find hard to remember: You have to press Shift-Return to run the code in one of the boxes, or to save markdown you wrote. On the Cell menu, there’s an option to Run All.

Menus and icons: Very self-explanatory. Explore them.

File menu: “Revert to checkpoint” lets you roll back to the previous save.

File menu: “Close and Halt.” Saves the current notebook file and closes it.

To quit Jupyter, go to the Terminal and Control-C (not Command).