Using List Comprehensions and Generator Expressions For Data Processing

In this tutorial we’ll get hands-on experience using list comprehensions, set comprehensions, and dictionary comprehensions. And we’ll learn when we to slightly tweak our comprehensions to turn them into more performant generator expressions.

Creating one list out of another list is a very common thing to do in Python, so common that Python includes a special construct just for this purpose: list comprehensions. We’ll get hands-on experience using list comprehensions, set comprehensions, and dictionary comprehensions during this tutorial. We’ll also learn how and when we can slightly tweak our comprehensions to turn them into more performant generator expressions.

We will learn some tricks for figuring out which of our “for” loops can be rewritten as comprehensions and which cannot. We will focus heavily on code readability and code clarity and we’ll discuss when comprehensions help readability and when they hurt.

All new skills will be acquired through practice. We’ll work through many exercises both individually and as a group. All students will also receive a cheat sheet which can be used for guidance during future comprehension-writing journeys.

A laptop with Python installed is required for this workshop.

Audience

This tutorial is for Python programmers who are familiar with looping in Python but haven’t fully embraced list comprehensions and generator expressions. Students are expected to be comfortable with Python’s syntax overall and be familiar with for loops in Python.

Students will walk out of this tutorial with an appreciation for what list comprehensions are good for and how to use them to write more descriptive and more readable Python code. Students will also have an understanding that generator expressions can be used to ease data processing, allowing code to be written in a fundamentally different (and often more clear) way.

Outline

• List Comprehensions (30 minutes)
  • Review of for loops in Python
  • List comprehensions for transformation
  • Copy-pasting from a loop to a comprehension
  • First exercise break
• More List Comprehensions (30 minutes)
  • Comprehensions with filtering
  • Identifying comprehension-ready loops
  • Writing readable comprehensions
  • Second exercise break
  • When to use list comprehensions and when not to use them
• Set & Dictionary Comprehensions (30 minutes)
  • Sets & Set Comprehensions
  • Dictionaries & Dictionary Comprehensions
  • Third exercise break
• Generator Expressions (45 minutes)
  • Python’s many functions that consume iterables
  • Generator expressions as lazy list comprehensions
  • Properties of lazy single-use iterables
  • Fourth exercise break
• Data Processing (45 minutes)
  • Generator piping for data processing
  • Generators combined with tuple, any, all, etc
  • Fifth exercise break
  • Cheat sheet and review of when and how we use these tools

I plan to provide quite a bit of exercise time during this tutorial. I will provide more than enough exercises for the allotted time so that students will not walk out of the room bored by the material.