What Is Generator Expression

This section provides a quick introduction of generator expression, which contains a 'for' clause enclosed in parentheses and returns a generator iterator.

What Is Generator Expression - A generator expression is a special expression that contains a "for" clause enclosed in parentheses and returns a generator iterator.

Here is the syntax of a generator expression:

(EXPRRESSION for VAR in ITERABLE)

The logic of using a generator expression can be described as:

Basically, a generator expression converts an existing iterable into another iterable by applying a given expression to all items.

A generator expression can also be viewed as as shortened version of a generator function without name and parameters. So x and y in the following code are equivalent.

def f():
   for VAR in ITERABLE:
      yield EXPRRESSION

x = f()

y = (EXPRRESSION for VAR in ITERABLE)

In order to understand the behavior of a generator expression, I wrote the following tutorial example, generator_expression.py. It uses a custom iterable class to trace what happens when a generator expression is evaluated.

#  generator_expression.py
#- Copyright 2022 (c) HerongYang.com. All Rights Reserved.
#
class top3:
   items = ['C', 'Java', 'JavaScript']

   def __init__(self):
      print('top3.__init__(): initializing a new object')
      self.i = 0

   def __iter__(self):
      print('top3.__iter__(): returning a new object')
      return top3()

   def __next__(self):
      print('top3.__next__(): returning the next item')
      if self.i < 3:
         self.i += 1
         return top3.items[self.i-1]
      else:
         raise StopIteration()

g = (x+x for x in top3())
print("object g: id and type =", id(g), type(g))

i = iter(g)
print("object i: id and type =", id(i), type(i))

print(next(i))
print(next(i))
print(next(i))
print(next(i))
print(next(i))

If you run this sample code, you should get:

herong$ python generator_expression.py

top3.__init__(): initializing a new object
top3.__iter__(): returning a new object
top3.__init__(): initializing a new object

object g: id and type = 4387195696 <class 'generator'>

object i: id and type = 4387195696 <class 'generator'>

top3.__next__(): returning the next item
CC

top3.__next__(): returning the next item
JavaJava

top3.__next__(): returning the next item
JavaScriptJavaScript

top3.__next__(): returning the next item
Traceback (most recent call last):
  File "generator_expression.py", line 33, in <module>
    print(next(i))
StopIteration

As you can see from the output, when a generator_expression is evaluated, the ITERABLE evaluated first, then iter(ITERABLE) is called immediately.

In other words, iter(ITERABLE) is called before iter(g) is called.

Table of Contents

 About This Book

 Running Python Code Online

 Python on macOS Computers

 Python on Linux Computers

 Built-in Data Types

 Variables, Operations and Expressions

 Statements - Execution Units

 Function Statement and Function Call

Iterators and Generators

 What Is Iterator Object

 What Is Iterable Object

 Iterable Objects of Built-in Data Types

 What Is Generator Iterator

What Is Generator Expression

 What Is Filtered Generator Expression

 What Is Double-Generator Expression

 List, Set and Dictionary Comprehensions

 Classes and Instances

 Modules and Module Files

 Packages and Package Directories

 "sys" and "os" Modules

 "pathlib" - Object-Oriented Filesystem Paths

 "pip" - Package Installer for Python

 SciPy.org - Python Libraries for Science

 pandas - Data Analysis and Manipulation

 Anaconda - Python Environment Manager

 Jupyter Notebook and JupyterLab

 References

 Full Version in PDF/EPUB