Python Notes and Examples
← prev | next →     Top-level ToC     /generators.html     (printable version)

First: Iterators.

1 Iterators

x = range(4)
type(x) #=> <class 'range'>

y = iter(x)
type(y) # an iterator object.

next(y) #=> 0
next(y) #=> 1
next(y) #=> 2
next(y) #=> 3
next(y) #=> exception: StopIteration

# Could've also done `y.__next__()` over and over.

If you want to be able to loop over instances of one of your own classes (the same way you can loop over lists, strings, dicts, and sets), then you’ll need to have your class implement the iterator protocol. To do this, your class must implement the __iter__() method; this will make it so you can call iter(your_instance) and get back an iterator. The __iter__() method should return an iterator — an object with a __next__() method — such that next(the_iterator) works on it.

Note: lists, strings, dicts, sets, and files all implement the iterator protocol. They are not iterators themselves:

>>> z = [11, 12, 13]
>>> next(z)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'list' object is not an iterator

When you try to loop over anything, Python implicitly calls iter(the_object) behind the scenes for you.

2 Generators

Functions that contain a yield return a generator. A generator implements the iterator protocol:

>>> def f():
...  yield 10
...  for i in range(3):
...   yield i
...  yield 100
... 
>>> x = f()
>>> next(x)
10
>>> next(x)
0
>>> next(x)
1
>>> next(x)
2
>>> next(x)
100
>>> next(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

>>> y = f()
>>> for i in y:
...  print(i)
... 
10
0
1
2
100

# Eagerly get all values from the iterator:
z = list(f())

You might use a generator in situations like this:

def get_compute_intensive_results(coll):
    for x in coll:
        yield time_consuming_computation(x)

Finally, note that there’s a shorthand for creating generators: it’s like a list comprehension but with parens instead of brackets.

foo = (x**2 for x in range(5))
next(foo) #=> 0
next(foo) #=> 1
next(foo) #=> 2
# etc.