Python Generators: A Practical Explanation

Generators are not, as you might first think, some kind of electrical device that generates electrical energy. In Python, a generator is a piece of code that generates a sequence of values and provides them one at a time, without storing all of them at once.

It is like a coffee machine that gives us a cup of coffee on request, but does not keep all the cups of coffee prepared inside it.It would be a waste of resources, and coffee would cool down.

We have seen something very similar to generator with range() function. It is something called iterable, which means it can be iterated with for example for loop. It does not store all values, but yields them on demand. Did I just say yields, as in provide, return, result? Yes, and we will use it soon.

But what type of object is a generator? Generator is a special object, returned by a generator function. That function must contain exactly yield to be classified as a generator function.

First, let us look at an example of a generator. First, let us define a function:

def sequence(n):
    for i in range(n):
        return i
print(sequence(5))

What will be the result of this program when we call it with argument 5? Can you guess?

You are probably familiar with range(5), which will produce:

0, 1, 2, 3, 4

but not 5, since upper limit is not included.

But, this time, the output will be:

0

Why only zero?

Because when i = 0, the function executes:

return 0

and immediately exits the whole function, not just current iteration.

This is inconvenient because it does not go through the remaining values at all.

Let us move one step forward and say that if we replace return with something called yield, we will be able to obtain all these values from the loop.

The word yield can be understood as something that is produced or returned — an output or result.

If we replace return with yield:

# return i
yield i

and print the function output value:

print(sequence(5))

we will get something like:

<generator object sequence at 0x000001A7B61AC2B0>

sequence(5) is a generator, while sequence is a generator function.

range(1, 5) is not a list, but a list-like object — it is a generator .It is similar to a list because we can use a for loop to obtain all of its elements.

Similar to range object, we can not directly print the sequence(5)- we must use for loop to iterate through its elements- kind of.

Kind of, because this generator object does not contain the elements themselves — it generates them whenever we request them.

Therefore, a generator is a function that contains yield, and when it is called, it becomes a generator. Or more formally, a generator object. We can assign it to a variable named, appropriately, generator.

generator = sequence(5)

Now, we can check what is the type of generator.

print(type(generator))
<class 'generator'>

That is why we need a for loop to access elements inside generator. Each iteration of the for loop triggers one iteration of the loop inside the function and returns one value.

for number in generator:
    print(number)
0
1
2
3
4

A generator produces values lazily — one at a time, when we request them.

It does not generate entire list in advance and then iterate over that list with another for loop.

Instead, every time we move to the next element, the function remembers where it stopped and provides the next result using yield.

If we generated a list in advance and then used a for loop to retrieve its elements, we would first have to create the entire list.

If the list were very large, this could require a significant amount of memory.

If we try to iterate through the generator again, what will we get?

for number in generator:
    print(number)
for number in generator:
    print(number)

Second time, nothing will be printed- the loop will not even come to print, since generator contains nothing now. It is empty ,and applying for on an empty something will not execute indented code.

Get Jasenko Krejić’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

This happens because the generator becomes exhausted, and we have to generate it again:

generator = sequence(5)

There is no way to revert the sequence to beginning. This is different from, for example, file handler, when we can use seek(0) after read to position cursor at the beginning again.

We can also directly obtain a list or tuple from a generator, using built-in list/tuple function. That way, we don’t have to do something like create an empty list/tuple and sequentially add elements to it.

print(list(sequence(5)))
[0, 1, 2, 3, 4]

Or:

print(tuple(sequence(5)))
(0, 1, 2, 3, 4)

It is similar to a list comprehension expression inside square brackets, except that we use parentheses instead.

numbers_list = [x for x in range(1, 5)]
print(numbers_list)
[1, 2, 3, 4]

Here, [x ….] is called list comprehension, and the expression inside [g isa called just expression.

But if we do this?

numbers = (x for x in range(1, 5))

You might have expected that we would get tuple instead of list if we used (), instead of [], but it is not the case.

print(numbers)
<generator object <genexpr> at 0x000001818641CC40>
print(type(numbers))
<class 'generator'>

What we got, it also a generator- the one that is very similar to generator we defined manually with generator function sequence.

Now that we have a generator numbers, we can use either for loop or list to get access to elements, like we did with generator generator object.

for number in numbers:
    print(number)
1
2
3
4

Stop for a moment, and think: what will this line of code return?

print(list(numbers))

It will output:

[]

Now, can you remember why? Of course, the generator numbers is exhausted, no more elements to generate, and we only get an empty list. We must recreate it again- numbers- to be able to use it.

numbers = (x for x in range(1, 5))

Generator objects are a special type of iterators, which are yet another interesting topic. Iterators, and therefore generators as well, can be used with next function. If we do:

number =next(numbers)
print(number)
1

next() fetches just one element from the generator. Just like getting a cup of coffee from a coffee machine. If we do it again, we will get another cup….I mean, element.

number = next(numbers)
print(number)
2

And so on. And if we add enough nexts, we will eventually get an error after 4 is printed:

    number = next(numbers)
             ^^^^^^^^^^^^^
StopIteration

With a for loop, we do not see this error because the loop automatically stops when the generator reaches the end.

But next has an option to provide default value instead of raising an exception if end is reached- it is a second, optional, argument. For a line that would otherwise cause a StopIteration error:

number = next(numbers, "EXAUSTED!")
print(number)
EXAUSTED!

You may be surprised, but the same applies with sequence() function we defined earlier. Even though we did not implement anything like next. That is because the moment Python sees yield in a function, it use it as generator function and adds next to its generator.

I hope this article helped you to understand the basic of generators, which can be very confusing at times, with similar concepts like iterators, comprehensions getting in the way.

To recap: a generator in Python produces values lazily, one at a time, using yield, which makes it memory-efficient and allows iteration with for loops or next() without storing all values in advance. Similar to coffee machine yielding one coffee at time, not storing them all at once. Generators, as you have seen, can be created either as a result of calling a generator function(the one that contains yield) or a generator expression. Once all their values have been read, or how it is usually called, “consumed”, they become exhausted and must be recreated to be used again.

If you found this article helpful, consider supporting my work with a small donation on Ko-fi:
https://ko-fi.com/jasenko/donate

If you are learning Python, and want to follow the PCEP Python exam syllabus, check my newest Udemy Course, it is a video tutorial packed with hands-on example, quizzes, coding exercises and two full mock exams:

https://www.udemy.com/course/pass-python-pcep-30-0x-video-lessons-and-mock-tests-2025/

If you want to learn more about AWS Lightsail service, check my course on Udemy on AWS Lightsail.

https://www.udemy.com/course/aws-lighsail-a-different-perspective

Also, if you want to prepare for the basic AWS Certified Cloud Practitioner exam, check another one of my courses, also on Udemy:

https://www.udemy.com/course/aws-certified-cloud-practitioner-crash-course