How Python For Loops Work Under the Hood

Diagram of a Python for loop rebuilt from iter(), next() and StopIteration, with a terminal showing a list_iterator

Python for loops do not count indexes. A for loop asks an iterator for the next item, again and again, until the iterator signals that nothing is left. Three pieces make this happen: the built-in iter() function, the built-in next() function, and the StopIteration exception.

This guide takes the for loop apart in four steps. You will work in the Python REPL first, then write a small Countdown class that a for loop can use. At the end you will run a real for loop and your hand-built version side by side and check that they print the same lines.

In short
  • A for loop calls iter() once, then calls next() for each item.
  • StopIteration is the signal that the items have run out. It is not an error.
  • Any class with __iter__ and __next__ works with for.
  • An exhausted iterator never restarts, so create a fresh one each time you loop.
  1. 0:00Intro
  2. 0:28What you'll build and need
  3. 1:21Call iter() on a list
  4. 2:21Call next() to fetch each item
  5. 3:08Catch StopIteration to end cleanly
  6. 4:06Write a class with __iter__ and __next__
  7. 5:01Trace how for calls your class
  8. 5:47Confirm for matches your manual loop
  9. 6:35Fix common iterator mistakes
  10. 7:20Four steps to rebuild a for loop
  11. 7:43A for loop is just iter() plus next()

Check what you need before you start

The goal is a working loop with no for keyword in it. It is built only from iter(), next() and one exception, and it prints exactly what a normal for loop prints. If your hand-built version and the real for agree, you understand the mechanism and not only the syntax.

This matters beyond lists. The same protocol drives files, dictionaries, generators and anything else you can loop over in Python. Once you know how it works, a for loop is no longer a mystery. You can predict what it does with any object you hand it.

  • Python 3 is installed and you can open the REPL
  • You are comfortable working with lists
  • You know how functions work, because iter() and next() are ordinary functions
  • You can write try/except blocks, because that is how the loop ends

Call iter() to turn a list into an iterator

A for loop first asks for an iterator. It never touches the list's indexes. To see this, open a terminal, type python3, and create a list called nums holding 10, 20 and 30. Then type it = iter(nums) and inspect the result. Python shows a list_iterator object, and type(it) confirms that its class is list_iterator.

This step separates two ideas. The list is iterable, which means it can produce an iterator. The iterator does the walking, one item at a time. The list never changes. Only the iterator keeps track of how far you have gone, so two iterators made from the same list move through it independently.

Your memory address will differ from the one shown. The hex number after 0x changes from machine to machine and from run to run. Look at the class name, not the address.

  • iter(nums) returns a list_iterator, not the list itself
  • type(it) shows <class 'list_iterator'>
  • Your 0x… address will not match the example, and that is expected
python3

Call next() to fetch one item at a time

Now that you hold an iterator, ask it for items. Each call to next(it) returns one value and moves the iterator forward by exactly one. This is the core of every for loop, because each pass through the loop is one hidden call to next().

In the same REPL, call next(it) four times. The first call returns 10, the second returns 20 and the third returns 30. The fourth raises StopIteration because there are no items left. A list iterator always returns items in the order they sit in the list. The indexes are still used, but they are hidden inside the iterator.

Iterators only move forward. There is no rewind and no reset. Once an iterator raises StopIteration it is finished for good. This explains one of the most common bugs, covered later in this guide.

  • Calls one to three return 10, 20 and 30, in that order
  • The fourth call raises StopIteration
  • No method moves an iterator backwards

Catch StopIteration to end the loop cleanly

Fetching items works, but the fourth call crashed. This step turns that crash into a clean ending, the same way for does. Keep calling next() forever, and treat StopIteration as the signal to stop rather than as an error.

Read the code from top to bottom. First, create the iterator. Then while True starts a loop that never ends by itself. Inside, try to get the next item. If StopIteration is raised, break out of the loop. Otherwise, print the item. Running it prints 10, 20 and 30.

The endless loop is deliberate. while True checks no condition, so the break is the only exit. That puts the iterator in charge of deciding when the loop is over. This is the same pattern a for loop follows, almost line for line. CPython runs these steps internally, only faster, which is why you never see StopIteration when you use for.

  • Create the iterator once, before the while loop
  • Put only the next() call inside try
  • Break on StopIteration and print otherwise
  • Check that the output is 10, 20, 30, the same as a for loop
it = iter([10, 20, 30])
while True:
    try: item = next(it)
    except StopIteration: break
    print(item)

Write a class with __iter__ and __next__

Lists already know how to hand out an iterator. Now teach your own class the same trick. Python does not care what your class is. If it has the two right methods, and they behave correctly, for will use it. Save the code in a file called countdown.py.

The Countdown class stores a starting number n in __init__. Its __iter__ method returns self, so the object acts as its own iterator. Its __next__ method raises StopIteration when n reaches zero. Otherwise it lowers n by one and returns the previous value.

Each method has one job. The built-in iter() calls __iter__, just as it did on the list earlier. The built-in next() calls __next__, which must either return the next value or raise StopIteration. These two methods are the whole iterator protocol.

  • __iter__ returns self because Countdown is both the iterable and the iterator
  • __next__ returns one value per call
  • __next__ raises StopIteration when n reaches zero
class Countdown:
    def __init__(self, n): self.n = n
    def __iter__(self): return self
    def __next__(self):
        if self.n == 0: raise StopIteration
        self.n -= 1; return self.n + 1

Trace how for talks to your class

Think of for x in Countdown(3) as a conversation between two sides. First, for calls iter(cd), and Countdown returns itself. Then for calls next() and receives 3. Two more calls return 2 and then 1. The last call to next() raises StopIteration, and the loop ends.

iter() runs once per loop, at the very start, before the first item. After that, for talks to the iterator only through next(). The loop catches StopIteration and exits quietly, so the exception never reaches your code. Any object that follows this exchange works with a for loop.

  • One iter() call happens before the first item
  • One next() call happens per item, plus a final call that raises StopIteration
  • No exception appears in your output

Compare the for loop with your manual loop

Now check that the class really works. At the bottom of countdown.py, add a plain for loop over Countdown(3). Below it, add the manual version: get an iterator, loop forever, print next(), and break on StopIteration. If the protocol is correct, both versions must print the same numbers in the same order.

Run python3 countdown.py. The for loop prints 3, 2, 1, and then the manual loop prints 3, 2, 1 again. Matching output shows that your class follows the iterator protocol correctly.

Each version must build its own Countdown(3). If both shared one object, the first loop would exhaust it and the second would print nothing. A fresh object for each version keeps the comparison fair.

  • Both blocks print 3, 2, 1
  • Each block creates its own Countdown(3)
  • The manual block uses while True, try/except and break
for x in Countdown(3): print(x)
it = iter(Countdown(3))
while True:
    try: print(next(it))
    except StopIteration: break

Fix the most common iterator mistakes

If your output does not match, the cause is usually one of three mistakes. Some of them never crash. The loop quietly does nothing, so you need to know what to look for.

A reused iterator is the most common case. The second loop prints nothing because the iterator is already exhausted, so create a new iterator or object. If __iter__ forgets to return self, Python raises TypeError: iter() returned non-iterator, so make __iter__ return self. If __next__ never raises StopIteration, the loop never ends, so raise it when the items run out.

To check for exhaustion quickly, call list() twice on the same Countdown object. If the second call returns an empty list, the first call used up the iterator. That is normal iterator behaviour, even if it surprises you.

  • Second loop prints nothing: create a new iterator or object
  • TypeError: iter() returned non-iterator: make __iter__ return self
  • Loop never ends: raise StopIteration when the items run out
  • Calling list() twice and getting an empty list the second time means the iterator is exhausted

Key takeaways

  • iter() turns a list into an iterator
  • next() fetches one item at a time
  • StopIteration ends the loop cleanly
  • __iter__ and __next__ make your own classes iterable
  • Exhausted iterators never restart, so make a fresh one
  • for = iter() + next() + StopIteration

Frequently asked questions

Does a Python for loop use indexes?

Not directly. A for loop asks the object for an iterator and calls next() on it repeatedly. For a list, indexes are used, but they are hidden inside the list iterator.

What is the difference between an iterable and an iterator?

An iterable, such as a list, can produce an iterator when you pass it to iter(). The iterator does the walking, returns one item per next() call, and remembers its position.

Why don't I see StopIteration when I use a for loop?

The for loop catches StopIteration and treats it as the signal to stop. It exits quietly, so the exception never reaches your code.

Why does my second loop over the same object print nothing?

The iterator was exhausted by the first loop, and iterators only move forward. Create a new iterator or a new object for each loop.

Why does __iter__ return self?

In a class like Countdown, the object is both the iterable and the iterator, so iter() can hand back the object itself. If __iter__ returns nothing, Python raises TypeError: iter() returned non-iterator.

Can I reset an iterator to start over?

No. An iterator has no rewind or reset. Once it raises StopIteration it is finished, so you need to create a fresh one to loop again.

Watch the full video on YouTube →

Souy Soeng

Souy Soeng

Hi there 👋, I’m Soeng Souy (StarCode Kh)
-------------------------------------------
🌱 I’m currently creating a sample Laravel and React Vue Livewire
👯 I’m looking to collaborate on open-source PHP & JavaScript projects
💬 Ask me about Laravel, MySQL, or Flutter
⚡ Fun fact: I love turning ☕️ into code!

Post a Comment

CAN FEEDBACK
Ad