Python Iterables and Iterators Explained

Python iterables and iterators are the two closely related concepts that power every for loop you have written, hidden behind clean syntax that the language provides. When you write a loop that processes a list of names or a range of numbers, you are using the iteration protocol without needing to see its internals. But understanding what iterables and iterators actually are changes how you think about Python's data model. It explains why you can loop over a list, a string, a dictionary, a file, and a generator all with the same for keyword. It also unlocks more advanced patterns like building your own iterable objects and processing data one item at a time without loading everything into memory.

An iterable is any Python object that can be asked to produce its members one at a time. Lists, tuples, strings, sets, dictionaries, range objects, file handles, and generators are all iterables. The key property of an iterable is that you can pass it to Python's built-in iter() function and receive an iterator in return. The iterator is a separate object whose sole job is to remember where you are in the traversal and hand over the next item each time you ask. This separation between the thing being iterated over and the thing doing the iterating is what makes Python's iteration model both flexible and memory efficient.

If you have worked through the earlier section on Python control flow, you already know that for loops are Python's primary tool for repeated execution. But the for loop is not magic. Under the hood, it creates an iterator from the iterable you supply and then calls next() on that iterator in a loop until the iterator signals that it has no more items. The StopIteration exception that the iterator raises at exhaustion is caught silently by the for loop machinery, and the loop body stops executing. This design means any object that follows the iteration protocol can plug into Python's looping syntax without the loop needing to know anything about the object's internal structure.

What makes something an iterable

The formal definition of an iterable in Python is an object that implements the iter() method, which must return an iterator object. In practice, many Python objects also support iteration through the older getitem() protocol, which the interpreter uses as a fallback. If an object defines getitem() for integer indexes starting at zero and raises IndexError when the items run out, Python can construct an iterator from it automatically. This backward compatibility mechanism is why you rarely see iter() defined explicitly in older Python code, even though modern code should use iter() for clarity.

Every built-in collection type in Python is an iterable. Lists, tuples, sets, and dictionaries all implement the iteration protocol so that for loops can walk through their contents. Strings are iterables that yield individual characters. File objects are iterables that yield lines of text, which is why you can loop over an open file with for line in file and process enormous log files without reading the entire file into memory at once. Even objects that produce values on the fly, like range objects, enumerate wrappers, and zip iterators, are all iterables that generate their values when asked rather than storing them in advance.

Understanding that collections are iterables also helps you see why Python's data processing functions work the way they do. Built-in functions like sum(), min(), max(), any(), all(), and sorted() all accept iterables as their primary argument, not just lists. You can pass a tuple, a set, a generator expression, or a custom iterable object to any of these functions and they will consume it through the same iteration protocol that a for loop uses. This uniformity is one of the reasons Python feels cohesive even as you move from basic scripting into more sophisticated data transformation tasks.

What an iterator does differently

An iterator is the object that does the actual work of stepping through an iterable's items. It keeps track of the current position so that each call to next() returns the next item in sequence. Crucially, an iterator is also an iterable itself: it implements iter() by returning self. This means you can pass an iterator anywhere an iterable is expected, and it will work correctly. But unlike a list or a tuple, an iterator is consumed as you use it. Once you have called next() enough times to exhaust the iterator, it is empty forever. You cannot rewind it or start over; you must create a fresh iterator from the original iterable.

This one-shot nature of iterators is the most common source of confusion when beginners first encounter the iteration protocol. If you call list() on an iterator to collect its remaining items, then try to loop over it again, you will get nothing because the iterator has already been exhausted. The original iterable, if it is not itself an iterator, can still produce fresh iterators on demand. A list is not consumed by iteration: you can write for x in my_list as many times as you like, and each time Python creates a new iterator that walks the list from the beginning. The list itself never changes state during iteration, which is why it behaves predictably across multiple passes.

The iterator protocol also explains why you can use iterators with functions that expect iterables. The built-in next() function is the explicit interface for advancing an iterator. When you call next(it), the iterator's next() method runs, producing the next value or raising StopIteration. You can also supply a default value as the second argument to next(), which the function returns instead of raising StopIteration when the iterator is empty. This pattern is useful when you want to handle exhaustion gracefully without a try-except block.

How for loops use iterables and iterators

Consider the simplest possible for loop:

python

numbers = [10, 20, 30]
for n in numbers:
    print(n)

What Python actually does under the hood is approximately this sequence. First, it calls iter(numbers) to obtain an iterator object from the list. Second, it enters an internal loop that repeatedly calls next() on that iterator. Third, it assigns each returned value to the loop variable n and executes the body of the loop. Fourth, when the iterator raises StopIteration, Python catches it, breaks out of the loop, and continues with the code after the for block. You can verify this behaviour by manually constructing the equivalent while loop:

python

numbers = [10, 20, 30]
iterator = iter(numbers)
while True:
    try:
        n = next(iterator)
        print(n)
    except StopIteration:
        break

The two code blocks produce identical output, but the for loop version is shorter, clearer, and less error-prone. The for loop also handles cleanup correctly: if the loop body raises an exception, the for loop still releases the iterator properly, which matters when the iterator holds external resources like file handles or network connections. The manual while loop would need additional try-finally logic to match that behaviour.

Because every for loop follows this same pattern, you can substitute any iterable in place of the list. A tuple of coordinates, a set of unique user IDs, a dictionary of configuration keys, a range of integers, a file of log entries, or a generator that reads from a database cursor all work with the same for syntax. The loop does not care what the iterable is, only that the iterable can produce an iterator and the iterator can produce values one at a time.

The iterable-iterator relationship in practice

The relationship between iterables and iterators is easiest to remember with a concrete metaphor. Think of a book as the iterable and a bookmark as the iterator. The book contains all the pages, but it does not track which page you are currently reading. You can create as many bookmarks as you like, each starting at page one, and each bookmark tracks its own position independently. Reading a page through one bookmark does not affect the position of any other bookmark, and the book itself remains unchanged no matter how many times you read it.

In Python terms, the list is the book. Calling iter() on the list creates a bookmark (the iterator). Calling next() on the bookmark reads the next page and advances the bookmark. When you reach the last page, the bookmark signals that there are no more pages to read. You can create a fresh bookmark and read the book again from the start. The list never changes, and each iterator maintains its own independent position.

This separation of concerns is not just a theoretical nicety. It is the reason Python can support nested loops over the same collection without interference. When you write a nested loop that iterates over a list inside another loop that iterates over the same list, each loop gets its own iterator. The inner loop's progress does not disturb the outer loop's position because they are using different bookmarks in the same book. This would not work if the list itself tracked the current position, because the inner loop would advance the shared position and the outer loop would skip items.

Iterators and memory efficiency

One of the most practical benefits of the iterator model is lazy evaluation, the ability to produce values on demand rather than computing and storing them all at once. A range object with a billion integers does not allocate memory for a billion integers. It stores the start, stop, and step values, and each call to next() computes the next integer formulaically. The iterator only ever holds one integer in memory at a time, which means you can loop over an enormous range without exhausting your computer's RAM.

File iteration works the same way. When you iterate over a file object with a for loop, Python reads one line from disk at a time. A log file that is ten gigabytes on disk can be processed with a simple for loop that never holds more than one line in memory. If you called readlines() instead, Python would read all ten gigabytes into a list at once, which would likely crash your program. Understanding the iterator model helps you make the right choice between these approaches.

This lazy pattern extends to generators, which are covered later in this section. A generator is a special kind of iterator that you define with a function containing the yield keyword. Each call to next() runs the function until the next yield statement, then pauses. The function's local variables are preserved between calls, so the generator can maintain complex state without storing all its results. Generators are the Pythonic way to build data pipelines that process streams of data without materializing the entire stream in memory.

Common misconceptions about iterables

A frequent point of confusion is the assumption that every iterable is also an iterator. It is not. Lists, tuples, strings, sets, and dictionaries are all iterables, but none of them is an iterator. You cannot call next() directly on a list. If you try, Python raises a TypeError telling you that the list object is not an iterator. The list must first be passed to iter() to obtain an iterator that can be advanced with next(). This is the correct and intentional design: iterables are sources of data, and iterators are the mechanisms that traverse those sources.

Another misconception is that iterators are always faster than other approaches. While iterators enable lazy processing and can reduce memory usage, they introduce function-call overhead on every next() call. For small collections where all the data fits comfortably in memory, iterating over a plain list is usually faster than iterating over a generator that yields the same values. The performance benefit of iterators is not raw speed but the ability to process datasets that would otherwise not fit in memory at all. Choose iterators when the data is too large to materialize, when the data arrives incrementally from an external source, or when you need to compose multiple processing stages into a pipeline.

A third misconception is that all iterables support multiple independent traversals. While lists, tuples, and strings do, some iterables are themselves iterators and can only be traversed once. Files are a common example. A file object is both an iterable and an iterator: it returns itself from iter(), and each call to next() reads the next line. Once you have read to the end of the file, the file object is exhausted, and you must reopen the file to read it again. Generators behave the same way: a generator object is an iterator, and once you have consumed its values, it cannot be restarted.

Recognizing which iterables can be reused and which are single-use helps you avoid subtle bugs. If a function accepts an iterable argument and consumes it with a for loop, passing a generator or a file object means the caller can never iterate over that data again. The next article, which covers the Python iterator protocol in detail, shows you exactly how iter() and next() work together to define this behaviour for any object you design.