Create Custom Iterators in Python

Building a custom iterator in Python means writing a class that implements iter() and next(), the two methods behind every for loop, comprehension, and iterable-consuming built-in function in the language. You use these methods indirectly every time you loop over a list or a dictionary, but the real power of Python's iteration model shows up when you design your own classes and want them to work with the same for loop syntax that built-in types support. Once your class follows the protocol, iter() returning an iterator and next() returning the next item or raising StopIteration, it plugs into the same iteration machinery as every built-in collection.

Custom iterators are not just for library authors or framework developers. Any time you have a class that represents a collection of items, a sequence of data pages from an API, a stream of sensor readings, or a range of values computed on the fly, making that class iterable with a custom iterator lets you use it with for loops, list comprehensions, the sum() and sorted() built-in functions, and every other Python tool that consumes iterables. The effort required is modest, usually two short methods, and the payoff is code that reads like native Python rather than like a workaround.

Before you write a custom iterator, it helps to be comfortable with the Python iterator protocol, which defines the exact contract that iter() and next() must follow. Once you understand that contract, creating an iterator is a matter of applying the pattern to your own data and your own traversal logic.

The two-class pattern: separating iterable from iterator

The most robust way to create a custom iterable is to separate the data-holding class from the traversal class. The data class stores the items and implements iter() to return a new instance of the iterator class. The iterator class stores a reference to the data and a current position, implements iter() by returning self, and implements next() to advance the position and return the next item. This separation means each call to iter() on the data object produces a fresh iterator starting from the first item, which supports nested loops, multiple passes, and all the behaviour you expect from a built-in list.

Imagine you are building a simple paginated collection where PaginatedCollection stores records and creates iterators on demand. The iterator class handles the traversal:

python

class PaginatedIterator:
    def __init__(self, records):
        self.records = records
        self.position = 0
 
    def __iter__(self):
        return self
 
    def __next__(self):
        if self.position >= len(self.records):
            raise StopIteration
        record = self.records[self.position]
        self.position += 1
        return record

The data class, PaginatedCollection, does not track any iteration state. Its only job is to implement iter() by returning a new PaginatedIterator. With both classes in place, the collection works with for loops, list(), and comprehensions, and each operation creates its own independent iterator that starts from the first record.

The single-class pattern: when one pass is enough

Not every iterable needs to support multiple traversals. Some data sources produce values once and cannot be rewound, files being read from a network stream, live sensor data arriving in real time, or generator-style computations that consume their input as they go. For these cases, you can combine the iterable and iterator roles into a single class where iter() returns self and next() advances through the data.

python

class Countdown:
    def __init__(self, start):
        self.current = start
    def __iter__(self):
        return self
    def __next__(self):
        if self.current < 0:
            raise StopIteration
        value = self.current
        self.current -= 1
        return value

Looping over Countdown(3) prints 3, 2, 1, 0. Looping again produces nothing because self.current is already below zero and next() raises StopIteration immediately.

Building a range-like iterator from scratch

Python's built-in range is implemented in C for performance, but you can build the equivalent with the two-class pattern. The iterable StepRange stores start, stop, and step, delegating to StepRangeIterator for traversal:

python

class StepRangeIterator:
    def __init__(self, start, stop, step):
        self.current = start
        self.stop = stop
        self.step = step
    def __iter__(self):
        return self
    def __next__(self):
        if (self.step > 0 and self.current >= self.stop) or \
           (self.step < 0 and self.current <= self.stop):
            raise StopIteration
        value = self.current
        self.current += self.step
        return value

This handles both positive and negative steps correctly, raising StopIteration when the current value passes the stop boundary. You can use a StepRange in for loops and pass it to sum(), list(), or any function that accepts an iterable.

Using iter() with a sentinel value

The two-argument form of the iter() built-in function, iter(callable, sentinel), creates an iterator that repeatedly calls callable with no arguments until callable returns the sentinel value, at which point the iterator raises StopIteration. This pattern is especially useful when you are consuming data from an external source that uses a special value to signal the end of input, like a file reader that returns an empty string at EOF or a socket reader that returns None when the connection closes.

Here is a reader that fetches messages until a shutdown sentinel appears, combined with the sentinel iter() call:

python

class MessageReader:
    def __init__(self, messages):
        self.messages = messages
        self.index = 0
    def read_message(self):
        if self.index >= len(self.messages):
            return "SHUTDOWN"
        msg = self.messages[self.index]
        self.index += 1
        return msg
reader = MessageReader(["msg1", "msg2", "SHUTDOWN", "msg3"])
for msg in iter(reader.read_message, "SHUTDOWN"):
    print(f"Processing: {msg}")

The loop processes msg1 and msg2, then stops when read_message() returns SHUTDOWN. The message msg3 after the sentinel is never read. This pattern is built into Python's standard library in places like socket.makefile() and is worth knowing when you design your own consumption loops for streams and queues.

Iterators that maintain complex state

The iterator protocol does not limit you to simple counters and index positions. An iterator can maintain any internal state that helps it decide what to produce next. A tree iterator might maintain a stack of nodes yet to visit. A tokenizer iterator might buffer characters and track whether it is inside a quoted string. A paginating API iterator might store the current page number and a flag indicating whether more pages are available.

Here is an iterator that flattens a nested list structure, producing leaf values in depth-first order while maintaining a stack of sub-lists:

python

class FlattenIterator:
    def __init__(self, nested):
        self.stack = [iter(nested)]
    def __iter__(self):
        return self
    def __next__(self):
        while self.stack:
            if (item := next(self.stack[-1], None)) is None:
                self.stack.pop()
            elif isinstance(item, list):
                self.stack.append(iter(item))
            else:
                return item
        raise StopIteration

The iterator pushes a new sub-iterator onto the stack whenever it encounters a nested list, and pops exhausted iterators off the stack. The assignment expression fetches the next item and uses the two-argument form of next() with None as the exhaustion sentinel, so this version assumes the nested data never contains None as a leaf value. Given a nested input like [1, [2, [3, 4], 5], 6], it produces 1 through 6 in order, handling arbitrarily deep nesting without recursion.

When to choose a custom iterator over a generator

Generators, functions that use yield to produce values, are the most common way to create custom iterators in Python. A generator function automatically returns a generator iterator object that implements iter() and next() without you writing either method explicitly. For straightforward iteration where the logic is a linear sequence of steps, generators are the right choice. They are shorter, easier to read, and less error-prone than a hand-written iterator class.

But there are situations where a custom iterator class is better than a generator. If your iterator needs to expose additional methods beyond next(), a generator object only has send(), throw(), and close() in addition to the iterator protocol, and adding custom behaviour is awkward. If your iterator's state is complex enough that maintaining it across yield points becomes confusing, a class with clearly named attributes is easier to reason about. If you need to inspect or reset the iterator's internal state, a class gives you direct access to the attributes that hold that state.

The right choice often depends on the complexity of the iteration logic and whether the iterator is part of a public API. For internal data processing, generators are usually sufficient. For iterators that you expose to users of your library or framework, a well-documented iterator class with named attributes and a clear next() implementation is often more maintainable. Whichever approach you choose, the result is an object that works with Python's next() function and every other iteration consumer in the language.