Create Python Generator Functions with `yield`

Creating a generator function in Python is one of the simplest and most rewarding skills you can add to your toolkit after understanding how iterators work. The syntax is almost identical to writing a regular function: you use the def keyword, choose a descriptive name, define parameters in parentheses, and write a colon followed by an indented body. The only visible difference is that somewhere in that body, you write yield instead of return. That single keyword transforms the function from one that runs to completion and returns a single value into one that can pause, hand a value back, and resume on demand, potentially producing thousands or millions of values over its lifetime without ever storing them all at once.

If you have read the introduction to Python generators, you already know that generators are a concise way to create iterators that produce values lazily. This article focuses on the practical syntax: how to write the def line, where to place yield, how to structure the function body, and what patterns produce clean, maintainable generator functions. Every generator you write, from a three-line counter to a multi-stage data pipeline, follows the same fundamental structure.

Writing your first generator function

The simplest possible generator function contains a single yield statement and produces exactly one value. While a one-value generator is not particularly useful on its own, it demonstrates the core mechanism. Define a function with def, write yield followed by the value you want to produce, and call the function to get a generator object:

python

def one_value():
    yield 42
 
g = one_value()
print(next(g))

Calling one_value() does not print anything and does not execute the function body. It creates a generator object and returns it. When you call next(g), the function body runs from the beginning, executes the yield 42 statement, pauses, and returns 42 to the caller. A second call to next(g) would raise StopIteration because there are no more yield statements to execute and the function body reaches its end.

A more realistic first generator produces a sequence of values by placing yield inside a loop. The loop variable becomes part of the generator's paused state, preserved between calls:

python

def count_up_to(limit):
    current = 0
    while current < limit:
        yield current
        current += 1
 
for num in count_up_to(5):
    print(num)

The for loop calls next() on the generator object repeatedly. Each call resumes the function body, checks the while condition, executes yield current to hand back the current number, and increments the counter before the next pause. When current reaches limit, the while condition becomes false, the function body ends, and the generator raises StopIteration, causing the for loop to exit. At no point does the generator create a list of all five numbers; it computes each one on demand.

Generator functions with parameters

Generator functions accept parameters exactly like regular functions. You can define positional parameters, keyword arguments, default values, and even *args and **kwargs. The parameters are set when the generator object is created, and they remain available as local variables throughout the generator's lifetime. This means you can parameterize the behaviour of your generator without writing a class with an init method.

A practical example is a generator that reads a file and yields lines that match a search term. The filename and the search term are parameters, and the generator logic is a simple loop with a condition:

python

def search_lines(filename, term):
    with open(filename) as f:
        for line in f:
            if term in line:
                yield line.strip()

Each call to next() reads one line from the file, checks whether it contains the search term, and yields it if it does. The file is opened when the first next() call enters the with block, and Python guarantees that the file is closed when the generator is garbage collected or when its close() method is called. The generator never holds more than one line in memory, so you can search through files of any size without worrying about RAM.

You can also use default parameter values to make generators configurable without requiring the caller to specify every option. A range-like generator that defaults to starting at zero with a step of one lets callers write compact, readable code for the common case while still supporting custom start and step values when needed.

Using multiple yield statements

A generator function is not limited to a single yield inside a loop. You can scatter yield statements throughout the function body, including inside if-else branches, try-except blocks, and even nested functions that you define inside the generator. Each yield is an independent pause point, and the generator resumes from whichever yield it last executed.

This flexibility is valuable when the value-production logic has distinct phases. A generator that reads a CSV file might first yield the header row, then yield each data row, and optionally yield a summary row at the end. The three phases are three separate yield statements at different points in the function body, each governed by its own logic:

python

def process_csv(filename):
    with open(filename) as f:
        header = next(f).strip()
        yield header.split(",")
        row_count = 0
        for line in f:
            row_count += 1
            yield line.strip().split(",")
        yield ["end", str(row_count)]

The first yield hands back the header as a list of column names. The loop yields each data row as a list of values while counting them. After the loop, a final yield sends a summary marker with the row count. The caller does not need to know about these distinct phases; it simply iterates over the generator and receives whatever value comes next.

This multi-phase pattern is also useful for generators that produce setup and teardown values. A generator that manages a database connection might yield the connection object as its first value, yield query results in the middle, and yield a status summary at the end, all from a single function body whose local variables track the current phase.

Infinite generators

A generator that never reaches the end of its function body will never raise StopIteration, which means it will produce values forever. Infinite generators are useful for sequences like counters, random value streams, and sensor simulators where the consumer decides when to stop, not the producer.

The simplest infinite generator is an unbounded counter, paired here with islice to consume a fixed number of values:

python

def infinite_counter(start=0):
    current = start
    while True:
        yield current
        current += 1
 
from itertools import islice
first_five = list(islice(infinite_counter(10), 5))

This produces [10, 11, 12, 13, 14] without the infinite generator ever knowing or caring that only five values were requested.

Infinite generators are also the natural way to model ongoing data sources in tests. A generator that simulates a temperature sensor can yield values forever, and your test code can consume as many as it needs with islice or a loop that breaks after a condition is met. The same generator can drive a production system that runs for hours, yielding millions of values without any change to the generator code.

Handling cleanup with try-finally

Generator functions can use try-finally blocks to guarantee that resources are released even if the generator is not fully consumed. This is important when the generator opens files, acquires locks, or establishes network connections. If the caller stops iterating before the generator is exhausted, Python calls the generator's close() method, which raises a GeneratorExit exception inside the generator at the point where it is paused. The finally block catches this and runs cleanup.

A generator that opens a file and yields lines can use try-finally to ensure the file is closed regardless of how many lines are consumed:

python

def safe_read(filename):
    f = open(filename)
    try:
        for line in f:
            yield line.strip()
    finally:
        f.close()

If the caller iterates through every line, the for loop finishes, the try block completes, and the finally block closes the file. If the caller breaks out of the loop early, Python calls close() on the generator, which raises GeneratorExit at the yield point inside the try block, and the finally block runs, closing the file. In both cases, the file handle is released promptly, avoiding resource leaks.

The with statement provides an even safer alternative for most resource management scenarios. Because with guarantees cleanup when the block exits, you can open a file inside a with block within the generator, and Python handles the cleanup through the normal with mechanism. The try-finally pattern is necessary when you are managing resources that do not support the context manager protocol or when you need custom cleanup logic beyond what with provides.

Nesting generators with yield from

When one generator needs to produce all the values from another generator or iterable, writing an explicit for loop with yield for each item works but is verbose. The yield from statement, introduced in Python 3.3, delegates iteration to a sub-generator or any iterable. Instead of looping and yielding each value individually, you write yield from sub_generator, and Python handles the iteration internally. For example, a generator that writes yield from range(3) followed by yield from ["a", "b", "c"] would first yield 0, 1, 2 from the range, then yield "a", "b", "c" from the list, all without an explicit for loop. The yield from statement also handles bidirectional communication between the caller and the sub-generator. The next article covers passing values with yield, which builds on these patterns.