Find Slow Python Code While Debugging

Your code runs correctly, but it takes ten seconds when it should take one. Finding which part is slow is a debugging skill separate from finding which part is broken. Python includes built-in tools for measuring execution time, and the standard library provides a profiler that shows you exactly where your program spends its time.

This article covers three tools: manual timing for quick checks, the timeit module for precise measurements, and cProfile for comprehensive profiling. All three are built into Python and require no installation, unlike the breakpoints you set when debugging Python code with pdb.

Quick timing with perf_counter

The fastest way to check if a section of code is slow is to wrap it with timing calls. The time.perf_counter function returns a high-resolution timestamp. Call it before and after the code you want to measure, and subtract:

python

import time
 
start = time.perf_counter()
result = process_large_dataset(data)
elapsed = time.perf_counter() - start
 
print(f"Processing took {elapsed:.3f} seconds")

This tells you the total wall-clock time for that block. If the number is unexpectedly large, the bottleneck is somewhere inside that block. If it is small, the bottleneck is elsewhere.

This technique requires no setup and works inside any function, including test methods and Jupyter cells.

For comparing two approaches, wrap each one separately and print both timings. The difference tells you which approach is faster without needing a full profiler.

Precise micro-benchmarks with timeit

Manual timing is noisy. A single measurement can be affected by background processes, CPU throttling, or garbage collection. The timeit module solves this by running your code many times and reporting the best result.

Use timeit from the command line for quick checks on small expressions:

bash

python -m timeit "sum(range(1000))"

The output shows the number of loops, the best time per loop, and an estimate of how long it would take to run many iterations. The module automatically chooses how many times to run the expression to get a stable measurement.

Use timeit in your code when you need to compare two implementations of the same function:

python

import timeit
 
def approach_one():
    return "-".join(str(n) for n in range(100))
 
def approach_two():
    return "-".join(map(str, range(100)))
 
t1 = timeit.timeit(approach_one, number=10000)
t2 = timeit.timeit(approach_two, number=10000)
print(f"Approach one: {t1:.4f}s, Approach two: {t2:.4f}s")

The number argument controls how many times the function runs. Larger numbers give more stable results. The function is called in a loop, so avoid side effects that accumulate across iterations.

Whole-program profiling with cProfile

When you do not know which function is slow, use cProfile. It tracks every function call in your program and produces a report showing cumulative time, call counts, and per-call averages.

Run cProfile from the command line on any Python script:

bash

python -m cProfile -s cumtime my_script.py

The -s cumtime flag sorts output by cumulative time, putting the slowest functions at the top. The output shows several columns: the ncalls column shows how many times each function was called.

The tottime column shows total time spent inside the function itself. The cumtime column includes time spent in functions it called.

A function with high tottime does the heavy lifting itself. A function with low tottime but high cumtime delegates to slow children.

Both patterns are useful: the first tells you to optimize the function directly, and the second tells you to look at what it calls.

For large programs with thousands of function calls, save the profile data to a file for later analysis:

bash

python -m cProfile -o output.prof my_script.py

Instead of reading raw profiler output on the command line, load the saved file into the pstats module and query it interactively in Python:

python

import pstats
stats = pstats.Stats("output.prof")
stats.sort_stats("cumtime").print_stats(20)

This prints the 20 slowest functions by cumulative time. Change "cumtime" to "tottime" to sort by self-time. Change "ncalls" to find functions called most frequently.

Common patterns in profile output

A function called millions of times with a small per-call cost can be a bottleneck just from volume. String concatenation inside a tight loop, repeated dictionary lookups, and unnecessary list copies all fall into this category.

A function called once that takes five seconds is clearly the bottleneck. Look at what it does: file I/O, network calls, large data processing. These are the functions where optimization pays off most.

A recursive function with high call counts may indicate a missing memoization or an algorithm with exponential complexity. Check whether the recursion depth and call count match your expectation for the input size.

Profiling in tests

You can profile individual test functions to find slow tests. A test that takes three seconds does not seem like a problem until you have a hundred of them. Wrap the test body with cProfile calls or use pytest's built-in profiling plugins.

python

import cProfile
import pstats
 
def test_slow_operation():
    profiler = cProfile.Profile()
    profiler.enable()
    result = expensive_computation()
    profiler.disable()
    stats = pstats.Stats(profiler).sort_stats("cumtime")
    stats.print_stats(5)
    assert result == expected

This runs the test normally but prints the five slowest functions afterward. You can decide whether the test is genuinely slow or whether the code it tests needs optimization.

When not to profile

Do not profile until you have confirmed there is a performance problem. Premature optimization wastes time and produces code that is harder to read.

Write clear code first, measure second, and optimize only the parts that measurements identify as bottlenecks.

The next article covers common Python testing and debugging mistakes, which will help you avoid the pitfalls that make both testing and debugging harder than they need to be.