Performance Tips for Concurrent Python

Adding concurrency to a Python program should make it faster. When it does not, the cause is usually not the concurrency model itself but how it is applied. A thread pool with the wrong number of workers.

A lock held too long. Data serialized unnecessarily across process boundaries. An asyncio event loop stalled by a synchronous call hidden deep in a dependency.

This article covers the performance techniques that make the difference between concurrent code that delivers near-linear speedup and concurrent code that runs slower than the sequential original. If you have read about common concurrency mistakes, you know what to avoid. This article covers what to optimize.

The principles are universal across threading, asyncio, and multiprocessing: size your pools correctly, batch work to reduce coordination overhead, minimize time spent in locked sections, avoid unnecessary data copying, and measure before and after every change.

Profile first, optimize second

The most important performance tip is also the most frequently ignored. Do not guess where your bottleneck is. Profile it.

Concurrency adds overhead, and that overhead can hide in places you do not expect.

For threaded code, use cProfile to identify where threads spend time, but be aware it adds its own overhead that can distort timing:

python

import cProfile
import threading
 
def worker():
    total = sum(i * i for i in range(100_000))
 
def run_threads():
    threads = [threading.Thread(target=worker) for _ in range(4)]
    for t in threads: t.start()
    for t in threads: t.join()
 
cProfile.run("run_threads()")

cProfile.run() only installs its profiling hook on the thread that calls it, so this shows time spent starting and joining the threads but not the work each worker does internally. To profile inside worker threads too, call cProfile.Profile().runcall() from within each thread's target function instead.

A more accurate approach for concurrent code is to instrument key sections with time.perf_counter():

python

import time
from concurrent.futures import ThreadPoolExecutor
 
def timed_task(n):
    start = time.perf_counter()
    result = do_work(n)
    elapsed = time.perf_counter() - start
    return result, elapsed
 
with ThreadPoolExecutor() as executor:
    results = list(executor.map(timed_task, range(100)))
 
total_time = sum(r[1] for r in results)
avg_time = total_time / len(results)
print(f"Average task time: {avg_time:.4f}s")

For asyncio, use asyncio.Task.get_stack() and manual timing to identify slow coroutines. For multiprocessing, measure the serialization time separately from computation time.

Sizing thread pools correctly

The number of worker threads is the most impactful tuning parameter for ThreadPoolExecutor. Too few workers leave I/O bandwidth unused. Too many waste memory on thread stacks and increase context-switching overhead.

The optimal count depends on the ratio of I/O wait time to CPU time per task. If a task spends 90% of its time waiting for network responses (I/O) and 10% computing, you can run roughly 10 concurrent tasks per CPU core before CPU becomes the bottleneck. If the ratio is 50/50, only about 2 tasks per core.

python

from concurrent.futures import ThreadPoolExecutor
import time
 
def io_heavy_task(duration):
    time.sleep(duration)
    return duration
 
for workers in [2, 5, 10, 20, 50]:
    start = time.perf_counter()
    with ThreadPoolExecutor(max_workers=workers) as executor:
        list(executor.map(io_heavy_task, [0.1] * 100))
    elapsed = time.perf_counter() - start
    print(f"workers={workers:>2}: {elapsed:.2f}s")

The loop reruns the same 100 tasks with an increasing worker count each time, and the printed timings show how the total runtime drops as more workers can run the sleeps in parallel. Typical output for 100 tasks each sleeping 0.1 seconds looks like this:

text

workers= 2: 5.00s
workers= 5: 2.00s
workers=10: 1.00s
workers=20: 0.50s
workers=50: 0.21s

With 2 workers, 100 tasks take 5 seconds (50 batches of 2). With 20 workers, they take 0.5 seconds (5 batches of 20). Beyond a certain point, adding more workers gives diminishing returns.

The optimal is the smallest number that keeps all cores busy.

Minimizing lock contention

Every with lock: statement serializes threads. The longer a lock is held, the more time other threads spend waiting. Two techniques reduce the cost: minimize the work inside the lock, and batch updates to lock less frequently.

python

# Inefficient: lock held for every iteration
lock = threading.Lock()
counter = 0
 
for item in items:
    result = expensive_computation(item)
    with lock:
        counter += result
 
# Efficient: batch locally, lock once
local_total = 0
for item in items:
    local_total += expensive_computation(item)
 
with lock:
    counter += local_total

The batched version acquires the lock once instead of once per item. The computation runs without contention in every thread.

For read-heavy workloads, consider threading.RLock or a reader-writer lock pattern. Python's standard library does not include a reader-writer lock, but the concept applies: allow multiple simultaneous readers and serialize only writers. For most I/O-bound Python programs, the simpler approach of minimizing lock scope is sufficient.

Reducing asyncio overhead

Asyncio tasks are lightweight, but creating tens of thousands of them still has a cost. Each task allocates a coroutine frame and scheduling state. For extremely high concurrency, batch work into fewer tasks:

python

# Many small tasks: scheduling overhead per task
async def process_one(item):
    await asyncio.sleep(0.01)
    return item * 2
 
tasks = [asyncio.create_task(process_one(i)) for i in range(100000)]
# High scheduling overhead
 
# Fewer batched tasks: less overhead
async def process_batch(items):
    results = []
    for item in items:
        await asyncio.sleep(0.01)
        results.append(item * 2)
    return results
 
batch_size = 1000
batches = [range(i, i + batch_size) for i in range(0, 100000, batch_size)]
tasks = [asyncio.create_task(process_batch(b)) for b in batches]
# 100 tasks instead of 100,000

The batched version creates 100 tasks instead of 100,000, reducing scheduling overhead by three orders of magnitude.

For CPU-bound sections in async code, await asyncio.sleep(0) yields control to let other tasks run. This is a cooperative yield, not a timer-based sleep. Use it sparingly; it adds overhead and should not be needed in well-structured async code where I/O provides natural yield points.

Minimizing data transfer in multiprocessing

Every object sent through a multiprocessing.Queue or returned from a ProcessPoolExecutor is serialized with pickle. For small objects, the cost is negligible. For large data, serialization can dominate runtime.

python

# Inefficient: sends large data through pickle
def process(items):
    return [item * 2 for item in items]
 
large_data = list(range(1_000_000))
with ProcessPoolExecutor() as executor:
    result = executor.submit(process, large_data).result()

The list of one million integers is pickled and sent to the worker, then the result is pickled and sent back. The serialization time may exceed the computation time.

Fix: Use shared memory for large numeric data when possible, or restructure to avoid sending large data:

python

from multiprocessing import shared_memory
import numpy as np
 
# Create shared memory buffer, pass only the name and metadata
shm = shared_memory.SharedMemory(create=True, size=arr.nbytes)
shared_arr = np.ndarray(arr.shape, dtype=arr.dtype, buffer=shm.buf)
shared_arr[:] = arr[:]
 
# Worker attaches to shared memory by name
def worker(shm_name, shape, dtype):
    shm = shared_memory.SharedMemory(name=shm_name)
    arr = np.ndarray(shape, dtype=dtype, buffer=shm.buf)
    return np.sum(arr * 2)

The shared memory approach passes only a name string and metadata rather than the full array. The worker reads directly from the shared buffer.

Using chunksize with multiprocessing pools

Pool.map() and executor.map() divide input into chunks sent to workers. The chunksize parameter controls how many items are in each chunk.

python

from multiprocessing import Pool
import time
 
def fast_task(x):
    return x * 2
 
if __name__ == "__main__":
    with Pool(processes=4) as pool:
        for chunksize in [1, 10, 100, 1000]:
            start = time.perf_counter()
            pool.map(fast_task, range(10000), chunksize=chunksize)
            elapsed = time.perf_counter() - start
            print(f"chunksize={chunksize:>4}: {elapsed:.4f}s")

Small chunksizes give better load balancing when task durations vary because each worker gets many small batches. Large chunksizes reduce IPC overhead for uniform tasks because fewer chunk handoffs occur. Benchmark with your actual workload and task duration distribution.

Connection pooling for async database access

Creating a new database connection for every async task is expensive. Use a connection pool that reuses connections across tasks:

python

import asyncpg
 
async def main():
    pool = await asyncpg.create_pool(
        dsn="postgresql://localhost/mydb",
        min_size=5,
        max_size=20,
    )
 
    async with pool.acquire() as conn:
        rows = await conn.fetch("SELECT * FROM users")

The pool maintains persistent connections. acquire() takes an available connection from the pool. When the async with block exits, the connection returns to the pool.

This pattern eliminates the connection setup cost per request.

The same principle applies to HTTP sessions with aiohttp:

python

async def fetch_all(urls):
    async with aiohttp.ClientSession() as session:
        tasks = [fetch(session, url) for url in urls]
        return await asyncio.gather(*tasks)

A single ClientSession reuses TCP connections across all requests, avoiding the overhead of establishing a new connection for each URL.

When concurrency is not the answer

Sometimes the best performance tip is to not use concurrency at all. Before adding threads, processes, or async, verify that:

• The synchronous version is actually too slow for your requirements.
• The bottleneck is not in an external system that would be overwhelmed by concurrent requests.
• A faster algorithm or data structure cannot solve the problem in a single thread.
• Caching results from previous runs cannot eliminate the work entirely.

A well-optimized single-threaded program often outperforms a naive concurrent one. Use concurrency as a multiplier on already-efficient code, not as a substitute for efficiency.