Common Python Standard Library Mistakes

The Python standard library is well-designed, but many of its modules include sharp edges that can cause security problems, data loss, or hard-to-debug failures if used incorrectly. This article collects the most common mistakes across the modules covered in this section and shows the safe alternative for each.

Using random instead of secrets for security

The random module is for simulations and games, not security. Its Mersenne Twister algorithm is deterministic and can be predicted by an attacker who observes enough output. For passwords, tokens, and API keys, use secrets instead, which draws from the operating system's cryptographically secure random source.

python

import random
import secrets
 
api_key = "sk-" + "".join(random.choices("abcdef0123456789", k=32))
 
api_key = "sk-" + secrets.token_hex(16)

The random version is predictable. The secrets version uses cryptographically secure randomness. See Secure Passwords with Python secrets for details.

Using os.system instead of subprocess

os.system() passes commands through the shell, which is both a security risk and gives you no output capture. Any part of the command string built from user input can be hijacked with shell metacharacters like ; or &&, turning a simple file read into arbitrary command execution.

python

import os
 
filename = "user_input; rm -rf /"
os.system(f"cat {filename}")
 
import subprocess
 
result = subprocess.run(["cat", filename], capture_output=True, text=True)
print(result.stdout)

os.system(f"cat {filename}") would execute the injected rm -rf / command. subprocess.run() with a list keeps the argument separate from the command, preventing injection.

Unpickling untrusted data

pickle can execute arbitrary code during deserialization, since the format supports reconstructing arbitrary Python objects, including ones that run code as a side effect of being built. Never unpickle data from an untrusted source, such as a network request or a file a user uploaded.

python

import pickle
 
data = request.body
obj = pickle.loads(data)
 
import json
 
obj = json.loads(data)

Use JSON for data exchange with external systems. Reserve pickle for trusted local storage like caching. See Serialize Objects with Python pickle.

Forgetting timeouts on network calls

Network operations can hang indefinitely without a timeout, since the operating system has no way to know the difference between a slow response and a server that will never reply.

python

from urllib.request import urlopen
 
response = urlopen("https://api.example.com")
 
response = urlopen("https://api.example.com", timeout=10)

Always set timeout on urlopen(), socket connections, and any blocking network call. Ten seconds is a reasonable default for most APIs.

Not using with for file and resource management

Forgetting to close files, sockets, or database connections causes resource leaks that can exhaust file descriptors or hold locks other processes are waiting on, especially in long-running programs.

python

file = open("data.txt")
content = file.read()
 
with open("data.txt") as file:
    content = file.read()

The with statement guarantees the file is closed, even if an exception occurs. The same pattern applies to sqlite3 connections, network sockets, and locks.

Catching overly broad exceptions

A bare except: or except Exception: catches everything, including KeyboardInterrupt and SystemExit, making the program hard to stop with Ctrl+C and hiding bugs that should have crashed loudly instead of failing silently.

python

try:
    value = int(user_input)
except:
    value = 0
 
try:
    value = int(user_input)
except ValueError:
    value = 0

Catch the specific exception you expect. except ValueError handles bad input. except KeyboardInterrupt is never caught accidentally.

String formatting in SQL queries

Building SQL queries with string formatting invites SQL injection, since any value that ends up in the string is treated as part of the SQL command rather than as data.

python

import sqlite3
 
cursor.execute(f"SELECT * FROM users WHERE name = '{name}'")
 
cursor.execute("SELECT * FROM users WHERE name = ?", (name,))

Use ? placeholders and pass values as a tuple. The database driver handles escaping and prevents injection.

Using == to compare tokens

Regular string comparison is vulnerable to timing attacks, since == typically stops at the first mismatched character, and the tiny difference in how long that takes can leak information about the correct value one character at a time.

python

if user_token == expected_token:
    login()
 
import secrets
 
if secrets.compare_digest(user_token, expected_token):
    login()

secrets.compare_digest() compares in constant time, so an attacker cannot measure response time to guess the token character by character.

Manipulating the heap list directly

After building a heap with heapq, modifying the list directly breaks the heap property.

python

import heapq
 
heap = [3, 1, 4]
heapq.heapify(heap)
heap.append(2)
 
heapq.heappush(heap, 2)

Only use heappush, heappop, and heapify on a heap. Direct list operations invalidate the heap invariant. The smallest item at heap[0] may no longer be correct.

Using groupby on unsorted data

itertools.groupby only groups consecutive matching elements.

python

from itertools import groupby
 
data = [("B", 1), ("A", 2), ("B", 3)]
for key, group in groupby(data, key=lambda x: x[0]):
    print(key, list(group))
 
data.sort(key=lambda x: x[0])
for key, group in groupby(data, key=lambda x: x[0]):
    print(key, list(group))

Sort the data by the same key first. Otherwise elements with the same key that are not adjacent become separate groups.

Forgetting to encode strings before hashing

Hash functions in hashlib operate on bytes.

python

import hashlib
 
digest = hashlib.sha256("password").hexdigest()
 
digest = hashlib.sha256("password".encode()).hexdigest()

The first version raises TypeError. Always encode strings to bytes with .encode() before hashing.

Writing to stdout for errors

Mixing data output and error messages on stdout makes it impossible to separate them.

python

print("Error: file not found")
 
import sys
print("Error: file not found", file=sys.stderr)

Write error messages to sys.stderr. This lets users redirect normal output to a file while still seeing errors in the terminal.

Quick reference