Common File Handling Mistakes in Python

You can spend an afternoon learning the correct way to open, read, write, and close files in Python, but it takes weeks of debugging to encounter every way those operations can go wrong. Python file handling mistakes are not always obvious from the documentation because the official examples assume a clean environment: the file exists, the path is correct, the encoding matches, and the file is small enough to fit in memory. Real projects violate every one of those assumptions, and the mistakes that follow are predictable enough that learning them upfront saves you hours of confusion later.

This article collects the most common file handling mistakes, explains why each one causes problems, and gives you the correct pattern to use instead. It draws on every concept covered in this section, from the fundamentals of opening and closing files in Python through the patterns for reading and writing large files in Python. If you have worked through the earlier articles, the mistakes described here will feel familiar because you have already learned the correct approach alongside each concept. This article consolidates them into a single reference you can return to when debugging file-related problems.

Forgetting to close files

The most pervasive file handling mistake is also the simplest: calling open without a matching close. A file object consumes an operating system file descriptor, and the operating system limits how many descriptors a single process can hold. In a short script that opens one file and exits, the descriptor is reclaimed when the process ends. In a long-running program that opens files in a loop, each unclosed file accumulates until the limit is reached and the program crashes with an OSError about too many open files.

The correct fix is the with statement, which guarantees close is called when the block exits regardless of how the block ends. The with statement is not an advanced feature to adopt later; it should be your default from the very first file you open. If you see an open call without a corresponding with, you are looking at a potential resource leak. The only exception is when a function deliberately returns an open file object to its caller, transferring the responsibility to close it, but even then the caller should use with.

python

with open("data.txt", "r", encoding="utf-8") as file:
    content = file.read()

This three-line pattern is the universal template for safe file access. The file opens at the top of the with block, the work happens inside, and the file closes at the bottom. You never need to write close, and you never need to worry about whether an exception skipped it. The with statement is covered in detail in the article on using the Python with statement for files, but the short version is that every file you open should be opened inside a with block.

Confusing write mode with append mode

Write mode and append mode both let you add content to a file, but they do it in fundamentally different ways that produce opposite results when confused. Opening an existing file with write mode truncates it to zero length before your first write call. All previous content is gone, permanently, from the moment the open succeeds. Opening an existing file with append mode preserves every byte and positions the write pointer at the end, so your new content is added after the old.

The mistake pattern is opening a log file or journal with write mode on every run. The first run creates the file and writes an entry. The second run opens with write mode, erases the first entry, and writes a new one. After a hundred runs, the file contains exactly one entry: the most recent one. The fix is to open the file with append mode instead, so each run adds its entry without touching the entries that came before. The article on appending data to files in Python covers this distinction in depth, but the rule is simple: if you want the file to grow over time, use append mode.

Ignoring encoding

Python's default text encoding varies by platform. On macOS and Linux, it is UTF-8. On Windows, it is typically a locale-specific encoding like cp1252. A Python script that opens a text file without specifying an encoding works correctly on one machine and silently corrupts non-ASCII characters on another. The corruption is particularly insidious because it might not raise an error; the wrong encoding can produce garbled characters that look valid at a glance.

The fix is to always pass an explicit encoding when opening text files. UTF-8 is the right choice for nearly every modern text file, and writing encoding="utf-8" costs a few extra characters in exchange for guaranteed portability. If you are reading a file that you know uses a different encoding, specify that encoding instead. If you are writing a file that will be consumed by another program, write a comment or documentation noting which encoding you used so the consumer can open it correctly.

python

with open("report.txt", "r", encoding="utf-8") as file:
    text = file.read()

This explicit encoding parameter makes the contract between your code and the file system visible. Anyone reading your code knows that you expect UTF-8 text, and Python will raise a UnicodeDecodeError if the bytes on disk are not valid UTF-8, which is far better than silently producing incorrect output. The error tells you exactly where the mismatch is so you can fix it by specifying the correct encoding instead of spending hours wondering why your output looks wrong.

Loading large files entirely into memory

The read method with no arguments and the readlines method both load the entire file content into memory. For a small configuration file or a short text document, this is fine. For a server log file that has been accumulating for months, this can allocate gigabytes of memory and crash your program. The mistake is not using these methods at all; they are correct for small files. The mistake is using them without considering the file's size.

The fix depends on what you need to do with the file. For line-oriented text processing, use a for loop over the file object, which reads and yields one line at a time. For binary files or non-line-oriented formats, use chunked reading with a fixed buffer size. Both patterns keep memory usage constant regardless of file size. The articles on reading files in Python and reading and writing large files in Python cover these patterns in detail. The habit to develop is to ask yourself, before calling read or readlines, "Do I know how large this file is, and am I sure it fits in memory?" If the answer is not a confident yes, switch to a streaming pattern.

Forgetting that write does not add newlines

The write method writes exactly the string you give it, with no added characters. If you call write three times with three different strings and none of them end with a newline, the resulting file contains one long concatenated string with no line breaks. The mistake is assuming that write adds a newline automatically the way print does when you call it with default arguments.

The fix is to include the newline escape at the end of each string you want on its own line. The pattern file.write(line + "\n") or file.write(f"{data}\n") ensures that each write call produces a complete line. If you are writing a list of lines with writelines, each string in the list must already include its newline. When you later read the file back, each line will include its trailing newline, which you can remove with strip or rstrip as needed.

Hardcoding path separators

Typing a forward slash or backslash into a file path string might work on your development machine, but it creates a path that breaks on other operating systems. A backslash in a path string works on Windows but produces a literal backslash character on macOS and Linux. A forward slash works on macOS, Linux, and surprisingly on Windows when passed to Python's open function, but fails when passed to command-line tools or Windows system APIs.

The fix is to never type a separator character in a path string. Use pathlib and the forward slash operator to join path components, or use os.path.join if you are working in an older codebase. Both approaches insert the correct separator for the current platform. The article on working with file paths in Python covers both APIs in detail. Adopting pathlib for new code eliminates an entire category of cross-platform path bugs with no runtime cost.

Opening a file without checking if the path is a directory

If you accidentally pass a directory path to open with a read or write mode, Python raises IsADirectoryError. This mistake usually happens when iterating over the contents of a folder and forgetting to filter out subdirectories, or when a user-provided path turns out to be a directory instead of the expected file. The fix is to check path.is_file before opening, or to use path.iterdir with a filter that selects only files.

File handling mistakes are easy to make and usually easy to fix once you recognize the pattern. The common thread running through all of them is that the correct approach is already built into Python, often in the form of a single keyword or a one-line pattern change. Using with instead of manual open and close, passing an explicit encoding, choosing the right mode for the job, and processing files in streams rather than loading them wholesale are habits that prevent the vast majority of file handling bugs before they happen.