Read Files in Python

Python read files is one of the most common operations you will perform in real projects. A configuration file tells your program which settings to apply. A data file provides the input your script will process. A log file lets you review what happened during a previous run. In every case, you need to pull text from disk into your program's memory so you can work with it using the string methods, loops, and data structures you already know. Python gives you three main reading methods on the file object, and each one is suited to a different kind of reading task. Choosing the right method is the difference between a program that handles a hundred-megabyte data file gracefully and one that freezes because it tried to load the entire file into memory at once.

The open function and file modes were covered in the previous article on opening and closing files in Python. If you have not read that article yet, start there. The file object that open returns is the same object you will call read methods on, and understanding the mode that object was opened with determines which reading operations are allowed and how the data is presented to your code.

Reading an entire file with read()

The read method, called with no arguments, reads the entire content of a file and returns it as a single string. When you first open a file, the internal position marker is at the beginning, and read pulls everything from that point to the end. After read finishes, the position sits at the end, and calling it again returns an empty string because there is nothing left to consume.

python

with open("poem.txt", "r", encoding="utf-8") as file:
    content = file.read()
    print(content)

This pattern is clean and readable, and it is the right choice when you need the entire file content at once and you know the file is small enough to fit comfortably in memory. A short poem, a configuration file, a single day's log output, or a small JSON document all fit this description. The with statement ensures the file is closed after the block, even if the read itself raises an error.

The read method also accepts an optional size argument that limits how many characters it reads in a single call. Passing read(1024) reads at most 1024 characters and returns them as a string. If fewer than 1024 characters remain in the file, it returns whatever is left without error. Calling it again continues from where the previous call stopped. This chunked reading pattern is one way to process a large file without loading it all at once, though for line-oriented text files, iterating over the file object directly is usually more natural.

Reading one line at a time with readline()

The readline method reads a single line from the file and returns it as a string, including the trailing newline character at the end. If the file's last line does not end with a newline, which is common in text files created by some editors, readline returns the line without one. When readline reaches the end of the file, it returns an empty string, which is your signal to stop reading. This behavior makes it straightforward to build a while loop that processes every line.

python

with open("tasks.txt", "r", encoding="utf-8") as file:
    line = file.readline()
    while line:
        print(f"Task: {line.strip()}")
        line = file.readline()

The while loop condition checks whether line is truthy. A non-empty string is truthy in Python, and the empty string that signals end-of-file is falsy, so the loop stops automatically when there is nothing left to read. Each line retains its newline character, so the call to strip removes it before printing. If you forget to strip the newline, your output ends up double-spaced because print adds its own newline on top of the one already in the string.

The readline method gives you precise control over when and how you read each line. You can skip lines, stop early when you find what you are looking for, or apply different processing logic based on the content of each line before reading the next one. This fine-grained control makes readline ideal for file formats where the meaning of a line depends on a header or marker that appears earlier in the file.

Iterating over the file object directly

The most Pythonic and memory-efficient way to read a text file line by line is to use the file object itself as an iterator in a for loop. When you write for line in file, Python reads one line at a time behind the scenes, yields it to your loop body, and does not keep previous lines in memory after you move to the next one. This pattern handles files of any size because only one line is held in memory at any given moment.

python

with open("server.log", "r", encoding="utf-8") as file:
    for line in file:
        if "ERROR" in line:
            print(line.strip())

This loop scans a server log for lines containing the word ERROR and prints them. The file might be ten lines or ten million lines; the code is identical, the memory usage is constant, and the logic is easy to read. The for loop automatically stops when there are no more lines, so you do not need to check for an empty string or maintain a counter. This is the pattern you should reach for first whenever you need to process a file line by line.

The for loop approach also works well with Python's built-in enumerate function when you need line numbers, and with conditional logic inside the loop body for filtering or transforming lines before processing them further. Because each line comes in as a string, you can apply any of the string methods you learned in the Python strings section, such as strip, split, startswith, or find, directly on the loop variable.

Collecting all lines with readlines()

The readlines method reads all remaining lines from the file and returns them as a list of strings, with each string representing one line including its trailing newline. This is convenient when you need random access to any line in the file, want to count the total number of lines before processing, or need to pass the lines to another function that expects a list.

The list that readlines returns is a normal Python list, which means you can index into it, slice it, iterate over it multiple times, and pass it to any function that accepts a sequence. These conveniences come with a memory cost. For a file with a million lines, readlines builds a list of a million strings, each holding one line of text and its newline. The memory usage can easily exceed the file's size on disk because Python string objects have overhead beyond the raw character data. Reserve readlines for small files where you genuinely need the list structure, and prefer the for loop iterator pattern for line-by-line processing of files whose size you cannot predict.

Choosing the right reading strategy

The choice between read, iterating with a for loop, readline, and readlines depends on two questions: how large is the file, and what do you need to do with its content? If the file is small and you need the entire content as a single string for parsing or transformation, read is the simplest and most direct choice. If you need a list of all individual lines, perhaps to count them, shuffle them, or access them by index, readlines gives you that list at the cost of loading everything into memory.

For files of unknown or large size, the for loop iterator pattern is almost always the right answer. It handles line-by-line processing with constant memory usage and clean syntax. Reserve readline for the narrower case where you need to control the pace of reading explicitly, perhaps because you want to stop reading after a certain condition without iterating through the remaining lines, or because the file format requires you to read a header line, decide how to parse the body based on that header, and then read the body with a different strategy. Regardless of which method you choose, always use the with statement to open the file, which guarantees the file is closed when the block ends even if your reading code raises an exception.