Once a file is open, you need to pull data out. Python gives you four ways to read a text file, and the differences between them matter once your files grow past a few megabytes. This lesson walks through read(), readline(), readlines(), and iteration over the file object, shows when each one is appropriate, and looks at chunked reads, binary reads, and encoding errors.

A text file opened with open(path, "r", encoding="utf-8") gives you a file object with several read-flavored methods. They differ in how much they read at once and what they return.

Five rows, four ideas. The last two (iterating the file object and calling readline in a loop) do the same thing internally, but iteration is the idiomatic Python form.

The same product file read four different ways:

Every read includes the trailing \n from each line. You strip those yourself, usually with rstrip() or splitlines(). readlines() returns a list of strings, which means every line lives in memory at once, the same as read(). Iterating the file object yields one line at a time and never holds the whole file in memory.

That last property is what makes the iteration form the default. Use it unless there's a specific reason to do otherwise.

f.read() with no argument returns the entire contents of the file as a single str. It's the simplest method and fits when the file is small enough that "all of it in memory" is fine.

For a 200-byte file like this one, slurping is fine. For a 200 MB log file, it allocates 200 MB of str and pegs your memory. For a 20 GB file, it crashes.

f.read() allocates a str the size of the file. A 1 GB log file means roughly 1 GB of Python memory while you process it. For files larger than a few hundred megabytes, iterate the file object instead.

Two practical use cases for read():

For everything else, prefer the streaming forms.

read() has a positional argument too: read(size) returns up to size characters in text mode (or up to size bytes in binary mode). It's how you do chunked reads when you want bounded memory but don't want to deal with line boundaries:

Each call advances the cursor. When you hit the end of the file, read(size) returns an empty string, which is the signal to stop. Chunked reads come up again later in this lesson with binary files and large file processing.

f.readline() reads up to and including the next \n (or to the end of the file, whichever comes first) and returns it as a str. The newline character is kept in the result. When there's nothing left to read, it returns an empty string.

The empty string after the last line is the end-of-file signal. This is the loop pattern you'd use if you wanted explicit control over reading:

That works, but it's clunky. Python already gives you iteration over file objects, which does the same thing in one line. readline() directly fits two niche cases: reading the header from a file without consuming the body, or implementing a custom parser that needs to peek ahead and back off.

The first readline consumes only the first line. The read after it picks up from wherever the cursor sits, which is right after the first newline. Mixing reads on a single file object is allowed; the cursor moves with each call.

f.readlines() reads the entire file and returns a list of strings, one per line. Newlines are kept in each string, same as readline().

This looks convenient, and for small files it is. The problem is memory. readlines() keeps every line in a list, so a 1 GB log file becomes a 1 GB string buffer plus the per-string overhead of millions of small str objects. The per-object cost adds up: each str carries a header of around 50 bytes on CPython, so a file of 10 million short lines may use more memory as a list of strings than the file's size on disk.

The motivation for readlines() is usually "I want to iterate the lines." But iterating the file object already does that, line by line, without building the list. Compare:

Both produce the same output. The second form is strictly better for any file larger than "small". Use readlines() only when you need the list, like when you need random access to lines by index or want to process the file twice without reopening it.

readlines() reads the entire file and constructs a list of str objects. Memory usage is roughly the file size plus 50 bytes of overhead per line. For a 10 GB log, that's 10 GB plus the list overhead.

A file object opened in text mode is iterable, and iterating it yields one line per step. This is the idiomatic Python form and the one to use first.

Three properties make this the default:

enumerate is handy when you want the line number along with the content:

And for many real tasks, the parsing fits in a generator expression or a comprehension:

The file object behaves like any other iterable. Once that's clear, file processing is another form of "iterate and transform."

The diagram traces what happens when you iterate a file object. Python keeps a buffer in memory (typically 8 KB) and reads from disk in chunks. Each step of the loop scans the buffer for the next \n, yields the line up to and including that newline, and refills the buffer when needed. The full file never sits in memory at once. End-of-file raises StopIteration internally, which is how for loops know to stop.

Every line returned by readline, readlines, or file iteration includes the trailing \n (or \r\n on Windows files read in binary mode). Usually you want it gone before processing the content.

Three string methods come up here:

For most product data and log lines, line.rstrip() fits. It handles all three line-ending styles (\n, \r\n, \r) and also strips trailing spaces.

If you want all the lines as a list of clean strings (without writing a comprehension), the splitlines() method on the whole-file string is a one-liner:

splitlines() is smart about line endings: it handles \n, \r\n, and \r all the same way, and it doesn't keep the separator. The trade-off is that you've slurped the whole file, which is fine for small files and wrong for huge ones.

When you don't want to load the whole file at once but you also don't care about line boundaries, read(size) reads up to size characters in text mode (or bytes in binary mode). It fits binary files, files that aren't line-oriented (like a long XML document), and streaming-style processing.

Each call reads up to 20 characters and advances the cursor. The last call returns a short chunk because the file didn't have a full 20 characters left. The call after that returns an empty string, which is the end-of-file signal.

In text mode, the size is a character count, not a byte count, and Python takes care of not splitting a multi-byte UTF-8 character across two chunks. In binary mode, size is a byte count and there's no character awareness; you can split anything, anywhere.

Binary chunked reads are how you process large media files, network downloads, or anything where you want to push data through in fixed-size pieces.

For non-text files (images, PDFs, compiled archives), open the file with "rb" and read bytes instead of str.

The first 8 bytes of every PNG file are the same. Reading them gives you a fast way to verify the file type without parsing the rest. That kind of header check is what binary chunked reads are designed for.

Binary reads have a few practical differences from text reads:

The takeaway: text mode for human-readable formats, binary mode for everything else.

A common mistake with file reading is calling read() or readlines() on a file too big to fit in memory. The fix is almost always the same: iterate.

Consider a 4 GB log file with around 50 million lines. Three ways to count lines that contain the word "error":

The first form allocates a 4 GB string. The second form materializes a list of 50 million strings (which costs more memory than the file itself, because each str carries object overhead). The third form keeps one line in memory at a time.

The same principle applies to binary files. To hash a large file, read in fixed-size chunks:

The function never holds more than chunk_size bytes in memory regardless of the file's total size. 1 MB chunks are a reasonable default for large-file work; smaller chunks waste time on extra system calls, larger ones use more memory than necessary.

Memory Benchmark: Slurp vs Iterate

To make the cost concrete, run this benchmark. Generate a 100 MB text file, then process it two ways and watch resident memory.

The "after iterate" memory is barely higher than the baseline because nothing accumulated. The "after slurp" memory includes the entire file as a str. On a 20 MB file that's not catastrophic; on a 2 GB file you'd see the difference between "runs fine" and "out of memory."

Iterating a file object reads from an 8 KB buffer in memory. Slurping reads the entire file. The wall-clock difference is small for small files and large for large ones. When in doubt, iterate.

Text mode decodes bytes into str using the encoding you specified. When the bytes don't match the encoding, you get UnicodeDecodeError.

ASCII only knows the first 128 code points. The byte 0xc3 (the first byte of UTF-8's encoding of é) is outside that range, so the decoder gives up.

The ideal fix is to use the correct encoding. Pass encoding="utf-8" to the read and the file decodes cleanly. But sometimes you don't control the input: a partner system sends a CSV in mixed encoding, a log file has occasional binary garbage, a user uploaded a file labeled wrong. For those cases, the errors argument to open tells Python what to do when a byte can't be decoded.

The first three are the most commonly used.

"replace" substitutes a placeholder, which makes the bad data visible without crashing. "ignore" drops the bad bytes and merges the surrounding characters, which can produce text that looks normal but has lost information. "backslashreplace" keeps every byte in some form, which is the safest choice for logs and diagnostics.

None of these substitutes for picking the correct encoding. They're escape hatches for processing messy input where crashing isn't acceptable, like log analysis pipelines that have to handle gigabytes of mixed-encoding data without stopping.

A few real-world reading patterns that come up often.

Line-by-Line Processing

The most common case: read a text file one line at a time, transform each line, accumulate results.

The pattern is always the same: open in a with block, iterate the file, rstrip the newline, split or parse the line, store the result. For CSV files specifically, the csv module is better than split.

Parsing CSV-Like Text by Hand

Before the csv module, this is what you'd write for simple files where every row has the same shape and no commas inside fields:

This works for clean inputs. The moment a product name has a comma in it ("mug, large"), the hand-rolled split breaks. That's what the csv module is for. Use this pattern only when you control the data format and know it's simple.

Reading Config Files

Small text-based config files (the old .ini or simple key=value style) follow the same pattern as line-by-line processing, with a bit of structure.

partition is a useful alternative to split here: it splits on the first occurrence of the separator, so "key=value=with=equals" becomes ("key", "=", "value=with=equals") instead of being mangled. For anything more complex than a flat key-value file, use the configparser standard library module instead.

A few recurring mistakes worth listing.

Forgetting to Strip the Newline

The lines read from a file end in \n. "mug\n" == "mug" is False. Always rstrip (or splitlines on a slurped string) before comparing.

Calling read() Twice on the Same File

The first read consumed the file. The cursor is at the end, so the second read returns nothing. The same issue applies when iterating a file and then trying to iterate it again in the same with block:

If you need to re-read, either reopen the file or call f.seek(0) to move the cursor back to the start.

Using readlines() for Counting

This works, but it loads the whole file into a list to take the list's length. sum(1 for _ in f) does the same thing in constant memory.

Reading a Binary File in Text Mode

PNG, JPEG, PDF, and most other binary formats start with bytes that aren't valid in any text encoding. Open them with "rb".