TextIOWrapper‽ converting files to strings in Python

Ever encountered a TextIOWrapper object in Python when you really wanted a string? Converting an _io.TextIOWrapper object to a string is fortunately pretty easy: call the read method!

TextIOWrapper objects are files

If you use Python's built-in open function to read from a file, you'll end up with a _io.TextIOWrapper object. You can think of this as a file object.

>>> file = open("example.txt", mode="rt")
>>> type(file)
<class '_io.TextIOWrapper'>

If you open a file in read mode (the default mode), you should be able to call the read method on your file object to read your file into a string:

>>> contents = file.read()
>>> contents
'This is an example text-based file.\nIt existed before we read it.\n'

More on reading text files in reading files in Python.

_io.TextIOWrapper aren't the only "files"

Due to duck typing, we often care about the behaviors of an object more than the actual type of an object.

Python's io module includes a number of file-like objects, including StringIO objects, which are actually stored in memory within your Python process instead of a disk/drive.

Binary files in Python (which read bytes instead of strings) are often represented with io.BufferedReader.

>>> file = open("example.txt", mode="rb")
>>> type(file)
<class '_io.BufferedReader'>

Also Python's urllib module's Response objects act like binary files. They have read, close and other methods and attributes that files are meant to have:

>>> from urllib.request import urlopen
>>> response = urlopen("https://pseudorandom.name")
>>> response
<http.client.HTTPResponse object at 0x7ff34625f430>
>>> response.read()
b'Meagan Dunn\n'
>>> response.readable()
True
>>> response.close()

Python's _io.TextIOWrapper object is the usual file object type you'll see because that's the one you get back when working files in text mode (which allows us to read strings from a file rather than raw bytes).

Don't try to pass a file to str

What would happen if you passed your TextIOWrapper object to Python's built-in str function to convert to a string?

>>> file = open("example.txt", mode="rt")
>>> str(file)
"<_io.TextIOWrapper name='example.txt' mode='r' encoding='UTF-8'>"

Unfortunately, that just gives us back a generic string representation saying the object we've converted is an _io.TextIOWrapper object.

Why doesn't this work?

Well, reading a file isn't an entirely trivial operation. In fact, when you read from a file, if you read the same file a second time, you'll see that it's empty:

>>> file.read()
'This is an example text-based file.\nIt existed before we read it.\n'
>>> file.read()
''

If you need to read a file twice, you could use the file seek method:

>>> file.seek(0)
0
>>> file.read()
'This is an example text-based file.\nIt existed before we read it.\n'

Or, better yet, you could store the string you read into a variable and then work with that variable instead:

>>> with open("example.txt", mode="rt") as file:
...     contents = file.read()
...
>>> contents
'This is an example text-based file.\nIt existed before we read it.\n'

You can also read line-by-line

What if your file is really big?

In that case, you probably don't want to read it all at once. You could instead read your file line-by-line, by looping over it:

>>> with open("example.txt", mode="rt") as file:
...     for line in file:
...         print("Line:", line.rstrip())
...
Line: This is an example text-based file.
Line: It existed before we read it.

More on that in reading files line-by-line in Python.

Use read to convert _io.TextIOWrapper objects to strings

The most important thing to remember is that _io.TextIOWrapper objects are file objects in Python.

If you've found that you have a _io.TextIOWrapper file object, but you need a string, just call that file's read method!