Python With Open Statement: Learn How to Open a File

In this guide, I will show you how to use the with statement to simplify the way you open and handle files in your Python programs.

The with statement creates a context manager that simplifies the way files are opened and closed in Python programs. Without using the with statement a developer has to remember to close file handlers. This is automatically done by Python when using the with open…as pattern.

We will start by opening a file without the with statement and then we will see the advantages of doing it using the with statement.

Let’s get started!

Syntax to Open a File in Python without Using the With Statement

I have created a file called output.txt that has the following content:

$ cat output.txt 
Line1
Line2
Line3
Line4
Line5 

Let’s have a look at the definition of the open function from the Python documentation:

In this guide, we will focus on the first two arguments of the open function: file and mode.

Here is how we can open our file in read mode using the open function.

Read mode is the default one.

>>> f = open('output.txt')
>>> f.read()
'Line1\nLine2\nLine3\nLine4\nLine5\n'
>>> f.close()
>>> f.closed
True 

We use the open function to create a file handler (f) and then we use the file handler to read the content of the file using the read() function.

After reading the content of the file we use the close() function to close the handler.

Running f.closed we get back True as a confirmation that the file handler is closed.

Let’s see what the file object in the previous example looks like:

>>> f = open('output.txt')
>>> print(f)
<_io.TextIOWrapper name='output.txt' mode='r' encoding='UTF-8'>
>>> print(f.__dict__)
{'mode': 'r'} 

It’s of type TextIOWrapper, its default encoding is UTF-8 and it has an attribute called mode.

To find out the methods you can use on this file object run the following command in the Python shell:

>>> help(f)

Note: Read mode is the default mode used by Python to open files unless you pass a second parameter to the open function (see some examples below):

Mode	Example
r (read – text format)	f = open(filename, ‘r’)
rb (read – binary format)	f = open(filename, ‘rb’)
w (write – text format, truncates the file)	f = open(filename, ‘w’)
wb (write – binary format, truncates the file)	f = open(filename, ‘wb’)
r+ (read and write – text format)	f = open(filename, ‘r+’)
a (append – text format, appends to the end of the file)	f = open(filename, ‘a’)

How to Read a File Using With Open…As in Python

Let’s see what happens if we use the with statement when opening files in Python.

The syntax we will use is:

with open(file, mode) as file_object

When using the with statement a file is automatically closed when it’s not needed anymore. This is confirmed by the fact that in the following code f.closed returns True.

>>> with open('output.txt') as f:
...     data = f.read()
... 
>>> f.closed
True 

Clearing resources on your system is extremely important. Imagine if you create a program that opens hundreds of files and doesn’t close them. For how long it can go on before using all system resources?

So, to recap…

If you don’t use the with keyword you have to remember to call f.close() to free up resources once you don’t need your file anymore. Running f.close() is not required when using the with statement.

What happens if we try to read a file that has already been closed?

>>> f.closed
True
>>> f.read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: I/O operation on closed file. 

Python raises a ValueError exception.

Print All the Lines in a File Using With Open…As

Let’s find out how to print all the lines in a file after opening the file with the with statement.

We will use the file output.txt used in the previous example and loop through the lines in the file one at a time:

with open('output.txt', 'r+') as f:
    for line in f:
        print(line) 

We have passed r+ as the second parameter to open the file for reading and writing.

As you can see I’m using a for loop to go through the lines of the file using the file object.

$ python with_open_example.py 
Line1 

Line2 

Line3 

Line4 

Line5 
 

For some reason, the print statement is adding new line characters that are not present in the original file.

To get rid of them you can use the following syntax:

print(line, end='')

Our code becomes:

with open('output.txt', 'r+') as f:
    for line in f:
        print(line, end='')

[output]
$ python with_open_example.py 
Line1
Line2
Line3
Line4
Line5 

Looks good now 🙂

Open Multiple Files in a Single With Statement

Imagine you have to write a program that takes a single list of strings and writes each string in one of two files.

For example, let’s say we have the following list:

items = ['dog', 'cat', 'apple', 'pear', 'lion', 'banana']

And our program has to write animals to a file called animals.out and fruits to a different file called fruits.out.

Here is how you would do it using two open statements in a single with expression:

items = ['dog', 'cat', 'apple', 'pear', 'lion', 'banana']
   
with open('animals.out', 'w') as animals_f, open('fruits.out', 'w') as fruits_f:
    for item in items:
        if item in ['dog', 'cat', 'lion']:
            animals_f.write(item + '\n') 

        if item in ['apple', 'pear', 'banana']:
            fruits_f.write(item + '\n') 

Let’s run the program and confirm that the two files are created as we expect:

$ python with_open_example.py 
$ cat fruits.out 
apple
pear
banana
$ cat animals.out 
dog
cat
lion 

To be able to use two open statements in one with expression Python 2.7, Python 3.1 or newer is required.

Using Nested With Open Statements in Python

It’s also possible to nest with open statements instead of using two open statements in the same line.

Here is how we can update our previous program using two with nested statements:

items = ['dog', 'cat', 'apple', 'pear', 'lion', 'banana']
   
with open('animals.out', 'w') as animals_f:
    with open('fruits.out', 'w') as fruits_f:
        for item in items:
            if item in ['dog', 'cat', 'lion']:
                animals_f.write(item + '\n') 

            if item in ['apple', 'pear', 'banana']:
                fruits_f.write(item + '\n') 

Below you can see that program still does what it’s supposed to do 🙂

$ python with_open_example.py 
$ cat fruits.out 
apple
pear
banana
$ cat animals.out 
dog
cat
lion 

Using Python With Open to Work with Binary Files

We often work with text file, but what about binary files?

For example, how would you open a PNG file using what we have learned about the with statement?

In the current directory I have downloaded a picture called python.png:

$ ls -ltr
total 208
-rw-r--r--@ 1 myuser  mygroup  102916 22 Feb 20:13 python.png 

We already know that its size is 102916 bytes from the output of the ls command above.

Let’s open it and confirm its size in Python.

How do we get the number of bytes in the file using Python?

with open('python.png', 'rb') as png_file:
    bytes_count = 0

    while png_file.read(1):
        bytes_count += 1

print("The size of the file is: {}".format(bytes_count)) 

Here is what we have done in our code:

1. Use with open…as to open the PNG file in binary mode.
2. Read one byte at the time using a while loop until we reach the end of the file.
3. Increase the value of the bytes_count integer every time we read a byte.

The output of the program is:

$ python read_binary_file.py 
The size of the file is: 102916 

The file size calculated by our program matches the size shown by the ls command. Nice!

Using Try Finally as Equivalent of With Open…As

To give you a full understanding of the with statement I also want to show you an alternative way to write a logic that behaves like with open…as.

We will use a try…finally statement to make sure the file object is always closed after executing the try code block. I want to write something similar to the following:

>>> with open('output.txt') as f:
...     data = f.read()

Here is the code:

>>> f = open('output.txt')
>>> try:
...     data = f.read()
... finally:
...     f.close()
... 
>>> f.closed
True 

Can you see the advantage of using with open instead?

It definitely makes our code more concise. Imagine if we had to use try…finally statements for each file we open!

Conclusion

In this simple guide, we have seen how to use with open in Python to simplify the way we work with files.

Practice its syntax a few times and you will remember it without problems whenever you will need to use it in the future.

I have also written a tutorial about executing shell commands in Python and in one of the sections I show how to use the with statement to write the output of a command to a file.

Have a look at it to strengthen your Python programming core skills when working with the Operating System.

Related article: learn how to use the “with open” statement to write a Python list to a file.