"with" Statement for Context Manager

Python Tutorials - Herong's Tutorial Examples

∟"with" Statement for Context Manager

This section provides a quick introduction of 'with' statement, which wraps a standard 'try' statement with a context manager object.

What Is "with" Statement? A "with" statement is a compound statement that wraps a standard "try" statement with a context manager object. "with" statements were introduced in 2006 in Python 2.5.

A context manager object implements the following methods:

object.__enter__() - Performs any custom work and returns a handle object to be used by the sub-statement block. It is called when the "with" statement is entered.
object.__exit__() - Performs any custom work for both cases: no exception and exception occurred. It is called when the "with" statement is ended.

Here is the syntax of "with" statements:

with EXPRESSION as TARGET:
    SUITE

Logically, a "with" statement written in the above syntax is equivalent to the following code using a "try" statement:

manager = (EXPRESSION)
enter = type(manager).__enter__
exit = type(manager).__exit__
value = enter(manager)
hit_except = False

try:
    TARGET = value
    SUITE
except:
    hit_except = True
    if not exit(manager, *sys.exc_info()):
        raise
finally:
    if not hit_except:
        exit(manager, None, None, None)

The code below shows the different between a context manager object and non-context manager object:

# file object is a context manager object
>>> with open('sample.txt', 'r') as file:
...     data = file.read(80)
...
>>> data
'12345678901234567890\n'
>>> type(file)
< class '_io.TextIOWrapper'>

# range object is not a context manager object
>>> with range(0, 10) as sequence:
...     slice = sequence[2:8]
AttributeError: __enter__

To see the difference between using the file object in an assignment statement and in a "with" statement, I wrote the following Python sample code, with_test.py.

#  with_test.py
#- Copyright 2011 (c) HerongYang.com. All Rights Reserved.
#
import sys
mode = sys.argv[1]
path = sys.argv[2]

file = None
data = b''

if mode == 'assignment':
   print("Testing the assignment statement")
   file = open(path, 'r')
   data = file.read(80)

elif mode == 'with':
   print("Testing the 'with' statement")
   with open(path, 'r') as file:
      data = file.read(80)

else:
   print("Invalid mode: "+mode)

print("Data = "+data.decode())
print("File = "+str(file))

Test 1 - "No such file" Exception: The 'with' statement also behaves the same way as the assignment statement as I expected.

herong$ python with_test.py assignment junk
Testing the assignment statement
Traceback (most recent call last):
  File "with_test.py", line 11, in ...
    file = open(path, 'r')
IOError: [Errno 2] No such file or directory: 'junk'

herong$ python with_test.py with junk
Testing the 'with' statement
Traceback (most recent call last):
  File "with_test.py", line 16, in ...
    with open(path, 'r') as file:
IOError: [Errno 2] No such file or directory: 'junk'

This confirms that the (EXPRESSION) is evaluated before entering the context manager's internal "try" statement. So the "with" statement provides no help to manage exceptions occurred during the evaluation of (EXPRESSION).

Test 2 - No Exception: The output shows that the 'with' statement did an extra job to set the file object in "closed" status.

herong$ python with_test.py assignment sample.txt
Testing the assignment statement
Data = some text
File = <open file 'sample.txt', mode 'r' at 0x10ce625d0>

herong$ python with_test.py with sample.txt
Testing the 'with' statement
Data = some text
File = <closed file 'sample.txt', mode 'r' at 0x10aba75d0>

Test 3 - File Read Exception: I am not able to find any easy way to trigger a file read exception to test the behavior of the "with" statement. But according to the documentation, it will also close the file and throw the exception.

So the context manager interface of the "file" object helps to close the "file", no matter what happens to the sub-statement block. In other words, the "file" object wraps a "try" statement inside to reduce your coding effort:

# using "with" statement
with open(path, 'r') as file:
  data = file.read(80)
  ...

# using assignment statement
file = open(path, 'r')
try:
   data = file.read(80)
   ...
finally:
   file.close()