Abstract

It would be nice to have a less typing-intense way of writing:

the_lock.acquire()
try:
    ....
finally:
    the_lock.release()

This PEP proposes a piece of syntax (a ‘with’ block) and a “small-i” interface that generalizes the above.

Pronouncement

This PEP is rejected in favor of PEP 343.

Rationale

One of the advantages of Python’s exception handling philosophy is that it makes it harder to do the “wrong” thing (e.g. failing to check the return value of some system call). Currently, this does not apply to resource cleanup. The current syntax for acquisition and release of a resource (for example, a lock) is:

the_lock.acquire()
try:
    ....
finally:
    the_lock.release()

This syntax separates the acquisition and release by a (possibly large) block of code, which makes it difficult to confirm “at a glance” that the code manages the resource correctly. Another common error is to code the “acquire” call within the try block, which incorrectly releases the lock if the acquire fails.

Basic Syntax and Semantics

The syntax of a ‘with’ statement is as follows:

'with' [ var '=' ] expr ':'
    suite

This statement is defined as being equivalent to the following sequence of statements:

var = expr

if hasattr(var, "__enter__"):
    var.__enter__()

try:
    suite

finally:
    var.__exit__()

(The presence of an __exit__ method is not checked like that of __enter__ to ensure that using inappropriate objects in with: statements gives an error).

If the variable is omitted, an unnamed object is allocated on the stack. In that case, the suite has no access to the unnamed object.

Possible Extensions

A number of potential extensions to the basic syntax have been discussed on the Python Developers list. None of these extensions are included in the solution proposed by this PEP. In many cases, the arguments are nearly equally strong in both directions. In such cases, the PEP has always chosen simplicity, simply because where extra power is needed, the existing try block is available.

Implementation Notes

There is a potential race condition in the code specified as equivalent to the with statement. For example, if a KeyboardInterrupt exception is raised between the completion of the __enter__ method call and the start of the try block, the __exit__ method will not be called. This can lead to resource leaks, or to deadlocks. [XXX Guido has stated that he cares about this sort of race condition, and intends to write some C magic to handle them. The implementation of the ‘with’ statement should copy this.]

Open Issues

Should existing classes (for example, file-like objects and locks) gain appropriate __enter__ and __exit__ methods? The obvious reason in favour is convenience (no adapter needed). The argument against is that if built-in files have this but (say) StringIO does not, then code that uses “with” on a file object can’t be reused with a StringIO object. So __exit__ = close becomes a part of the “file-like object” protocol, which user-defined classes may need to support.

The __enter__ hook may be unnecessary - for many use cases, an adapter class is needed and in that case, the work done by the __enter__ hook can just as easily be done in the __init__ hook.

If a way of controlling object lifetimes explicitly was available, the function of the __exit__ hook could be taken over by the existing __del__ hook. An email exchange [1] with a proponent of this approach left one of the authors even more convinced that it isn’t the right idea…

It has been suggested [2] that the “__exit__” method be called “close”, or that a “close” method should be considered if no __exit__ method is found, to increase the “out-of-the-box utility” of the “with …” construct.

There are some similarities in concept between ‘with …’ blocks and generators, which have led to proposals that for loops could implement the with block functionality [3]. While neat on some levels, we think that for loops should stick to being loops.

Alternative Ideas

IEXEC: Holger Krekel – generalised approach with XML-like syntax (no URL found…).

Holger has much more far-reaching ideas about “execution monitors” that are informed about details of control flow in the monitored block. While interesting, these ideas could change the language in deep and subtle ways and as such belong to a different PEP.

Any Smalltalk/Ruby anonymous block style extension obviously subsumes this one.

PEP 319 is in the same area, but did not win support when aired on python-dev.

Backwards Compatibility

This PEP proposes a new keyword, so the __future__ game will need to be played.

Cost of Adoption

Those who claim the language is getting larger and more complicated have something else to complain about. It’s something else to teach.

For the proposal to be useful, many file-like and lock-like classes in the standard library and other code will have to have

__exit__ = close

or similar added.

Cost of Non-Adoption

Writing correct code continues to be more effort than writing incorrect code.

References

There are various python-list and python-dev discussions that could be mentioned here.

[1]

Off-list conversation between Michael Hudson and Bill Soudan (made public with permission) http://starship.python.net/crew/mwh/pep310/

[2]

Samuele Pedroni on python-dev https://mail.python.org/pipermail/python-dev/2003-August/037795.html

[3]

Thread on python-dev with subject [Python-Dev] pre-PEP: Resource-Release Support for Generators starting at https://mail.python.org/pipermail/python-dev/2003-August/037803.html

Copyright

This document has been placed in the public domain.