Abstract

Exceptions like AttributeError, IndexError, KeyError, LookupError, NameError, TypeError, and ValueError do not provide all information required by programmers to debug and better understand what caused them. Furthermore, in some cases the messages even have slightly different formats, which makes it really difficult for tools to automatically provide additional information to diagnose the problem. To tackle the former and to lay ground for the latter, it is proposed to expand these exceptions so to hold both the offending and affected entities.

Rationale

The main issue this PEP aims to solve is the fact that currently error messages are not that expressive and lack some key information to resolve the exceptions. Additionally, the information present on the error message is not always in the same format, which makes it very difficult for third-party libraries to provide automated diagnosis of the error.

These automated tools could, for example, detect typos or display or log extra debug information. These could be particularly useful when running tests or in a long running application.

Although it is in theory possible to have such libraries, they need to resort to hacks in order to achieve the goal. One such example is python-improved-exceptions [1], which modifies the byte-code to keep references to the possibly interesting objects and also parses the error messages to extract information like types or names. Unfortunately, such approach is extremely fragile and not portable.

A similar proposal [2] has been implemented for ImportError and in the same fashion this idea has received support [3]. Additionally, almost 10 years ago Guido asked in [11] to have a clean API to access the affected objects in Exceptions like KeyError, AttributeError, NameError, and IndexError. Similar issues and proposals ideas have been written in the last year. Some other issues have been created, but despite receiving support they finally get abandoned. References to the created issues are listed below:

• AttributeError: [11], [10], [5], [4], [3]
• IndexError: [11], [6], [3]
• KeyError: [11], [7], [3]
• LookupError: [11]
• NameError: [11], [10], [3]
• TypeError: [8]
• ValueError: [9]

To move forward with the development and to centralize the information and discussion, this PEP aims to be a meta-issue summarizing all the above discussions and ideas.

Examples

a = [1, 2, 3, 4, 5]
a[5]
IndexError: list index out of range

b = {'foo': 1}
b['fo']
KeyError: 'fo'

c = object()
c.foo
AttributeError: 'object' object has no attribute 'foo'

import string
string.foo
AttributeError: 'module' object has no attribute 'foo'

a = string.Formatter()
a.foo
AttributeError: 'Formatter' object has no attribute 'foo'

foo = 1
fo
NameError: global name 'fo' is not defined

a[b[c[0]]]

Proposal

Extend the exceptions AttributeError, IndexError, KeyError, LookupError, NameError, TypeError, and ValueError with the following:

• AttributeError: target ^w, attribute
• IndexError: target ^w, key ^w, index (just an alias to key)
• KeyError: target ^w, key ^w
• LookupError: target ^w, key ^w
• NameError: name, scope?
• TypeError: unexpected_type
• ValueError: unexpected_value ^w

Attributes with the superscript ^w may need to be weak references [12] to prevent any memory cycles. However, this may add an unnecessary extra complexity as noted by R. David Murray [13]. This is specially true given that builtin types do not support being weak referenced.

TODO(skreft): expand this with examples of corner cases.

To remain backwards compatible these new attributes will be optional and keyword only.

It is proposed to add this information, rather than just improve the error, as the former would allow new debugging frameworks and tools and also in the future to switch to a lazy generated message. Generated messages are discussed in [2], although they are not implemented at the moment. They would not only save some resources, but also uniform the messages.

The stdlib will be then gradually changed so to start using these new attributes.

Potential Uses

An automated tool could for example search for similar keys within the object, allowing to display the following::

a = {'foo': 1}
a['fo']
KeyError: 'fo'. Did you mean 'foo'?

foo = 1
fo
NameError: global name 'fo' is not defined. Did you mean 'foo'?

See [3] for the output a TestRunner could display.

Performance

Filling these new attributes would only require two extra parameters with data already available so the impact should be marginal. However, it may need special care for KeyError as the following pattern is already widespread.

try:
  a[foo] = a[foo] + 1
except:
  a[foo] = 0

Note as well that storing these objects into the error itself would allow the lazy generation of the error message, as discussed in [2].

References

[1]

Python Exceptions Improved (https://www.github.com/sk-/python-exceptions-improved)

[2] (1, 2, 3)

ImportError needs attributes for module and file name (http://bugs.python.org/issue1559549)

[3] (1, 2, 3, 4, 5, 6)

Enhance exceptions by attaching some more information to them (https://mail.python.org/pipermail/python-ideas/2014-February/025601.html)

[4]

Specificity in AttributeError (https://mail.python.org/pipermail/python-ideas/2013-April/020308.html)

[5]

Add an ‘attr’ attribute to AttributeError (http://bugs.python.org/issue18156)

[6]

Add index attribute to IndexError (http://bugs.python.org/issue18162)

[7]

Add a ‘key’ attribute to KeyError (http://bugs.python.org/issue18163)

[8]

Add ‘unexpected_type’ to TypeError (http://bugs.python.org/issue18165)

[9]

‘value’ attribute for ValueError (http://bugs.python.org/issue18166)

[10] (1, 2)

making builtin exceptions more informative (http://bugs.python.org/issue1182143)

[11] (1, 2, 3, 4, 5, 6)

LookupError etc. need API to get the key (http://bugs.python.org/issue614557)

[12]

weakref - Weak References (https://docs.python.org/3/library/weakref.html)

[13]

Message by R. David Murray: Weak refs on exceptions? (http://bugs.python.org/issue18163#msg190791)

Copyright

This document has been placed in the public domain.