Abstract

Exception objects are typically initialized with a message that describes the error which has occurred. Because further information may be available when the exception is caught and re-raised, or included in an ExceptionGroup, this PEP proposes to add BaseException.add_note(note), a .__notes__ attribute holding a list of notes so added, and to update the builtin traceback formatting code to include notes in the formatted traceback following the exception string.

This is particularly useful in relation to PEP 654 ExceptionGroups, which make previous workarounds ineffective or confusing. Use cases have been identified in the standard library, Hypothesis and cattrs packages, and common code patterns with retries.

Motivation

When an exception is created in order to be raised, it is usually initialized with information that describes the error that has occurred. There are cases where it is useful to add information after the exception was caught. For example,

• testing libraries may wish to show the values involved in a failing assertion, or the steps to reproduce a failure (e.g. pytest and hypothesis; example below).
• code which retries an operation on error may wish to associate an iteration, timestamp, or other explanation with each of several errors - especially if re-raising them in an ExceptionGroup.
• programming environments for novices can provide more detailed descriptions of various errors, and tips for resolving them.

Existing approaches must pass this additional information around while keeping it in sync with the state of raised, and potentially caught or chained, exceptions. This is already error-prone, and made more difficult by PEP 654 ExceptionGroups, so the time is right for a built-in solution. We therefore propose to add:

• a new method BaseException.add_note(note: str),
• BaseException.__notes__, a list of note strings added using .add_note(), and
• support in the builtin traceback formatting code such that notes are displayed in the formatted traceback following the exception string.

>>> try:
...     raise TypeError('bad type')
... except Exception as e:
...     e.add_note('Add some information')
...     raise
...
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
TypeError: bad type
Add some information
>>>

from hypothesis import given, strategies as st, target

@given(st.integers())
def test(x):
    assert x < 0
    assert x > 0


+ Exception Group Traceback (most recent call last):
|   File "test.py", line 4, in test
|     def test(x):
|
|   File "hypothesis/core.py", line 1202, in wrapped_test
|     raise the_error_hypothesis_found
|     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| ExceptionGroup: Hypothesis found 2 distinct failures.
+-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "test.py", line 6, in test
    |     assert x > 0
    |     ^^^^^^^^^^^^
    | AssertionError: assert -1 > 0
    |
    | Falsifying example: test(
    |     x=-1,
    | )
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    |   File "test.py", line 5, in test
    |     assert x < 0
    |     ^^^^^^^^^^^^
    | AssertionError: assert 0 < 0
    |
    | Falsifying example: test(
    |     x=0,
    | )
    +------------------------------------

Specification

BaseException gains a new method .add_note(note: str). If note is a string, .add_note(note) appends it to the __notes__ list, creating the attribute if it does not already exist. If note is not a string, .add_note() raises TypeError.

Libraries may clear existing notes by modifying or deleting the __notes__ list, if it has been created, including clearing all notes with del err.__notes__. This allows full control over the attached notes, without overly complicating the API or adding multiple names to BaseException.__dict__.

When an exception is displayed by the interpreter’s builtin traceback-rendering code, its notes (if there are any) appear immediately after the exception message, in the order in which they were added, with each note starting on a new line.

If __notes__ has been created, BaseExceptionGroup.subgroup and BaseExceptionGroup.split create a new list for each new instance, containing the same contents as the original exception group’s __notes__.

We do not specify the expected behaviour when users have assigned a non-list value to __notes__, or a list which contains non-string elements. Implementations might choose to emit warnings, discard or ignore bad values, convert them to strings, raise an exception, or do something else entirely.

Backwards Compatibility

System-defined or “dunder” names (following the pattern __*__) are part of the language specification, with unassigned names reserved for future use and subject to breakage without warning. We are also unaware of any code which would be broken by adding __notes__.

We were also unable to find any code which would be broken by the addition of BaseException.add_note(): while searching Google and GitHub finds several definitions of an .add_note() method, none of them are on a subclass of BaseException.

How to Teach This

The add_note() method and __notes__ attribute will be documented as part of the language standard, and explained as part of the “Errors and Exceptions” tutorial.

Reference Implementation

Following discussions related to PEP 654 [2], an early version of this proposal was implemented in and released in CPython 3.11.0a3, with a mutable string-or-none __note__ attribute.

CPython PR #31317 implements .add_note() and __notes__.

Rejected Ideas

class Explanation(Exception):
    def __str__(self):
        return "\n" + str(self)

try:
    raise AssertionError("Failed!")
except Exception as e:
    raise Explanation("You can reproduce this error by ...") from e

$ python example.py
Traceback (most recent call last):
File "example.py", line 6, in <module>
    raise AssertionError(why)
AssertionError: Failed!
                                                    # These lines are
The above exception was the direct cause of ...     # confusing for new
                                                    # users, and they
Traceback (most recent call last):                  # only exist due
File "example.py", line 8, in <module>              # to implementation
    raise Explanation(msg) from e                   # constraints :-(
Explanation:                                        # Hence this PEP!
You can reproduce this error by ...

@contextlib.contextmanager
def add_exc_note(note: str):
    try:
        yield
    except Exception as err:
        err.add_note(note)
        raise

with add_exc_note(f"While attempting to frobnicate {item=}"):
    frobnicate_or_raise(item)

Acknowledgements

We wish to thank the many people who have assisted us through conversation, code review, design advice, and implementation: Adam Turner, Alex Grönholm, André Roberge, Barry Warsaw, Brett Cannon, CAM Gerlach, Carol Willing, Damian, Erlend Aasland, Etienne Pot, Gregory Smith, Guido van Rossum, Irit Katriel, Jelle Zijlstra, Ken Jin, Kumar Aditya, Mark Shannon, Matti Picus, Petr Viktorin, Will McGugan, and pseudonymous commenters on Discord and Reddit.

References

[1]

this principle was established in the 2003 mail thread which led to PEP 3134, and included a proposal for a group-of-exceptions type! https://mail.python.org/pipermail/python-dev/2003-January/032492.html

[2]

particularly those at https://bugs.python.org/issue45607, https://discuss.python.org/t/accepting-pep-654-exception-groups-and-except/10813/9, https://github.com/python/cpython/pull/28569#discussion_r721768348, and

[3]

We note that the exceptiongroup backport package maintains an exception hook and monkeypatch for TracebackException for Pythons older than 3.11, and encourage library authors to avoid creating additional and incompatible backports. We also reiterate our preference for builtin support which makes such measures unnecessary.

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.