Why exception name bindings are deleted while exiting the `except` block in Python, and how to break reference cycles in traceback object's stack frame local namespace

Why exception name bindings are deleted while exiting the except block?

TL;DR: To prevent the creation of reference cycles.

Traceback object attached with any exception object has a stack frame of execution containing the local, global, builtin namespaces, the code object, and other necessary entities attached. This might seem complex at first but when we follow the attribute chain to get a higher level view, it would be easier to reason about.

Let's define a simple function that will do thing but return an intentionally raised exception (which is a bad idea, i'll explain the why in a jiffy):

def foobar(num):
    spam = 'egg'
    baz = num
    try:
        raise RuntimeError
    # Bind `RuntimeError` object to name `exc`
    except RuntimeError as exc:
        return exc

The local namespace should contain three name-object mappings for names num, spam, and baz.

Now, let's run the function, get the returned exception, and follow the necessary attribute chain:

>>> exc = foobar(10)
>>> exc
RuntimeError()

# Get the traceback object
>>> exc.__traceback__
<traceback at 0x7f3adc452488>
>>> isinstance(exc.__traceback__, types.TracebackType)
True

# Get the execution frame attached to the traceback object
>>> exc.__traceback__.tb_frame
<frame at 0x2c67f98, file '<input-1109-ea03e670bfdb>', line 7, code foobar>

# Get the local namespace as seen by the execution
# frame i.e. all local variables
>>> exc.__traceback__.tb_frame.f_locals
{'num': 10, 'spam': 'egg', 'baz': 10}

Now, the problem with keeping this exception object alive by keeping a reference beyond the except block is that the attached frame object can form a reference cycle as it keeps references to the local variables (so is the function itself).

Thus, any object reachable from objects of the local namespace would be collected late even if (C)Python's own reference cycle breaking gc takes place (which will break the cycles). This will lead to a higher memory consumption (even if momentarily).

In our example, we've returned the exception object, but if we were to return something else, the name binding exc = RuntimeError (resulting from except RuntimeError as exc) would have been removed, and we would get a NameError when exc is referred outside the except block (even in function's local scope).

How to break reference cycles in traceback object's stack frame local namespace?

Even if the gc is very much capable of breaking reference cycles, we might want to make sure that the frame object is cleared after any needed operation. There are couple of ways to do that:

First way:

We can use a try-finally construct to put the frame deletion code in the finally block:

try:
    exc = foobar(10)
    # Do work with `exc` here
finally:
    del exc

Here, we've del-eted the exception object but as one can imagine deleting the frame object (exc.__traceback__.tb_frame) would do as well.

Note: A similar approach is followed by the try-except construct by default.

For example, the following code:

try:
    raise RuntimeError
except RuntimeError as e:
    print(e.args)

is basically run by the interpreter as:

try:
    raise RuntimeError
except RuntimeError as e:
    try:
        print(e.args)
    finally:
        del e

Second way:

We can use the clear method of the frame object to get the local namespace cleared i.e. clear all frame referred local variables:

# Frame referred local variables
>>> exc.__traceback__.tb_frame.f_locals
{'num': 10, 'spam': 'egg', 'baz': 10}

# Calling `clear`
>>> exc.__traceback__.tb_frame.clear()

# Empty local namespace
>>> exc.__traceback__.tb_frame.f_locals
{}

Note:

In above, I've used CPython, which is the standard/reference implementation of Python, and some topics like garbage collection is heavily implementation specific. Hence, other implementations like PyPy, IronPython, Jython (should/might) have different implementation details.

That's that! Please check the relevant docs to get a better understanding of the Python interpreter stack.

Also, check out the inspect, and gc modules:

• https://docs.python.org/3/library/inspect.html
• https://docs.python.org/3/library/gc.html