Skip to main content

Why exceptions are deleted while exiting the `except` block in Python, and breaking reference cycles in traceback object's stack frame local namespace

Why exceptions are deleted while exiting the except block

TL;DR: To avoid reference cycles.

Traceback object attached with any exception object has a stack frame of execution containing the local, global, builtin namespaces, the code object, and other necessary entities attached. This might seem complex but to get a higher level overview we can follow the attribute chain.

Let's define a simple function that will do thing but return us an intentionally raised exception (which is a bad idea that i'll explain later):

def foobar(num):
    spam = 'egg'
    baz = num
    try:
        raise RuntimeError
    except RuntimeError as exc:
        return exc

The local namespace should contain three name-object mappings for names num, spam, and baz.

Now, let's run the function, get the returned exception, and follow necessary attribute chain:

>>> exc = foobar(10)
>>> exc
RuntimeError()

# Get the traceback object
>>> exc.__traceback__
<traceback at 0x7f3adc452488>
>>> isinstance(exc.__traceback__, types.TracebackType)
True

# Get the execution frame attached to the traceback object
>>> exc.__traceback__.tb_frame
<frame at 0x2c67f98, file '<input-1109-ea03e670bfdb>', line 7, code foobar>

# Get the local namespace as seen by the execution
# frame i.e. all local variables of the function
>>> exc.__traceback__.tb_frame.f_locals
{'num': 10, 'spam': 'egg', 'baz': 10}

Now, the problem with keeping this exception object alive by keeping a reference beyond the except block is that the attached frame object can form a reference cycle. Then, any object reachable from objects of the local namespace would be collected late even if the (C)Python's own reference cycle breaking gc takes place (which will break the cycles), which obviously leads to higher memory consumption.

Breaking reference cycles in traceback object's stack frame local namespace

Even if the gc is very much capable of breaking reference cycles, we might want to make sure that the frame object is cleared after any needed operation. There are couple of ways to do that:

First way:

We can use a try-finally construct to put the frame deletion code in the finally block:

try:
    exc = foobar(10)
    # Do work here
finally:
    del exc

Here, we've del-eted the exception object but as one can imagine deleting the frame object (exc.__traceback__.tb_frame) would do as well.

A similar approach is followed by try-except by default.

For example, the following code:

try:
    raise RuntimeError
except RuntimeError as e:
    print(e.args)

is basically run the interpreter as:

try:
    raise RuntimeError
except RuntimeError as e:
    try:
        print(e.args)
    finally:
        del e

Second way:

We can use the clear method of the frame object to get the local namespaces cleared i.e. to clear all frame referred local variables:

>>> exc.__traceback__.tb_frame.f_locals
{'num': 10, 'spam': 'egg', 'baz': 10}

>>> exc.__traceback__.tb_frame.clear()

>>> exc.__traceback__.tb_frame.f_locals
{}

That's that! Please check out the relevant docs to get a better understanding of the Python interpreter stack.

  • https://docs.python.org/3/library/inspect.html
  • https://docs.python.org/3/library/gc.html

Comments

Comments powered by Disqus