This is a backport of the BaseExceptionGroup and ExceptionGroup classes from Python 3.11.

Alex Grönholm

Last update: Dec 15, 2022

Related tags

Miscellaneous exceptiongroup

Overview

This is a backport of the BaseExceptionGroup and ExceptionGroup classes from Python 3.11.

It contains the following:

The exceptiongroup.BaseExceptionGroup and exceptiongroup.ExceptionGroup classes
A utility function (exceptiongroup.catch()) for catching exceptions possibly nested in an exception group
Patches to the TracebackException class that properly formats exception groups (installed on import)
An exception hook that handles formatting of exception groups through TracebackException (installed on import)

The only difference with the Python 3.11 standard library implementation is that there is no __note__ attribute in BaseExceptionGroup or ExceptionGroup.

If this package is imported on Python 3.11 or later, the built-in implementations of the exception group classes are used instead, TracebackException is not monkey patched and the exception hook won't be installed.

.. seealso:: https://docs.python.org/3/library/exceptions.html

Catching exceptions

Due to the lack of the except* syntax introduced by PEP 654 in earlier Python versions, you need to use exceptiongroup.catch() to catch exceptions that are potentially nested inside an exception group. This function returns a context manager that calls the given handler for any exceptions matching the first argument.

So, the following Python 3.11+ code:

try:
    ...
except* (ValueError, KeyError) as exc:
    print('Caught exception:', type(exc))

would be written as follows:

from exceptiongroup import catch

def handler(exc: Exception) -> None:
    print('Caught exception:', type(exc))

with catch((ValueError, KeyError), handler):
    ...

Note

Just like with except*, you cannot handle BaseExceptionGroup or ExceptionGroup with catch().

Comments

AttributeError: 'TracebackException' object has no attribute '_format_syntax_error' when running the test suite of `pylint`

When running the test suite of pylint on a system with exceptiongroup package installed, the test suite suddenly crashes with the following traceback:

tests/checkers/unittest_non_ascii_name.py::TestNonAsciiChecker::test_check_import[from_okay_module_import_bad_as_good_and_star] 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 268, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 322, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/main.py", line 347, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/flaky/flaky_pytest_plugin.py", line 94, in pytest_runtest_protocol
INTERNALERROR>     self.runner.pytest_runtest_protocol(item, nextitem)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/runner.py", line 111, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/runner.py", line 130, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/flaky/flaky_pytest_plugin.py", line 141, in call_and_report
INTERNALERROR>     report = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 55, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/skipping.py", line 265, in pytest_runtest_makereport
INTERNALERROR>     rep = outcome.get_result()
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/runner.py", line 365, in pytest_runtest_makereport
INTERNALERROR>     return TestReport.from_item_and_call(item, call)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/reports.py", line 345, in from_item_and_call
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/python.py", line 1795, in repr_failure
INTERNALERROR>     return self._repr_failure_py(excinfo, style=style)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/nodes.py", line 475, in _repr_failure_py
INTERNALERROR>     return excinfo.getrepr(
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/_code/code.py", line 666, in getrepr
INTERNALERROR>     return fmt.repr_excinfo(self)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/_code/code.py", line 926, in repr_excinfo
INTERNALERROR>     reprtraceback = self.repr_traceback(excinfo_)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/_code/code.py", line 867, in repr_traceback
INTERNALERROR>     reprentry = self.repr_traceback_entry(entry, einfo)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/_code/code.py", line 818, in repr_traceback_entry
INTERNALERROR>     s = self.get_source(source, line_index, excinfo, short=short)
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/_code/code.py", line 756, in get_source
INTERNALERROR>     lines.extend(self.get_exconly(excinfo, indent=indent, markall=True))
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/_code/code.py", line 768, in get_exconly
INTERNALERROR>     exlines = excinfo.exconly(tryshort=True).split("\n")
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/_pytest/_code/code.py", line 585, in exconly
INTERNALERROR>     lines = format_exception_only(self.type, self.value)
INTERNALERROR>   File "/usr/lib/python3.8/traceback.py", line 140, in format_exception_only
INTERNALERROR>     return list(TracebackException(etype, value, None).format_exception_only())
INTERNALERROR>   File "/usr/lib/python3.8/site-packages/exceptiongroup/_formatting.py", line 149, in traceback_exception_format_exception_only
INTERNALERROR>     yield from self._format_syntax_error(stype)
INTERNALERROR> AttributeError: 'TracebackException' object has no attribute '_format_syntax_error'

Uninstalling exceptiongroup from the system resolves the problem. I wasn't able to make a clear reproducer in a venv yet — I suspect one of the pytest plugins installed on my system may be involved.

The pytest banner is:

platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0 -- /tmp/portage/dev-python/pylint-2.14.1/work/pylint-2.14.1-python3_8/install/usr/bin/python3.8
cachedir: .pytest_cache
PySide2 5.15.2 -- Qt runtime 5.15.4 -- Qt compiled 5.15.4
metadata: {'Python': '3.8.13', 'Platform': 'Linux-5.17.12-gentoo-dist-x86_64-AMD_Ryzen_5_3600_6-Core_Processor-with-glibc2.34', 'Packages': {'pytest': '7.1.2', 'py': '1.11.0', 'pluggy': '1.0.0'}, 'Plugins': {'timeout': '2.1.0', 'pytest_param_files': '0.3.4', 'tornasync': '0.6.0.post2', 'nbval': '0.9.6', 'regressions': '2.3.1', 'mock': '3.7.0', 'django': '4.5.2', 'qt': '4.0.2', 'pylama': '8.3.8', 'pyfakefs': '4.5.6', 'console-scripts': '1.3.1', 'xdoctest': '1.0.0', 'metadata': '2.0.1', 'trio': '0.7.0', 'httpbin': '1.0.2', 'freezegun': '0.4.2', 'toolbox': '0.4', 'testinfra': '6.7.0', 'pytest_codeblocks': '0.15.0', 'localserver': '0.6.0', 'xdist': '2.5.0', 'forked': '1.4.0', 'asyncio': '0.18.3', 'aiohttp': '1.0.4', 'flaky': '3.7.0', 'datadir': '1.3.1', 'expect': '1.1.0', 'rerunfailures': '10.2', 'shutil': '1.7.0', 'virtualenv': '1.7.0', 'anyio': '3.6.1', 'skip-markers': '1.3.0', 'lazy-fixture': '0.6.3', 'betamax': '0.8.1', 'kgb': '7.0', 'services': '2.2.1', 'requests-mock': '1.9.3', 'subtesthack': '0.1.2', 'xprocess': '0.19.0', 'helpers-namespace': '2021.12.29', 'pkgcore': '0.12.11', 'typeguard': '2.13.3', 'httpx': '0.21.0', 'subtests': '0.8.0', 'xvfb': '2.0.0', 'hypothesis': '6.46.11', 'shell-utilities': '1.5.0', 'snapshottest': '0.6.0', 'plus': '0.2', 'Faker': '13.12.1'}, 'JAVA_HOME': '/etc/java-config-2/current-system-vm'}
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/tmp/portage/dev-python/pylint-2.14.1/work/pylint-2.14.1/.hypothesis/examples')
rootdir: /tmp/portage/dev-python/pylint-2.14.1/work/pylint-2.14.1, configfile: setup.cfg, testpaths: tests
plugins: timeout-2.1.0, pytest_param_files-0.3.4, tornasync-0.6.0.post2, nbval-0.9.6, regressions-2.3.1, mock-3.7.0, django-4.5.2, qt-4.0.2, pylama-8.3.8, pyfakefs-4.5.6, console-scripts-1.3.1, xdoctest-1.0.0, metadata-2.0.1, trio-0.7.0, httpbin-1.0.2, freezegun-0.4.2, toolbox-0.4, testinfra-6.7.0, pytest_codeblocks-0.15.0, localserver-0.6.0, xdist-2.5.0, forked-1.4.0, asyncio-0.18.3, aiohttp-1.0.4, flaky-3.7.0, datadir-1.3.1, expect-1.1.0, rerunfailures-10.2, shutil-1.7.0, virtualenv-1.7.0, anyio-3.6.1, skip-markers-1.3.0, lazy-fixture-0.6.3, betamax-0.8.1, kgb-7.0, services-2.2.1, requests-mock-1.9.3, subtesthack-0.1.2, xprocess-0.19.0, helpers-namespace-2021.12.29, pkgcore-0.12.11, typeguard-2.13.3, httpx-0.21.0, subtests-0.8.0, xvfb-2.0.0, hypothesis-6.46.11, shell-utilities-1.5.0, snapshottest-0.6.0, plus-0.2, Faker-13.12.1
asyncio: mode=auto

opened by mgorny 20

issue #41: backport the pure python suggestion code from 3.12

First attempt. Code is from here: https://github.com/python/cpython/commit/bbc7cd649a6ef56eb09278f3e746ca89b9d592c9

I suppose a few more tests would be good?

opened by cfbolz 16
PEP 678: new API for add_note()

Just after my last PR - and after we'd asked the Steering Council to consider the PEP - we had some translation-related concerns that ended up prompting a slight redesign. So here's another PR with the new interface!

(and I'm sorry for the churn! Thanks for maintaining the backport 💖)

opened by Zac-HD 13
Document how to print an exceptiongroup traceback when the monkey patching doesn't work / is not applied.
First, thanks for working on this project.

I updated to the latest cattr 22.1.0 from @Tinche who started to use the exceptiongroup backport to report various errors occurring while "structuring" objects. All fine.

Unfortunately, I think I have an installed package that messes with the monkeypatching of the traceback formatting so when I run my buggy object structuring which fails, I get this console output:

Traceback (most recent call last): File "debug.py", line 40, in <module> main() File "debug.py", line 34, in main pressure_rule_structured = DATA_QUALITY_CONVERTER.structure( File "/opt/python3toolbox/.venv/lib/python3.8/site-packages/cattrs/converters.py", line 281, in structure return self._structure_func.dispatch(cl)(obj, cl) File "<cattrs generated structure DataQuality.rule_objects.RangeConfig>", line 44, in structure_RangeConfig cattrs.errors.ClassValidationError: While structuring RangeConfig (4 sub-exceptions)

When I run this same code in an virtualenv with less packages, I get the traceback "trees" akin to the ones shown here: https://github.com/python-attrs/cattrs/issues/258#issue-1215069520

I tried for some time to try to force the exceptiongroup exception formatting and it resulted in different form attribute errors or the likes deep in the exceptiongroup code which makes me think there are 2 monkey patching at play.

My question: is there anyway to force the display of this exceptiongroup formatting even if someone else elsewhere monkeypatched traceback.TracebackException? All my incantations involving traceback stdlib module and exceptiongroup._formatting failed miserably.
opened by matmel 12

Bizarre error in `sys.excepthook` with recursive calls

Error in sys.excepthook:
Traceback (most recent call last):
  File "/root/.pyenv/versions/3.10.4/lib/python3.10/site-packages/exceptiongroup/_formatting.py", line 71, in exceptiongroup_excepthook
    sys.stderr.write("".join(traceback.format_exception(etype, value, tb)))
  File "/root/.pyenv/versions/3.10.4/lib/python3.10/traceback.py", line 136, in format_exception
    return list(te.format(chain=chain))
  File "/root/.pyenv/versions/3.10.4/lib/python3.10/site-packages/exceptiongroup/_formatting.py", line 206, in format
    yield from exc.exceptions[i].format(chain=chain, _ctx=_ctx)
  File "/root/.pyenv/versions/3.10.4/lib/python3.10/site-packages/exceptiongroup/_formatting.py", line 142, in format
    if exc.__cause__ is not None:
AttributeError: 'PatchedTracebackException' object has no attribute '__cause__'. Did you mean: '__class__'?

I'm pretty sure this is because the upstream __init__ method doesn't assign __cause__ for recursive calls, and so if you happen to hit this (on exceptiongroup == 1.0.0rc9) you subsequently crash with an AttributeError.

I think the obvious getattr(exc, "__cause__", None) patch is probably the best way forward; happy to write that if you'd like. Unfortunately I don't have a good reproducing case or indeed any way to reproduce this locally - it's consistently crashing only in a particular build and CI system, and at time of writing I've worked out why but not constructed a repro.

bug

opened by Zac-HD 11

Traceback broken for AttributeError with name only

Observed behavior

Running pytest on the snippet below results in an error while attempting to construct the traceback.

from collections import abc


class NamedAttributeError(AttributeError):
    def __init__(self, name: str) -> None:
        self.name: str = name


def test_exceptiongroup_error() -> None:
    raise NamedAttributeError(name="mykey")

tests/test_exceptiongroup.py 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/main.py", line 270, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/main.py", line 349, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/runner.py", line 112, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/runner.py", line 131, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/runner.py", line 222, in call_and_report
INTERNALERROR>     report: TestReport = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_callers.py", line 55, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/skipping.py", line 265, in pytest_runtest_makereport
INTERNALERROR>     rep = outcome.get_result()
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/runner.py", line 366, in pytest_runtest_makereport
INTERNALERROR>     return TestReport.from_item_and_call(item, call)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/reports.py", line 349, in from_item_and_call
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/python.py", line 1823, in repr_failure
INTERNALERROR>     return self._repr_failure_py(excinfo, style=style)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/nodes.py", line 484, in _repr_failure_py
INTERNALERROR>     return excinfo.getrepr(
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/_code/code.py", line 669, in getrepr
INTERNALERROR>     return fmt.repr_excinfo(self)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/_code/code.py", line 944, in repr_excinfo
INTERNALERROR>     reprtraceback = self.repr_traceback(excinfo_)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/_code/code.py", line 871, in repr_traceback
INTERNALERROR>     reprentry = self.repr_traceback_entry(entry, einfo)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/_code/code.py", line 822, in repr_traceback_entry
INTERNALERROR>     s = self.get_source(source, line_index, excinfo, short=short)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/_code/code.py", line 760, in get_source
INTERNALERROR>     lines.extend(self.get_exconly(excinfo, indent=indent, markall=True))
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/_code/code.py", line 772, in get_exconly
INTERNALERROR>     exlines = excinfo.exconly(tryshort=True).split("\n")
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/_pytest/_code/code.py", line 588, in exconly
INTERNALERROR>     lines = format_exception_only(self.type, self.value)
INTERNALERROR>   File "/usr/lib/python3.9/traceback.py", line 140, in format_exception_only
INTERNALERROR>     return list(TracebackException(etype, value, None).format_exception_only())
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/exceptiongroup/_formatting.py", line 125, in __init__
INTERNALERROR>     suggestion = _compute_suggestion_error(exc_value, exc_traceback)
INTERNALERROR>   File "/home/sander/.virtualenvs/sandbox-39/lib/python3.9/site-packages/exceptiongroup/_formatting.py", line 451, in _compute_suggestion_error
INTERNALERROR>     obj = exc_value.obj
INTERNALERROR> AttributeError: 'NamedAttributeError' object has no attribute 'obj'

==================================== no tests ran in 0.01s ====================================

Details

This behavior was introduced in 1.0.3. It seems to assume that if the name attribute is set, the obj attribute will be set as well. As far as I understand there are two issues with that assumption:

The Python documentation doesn't mention any invariant that either both or neither should be set. It seems valid to me to set only the name and not the object.
The name and obj attributes are only introduced in Python 3.10. In Python 3.9 and below it is perfectly valid to introduce a custom subclass that happens to have one or both of these names, which might have different meanings entirely than the ones defined in 3.10+.

The second issue could be argued to be of lesser importance (and more difficult to address). The first I would call a breaking bug.

Environment

I'm using the latest versions of pytest-7.2.0 and exceptiongroup-1.0.3 at the time of writing, but I believe older pytest versions are also susceptible to this issue.

opened by sanderr 7

At most two tracebacks displayed in case of cascading errors

I have a problem since the 1.0.0 release. Only the two first tracebacks are displayed in case of cascading errors. I'm actually not using exceptiongroup features, but since pytest is using it, it changes the normal tracebacks behavior in my tests.

EXPECTED: Here is a code with 3 nested functions producing cascading errors:

#!/usr/bin/env python

def func3():
    raise Exception("LEVEL_3")

def func2():
    try:
        func3()
    except Exception:
        raise Exception("LEVEL_2")

def func1():
    try:
        func2()
    except Exception:
        raise Exception("LEVEL_1")

if __name__ == "__main__":
    func1()

And here is the output I get:

Traceback (most recent call last):
  File "/home/felix/projects/bliss/./repro_bug.py", line 9, in func2
    func3()
  File "/home/felix/projects/bliss/./repro_bug.py", line 5, in func3
    raise Exception("LEVEL_3")
Exception: LEVEL_3

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/felix/projects/bliss/./repro_bug.py", line 15, in func1
    func2()
  File "/home/felix/projects/bliss/./repro_bug.py", line 11, in func2
    raise Exception("LEVEL_2")
Exception: LEVEL_2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/felix/projects/bliss/./repro_bug.py", line 20, in <module>
    func1()
  File "/home/felix/projects/bliss/./repro_bug.py", line 17, in func1
    raise Exception("LEVEL_1")
Exception: LEVEL_1

PROBLEM: Now if I import exceptiongroup at the beginning to monkeypatch tracebacks, the last level is lost. I can add more error levels, but I only see LEVEL_1 and LEVEL_2 at the end.

Traceback (most recent call last):
  File "/home/felix/projects/bliss/./repro_bug.py", line 21, in func1
    func2()
  File "/home/felix/projects/bliss/./repro_bug.py", line 17, in func2
    raise Exception("LEVEL_2")
Exception: LEVEL_2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/felix/projects/bliss/./repro_bug.py", line 26, in <module>
    func1()
  File "/home/felix/projects/bliss/./repro_bug.py", line 23, in func1
    raise Exception("LEVEL_1")
Exception: LEVEL_1

bug

opened by proto3 7

Initialization behavior differs from 3.11 due to init

Due to __init__ receiving the same arguments as __new__, a custom signature can break exception formatting. Additionally, the same signature can have unexpected behavior if arguments are modified before passing them to ExceptionGroup.__new__.

I'm not sure how cpython implements this, but it does work differently.

Here's the behavior with the official example in the 3.11 reference:

errors.py

from exceptiongroup import ExceptionGroup, print_exception


class Errors(ExceptionGroup):
    """
    Example from the stdlib docs:
    https://docs.python.org/3.11/library/exceptions.html#ExceptionGroup:~:text=message%20from%20it.-,class,-Errors(ExceptionGroup
    """

    def __new__(cls, errors, exit_code):
        self = super().__new__(Errors, f"exit code: {exit_code}", errors)
        self.exit_code = exit_code
        return self

    def derive(self, excs):
        return Errors(excs, self.exit_code)


print_exception(Errors([Exception("uhoh")], 2))

$ python3.11 errors.py 
  | Errors: exit code: 2 (1 sub-exception)
  +-+---------------- 1 ----------------
    | Exception: uhoh
    +------------------------------------
$ python3.9 errors.py
Traceback (most recent call last):
  File "/Users/kevin/code/python-exrg/errors.py", line 19, in <module>
    print_exception(Errors([Exception("uhoh")], 2))
  File "/opt/homebrew/Cellar/[email protected]/3.9.15/Frameworks/Python.framework/Versions/3.9/lib/python3.9/functools.py", line 888, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "/Users/kevin/Library/Python/3.9/lib/python/site-packages/exceptiongroup/_formatting.py", line 402, in print_exception
    for line in PatchedTracebackException(
  File "/Users/kevin/Library/Python/3.9/lib/python/site-packages/exceptiongroup/_formatting.py", line 186, in __init__
    for exc in e.exceptions:
  File "/Users/kevin/Library/Python/3.9/lib/python/site-packages/exceptiongroup/_exceptions.py", line 98, in exceptions
    return tuple(self._exceptions)
TypeError: 'int' object is not iterable

The second issue with the same signature:

custom.py

from exceptiongroup import ExceptionGroup, print_exception


class CustomMessage(ExceptionGroup):
    """
    Passing custom arguments to ExceptionGroup.__new__
    """

    def __new__(cls, msg: str, errors):
        count = f"there were {len(errors)} errors!"
        if msg:
            msg += f", {count}"
        else:
            msg = count

        return super().__new__(CustomMessage, msg, errors)


print_exception(CustomMessage("Oh no", [Exception("uhoh")]))

$ python3.11 custom.py 
  | CustomMessage: Oh no, there were 1 errors! (1 sub-exception)
  +-+---------------- 1 ----------------
    | Exception: uhoh
    +------------------------------------
$ python3.9 custom.py
  | CustomMessage: Oh no (1 sub-exception)
  +-+---------------- 1 ----------------
    | Exception: uhoh
    +------------------------------------

(note the message next to CustomMessage)

opened by KevinMGranger 5

exceptiongroup monkeypatching traceback.TracebackException.init breaks PyPy's Name/AttributeError suggestions

PyPy implements CPython 3.10's "Did you mean" suggestions for NameError and AttributeError. We backported this feature to 3.9, and it is implemented in normal Python code in the traceback module. In CPython, the suggestions are implemented in PyErr_Display in C, but traceback will also get an implementation in 3.12: https://github.com/python/cpython/issues/97008

At some point between versions 1.0.0rc9 and 1.0.2, the monkeypatching of exceptiongroup started breaking this feature:

$ pypy3.9
Python 3.9.12 (51afa45d7c16841bf2b7c58bc5a52b13d78d983f, Jul 14 2022, 15:06:05)
[PyPy 7.3.10-alpha0 with GCC 10.2.1 20210130 (Red Hat 10.2.1-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> dira
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'dira' is not defined. Did you mean: dir?
>>>> import exceptiongroup
>>>> dira
Traceback (most recent call last):
  File "/home/cfbolz/bin/pypy-c-jit-105829-51afa45d7c16-linux64/lib/pypy3.9/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<stdin>", line 1, in <module>
NameError: name 'dira' is not defined

Git bisect points to a0ec8cc96a351eaf6904aa4831b493b04557c407. The reason for the breackage is that PyPy's TracebackException.__init__ has another check for NameError/AttributeError, just after the special case for SyntaxError:

...
        if exc_type and issubclass(exc_type, SyntaxError):
            # Handle SyntaxError's specially
            self.filename = exc_value.filename
            lno = exc_value.lineno
            self.lineno = str(lno) if lno is not None else None
            self.text = exc_value.text
            self.offset = exc_value.offset
            self.msg = exc_value.msg
        elif exc_type and issubclass(exc_type, (NameError, AttributeError)) and \
                getattr(exc_value, "name", None) is not None:
            suggestion = _compute_suggestion_error(exc_value, exc_traceback)
            if suggestion:
                self._str += ". Did you mean: %s?" % (suggestion, )
        if lookup_lines:
            self._load_lines()

I know that the monkeypatching can be turned off, but I wasn't even using exceptiongroup myself, only hypothesis which does.

I'm open to working on a fix, but would need some pointers on how you would want to do this. Ultimately the traceback module is to blame here, in my opinion, because it does not really have any useful ways to extend its functionality without copying large parts of it outright.

/cc @mattip

opened by cfbolz 4

error when formatting HTTPError

HTTPError has some weird behavior which causes exceptiongroup to error out when rendering the traceback. There is a cpython issue at https://github.com/python/cpython/issues/98778 about HTTPError's weirdness. But it'd be great if exceptiongroup could help handle this better, so error handling works as expected, and not waiting for a cpython fix and releases.

I found this while using pytest, which uses exceptiongroup.

The setup:

$ cat test_httperror.py
from urllib.error import HTTPError

def test():
    raise HTTPError('url', 408, 'timeout', None, None)

$ pip freeze | egrep 'pytest|exceptiongroup'
exceptiongroup==1.0.1
pytest==7.2.0

The error:

$ pytest test_httperror.py
================================================================================================================================================== test session starts ==================================================================================================================================================
platform linux -- Python 3.7.10, pytest-7.2.0, pluggy-1.0.0
rootdir: /src/allura, configfile: pytest.ini
collected 1 item

test_httperror.py
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/main.py", line 270, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/main.py", line 349, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/runner.py", line 112, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/runner.py", line 131, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/runner.py", line 222, in call_and_report
INTERNALERROR>     report: TestReport = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_callers.py", line 55, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/skipping.py", line 265, in pytest_runtest_makereport
INTERNALERROR>     rep = outcome.get_result()
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/pluggy/_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/runner.py", line 366, in pytest_runtest_makereport
INTERNALERROR>     return TestReport.from_item_and_call(item, call)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/reports.py", line 349, in from_item_and_call
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/python.py", line 1823, in repr_failure
INTERNALERROR>     return self._repr_failure_py(excinfo, style=style)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/nodes.py", line 490, in _repr_failure_py
INTERNALERROR>     truncate_locals=truncate_locals,
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/_code/code.py", line 669, in getrepr
INTERNALERROR>     return fmt.repr_excinfo(self)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/_code/code.py", line 944, in repr_excinfo
INTERNALERROR>     reprtraceback = self.repr_traceback(excinfo_)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/_code/code.py", line 871, in repr_traceback
INTERNALERROR>     reprentry = self.repr_traceback_entry(entry, einfo)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/_code/code.py", line 822, in repr_traceback_entry
INTERNALERROR>     s = self.get_source(source, line_index, excinfo, short=short)
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/_code/code.py", line 760, in get_source
INTERNALERROR>     lines.extend(self.get_exconly(excinfo, indent=indent, markall=True))
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/_code/code.py", line 772, in get_exconly
INTERNALERROR>     exlines = excinfo.exconly(tryshort=True).split("\n")
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/_pytest/_code/code.py", line 588, in exconly
INTERNALERROR>     lines = format_exception_only(self.type, self.value)
INTERNALERROR>   File "/usr/local/lib/python3.7/traceback.py", line 140, in format_exception_only
INTERNALERROR>     return list(TracebackException(etype, value, None).format_exception_only())
INTERNALERROR>   File "/var/local/env-allura/lib/python3.7/site-packages/exceptiongroup/_formatting.py", line 106, in __init__
INTERNALERROR>     self.__notes__ = getattr(exc_value, "__notes__", None)
INTERNALERROR>   File "/usr/local/lib/python3.7/tempfile.py", line 475, in __getattr__
INTERNALERROR>     file = self.__dict__['file']
INTERNALERROR> KeyError: 'file'

================================================================================================================================================= no tests ran in 0.10s =================================================================================================================================================

opened by brondsem 4

Display `__note__` from monkeypatched `TracebackException` type

:wave: PEP-678 author here - I've been looking at how to port Hypothesis over to use ExceptionGroup (https://github.com/HypothesisWorks/hypothesis/pull/3191). If this backport started displaying the __note__ string if present, I'd be able and very happy to throw out all* the old print-based code entirely! I'm happy to implement this, if you'd be willing to take a PR and cut the release?

*aside from a print-based, single-exception-only fallback for if monkeypatching is disabled on Python <3.11.

opened by Zac-HD 4

Add note helper?

Just curious, I find myself using:

if sys.version_info < (3, 11):
    err.__notes__ = getattr(err, "__notes__", []) + [msg]  # type: ignore[attr-defined]
else:
    err.add_note(msg)  # pylint: disable=no-member

in order to add notes to the exceptions contained in the exception group. (See cattrs, where I got the idea from). There's already a helper function (exceptiongroup.catch()), would an add_note() helper be helpful and in scope?

This would be the definition, I think:

def add_note(err: BaseException, msg: str) -> None:
    if sys.version_info < (3, 11):
        err.__notes__ = getattr(err, "__notes__", []) + [msg]
    else:
        err.add_note(msg)

enhancement

opened by henryiii 1

exceptiongroup override of sys.excepthook changes stdlib code module behavior

stdlib code.InteractiveInterpreter has two methods that include a check of whether sys.excepthook is the same object as sys.__excepthook__.

If the exceptiongroup formatting override is applied then these conditionals take their alternate branch. For applications which overrode the write method, the error content is now sent to (probably) the real process stderr instead of handled by whatever logic the write method implements.

For example, Twisted Manhole subclasses InteractiveInterpreter and overrides write to let it send output over telnet or ssh connections.

The possible failure here went unnoticed for many years but recently I tried to add some Hypothesis-based tests to Twisted. Hypothesis apparently has a dependency on exceptiongroup and in some environments (this part is still a mystery to me!) the result is that some Twisted Manhole unit tests fail because they fail to capture the exception output because the modified sys.excepthook now causes code.InteractiveInterpreter.showtraceback to call the exceptiongroup hook instead of the instance's own write method.

opened by exarkun 4
Add exceptiongroup support to third party projects

As requested on https://github.com/python-trio/trio/pull/2213, we should try to add exceptiongroup support to both IPython and Apport. This issue tracks all efforts to add exceptiongroup support to other projects.

IPython seems to use customized versions of format_exception() and format_exception_only(), so it's likely to need an update to display exception groups, even on Python 3.11.

Apport, on the other hand, uses format_exc() and print_exc() straight from the traceback module, so a conditional import might do the trick.

opened by agronholm 6

exception information missing from unraisable exceptiongroups

https://github.com/python/cpython/issues/95572

Python 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import exceptiongroup
>>> class Foo:
...     def __del__(self):
...         raise exceptiongroup.BaseExceptionGroup("the bad", [Exception("critical debug information")])
... 
>>> f = Foo()
>>> del f
Exception ignored in: <function Foo.__del__ at 0x7f5823e49870>
Traceback (most recent call last):
  File "<stdin>", line 3, in __del__
exceptiongroup.ExceptionGroup: the bad (1 sub-exception)

opened by graingert 6

This is a backport of the BaseExceptionGroup and ExceptionGroup classes from Python 3.11.

Related tags

Overview

Catching exceptions

Comments

Observed behavior

Details

Environment

Owner

Alex Grönholm

Izy - Python functions and classes that make python even easier than it is

Macros in Python: quasiquotes, case classes, LINQ and more!

Python Classes Without Boilerplate

Simple but maybe too simple config management through python data classes. We use it for machine learning.

Python Interactive Graphical System made during Computer Graphics classes (INE5420-2021.1)

WATTS provides a set of Python classes that can manage simulation workflows for multiple codes where information is exchanged at a coarse level

Simple tooling for marking deprecated functions or classes and re-routing to the new successors' instance.

An extended version of the hotkeys demo code using action classes

Pyrmanent - Make all your classes permanent in a flash 💾

On this repo, you'll find every codes I made during my NSI classes (informatical courses)

Run python scripts and pass data between multiple python and node processes using this npm module

Data Structures and Algorithms Python - Practice data structures and algorithms in python with few small projects

Built with Python programming language and QT library and Guess the number in three easy, medium and hard rolls

Built with Python programming language and QT library and Guess the number in three easy, medium and hard rolls

Cirq is a Python library for writing, manipulating, and optimizing quantum circuits and running them against quantum computers and simulators

A simple script written using symbolic python that takes as input a desired metric and automatically calculates and outputs the Christoffel Pseudo-Tensor, Riemann Curvature Tensor, Ricci Tensor, Scalar Curvature and the Kretschmann Scalar

An awesome list of AI for art and design - resources, and popular datasets and how we may apply computer vision tasks to art and design.

Msgpack serialization/deserialization library for Python, written in Rust using PyO3 and rust-msgpack. Reboot of orjson. msgpack.org[Python]

PyPIContents is an application that generates a Module Index from the Python Package Index (PyPI) and also from various versions of the Python Standard Library.