Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add exceptiongroup support to retry_if_exception #501

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

jakkdl
Copy link
Contributor

@jakkdl jakkdl commented Oct 31, 2024

This makes tenacity transparently handle any exceptions wrapped inside an ExceptionGroup, so it'll handle a ValueError() the same as ExceptionGroup("...", [ValueError()]).
If you want it to be more explicit it could be added as a parameter "allow_group".

I have not implemented it for retry_if_exception_cause_type as it gets quite messy conceptually: "retry if any of the causes of any of the raised exceptions, recursively checked both on any exceptiongroup and their containing exceptions, is of one or more types". This doesn't sound like something any sane person would ever want.

I have not implemented retry_if[_not]_exception_message. This one is less weird and I can probably just do it as "retry if the messages of all the exceptions in a group matches" and ignore the exceptiongroup message, but that's still somewhat sketchy.

When reading the tests I also found a random unused _retryable helper. I'm guessing I can just remove it?

Since #480 is not merged I also quickly added py313 to tox & CI

TODO:

  1. write changelog
  2. write docs

@Zac-HD

Copy link
Contributor

mergify bot commented Oct 31, 2024

⚠️ No release notes detected. Please make sure to use reno to add a changelog entry.

@@ -79,6 +83,9 @@ def __call__(self, retry_state: "RetryCallState") -> bool:
exception = retry_state.outcome.exception()
if exception is None:
raise RuntimeError("outcome failed but the exception is None")
if isinstance(exception, BaseExceptionGroup):
# look for any exceptions not matching the predicate
return exception.split(self.predicate)[1] is None
Copy link

@LotfiRafik LotfiRafik Feb 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means that retry will not happen if the exception group contains both the exception for which we should retry and other exceptions.. :(
i prefer to retry if the exception group contains one of the configured exception(s) :

return exception.subgroup(self.predicate) is not None

or make it parametrable ("contains", "only"..)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm. I can see arguments either way

I think current implementation is good because it requires you to be explicit about all the errors you expect, and if you suddenly start seeing new errors you'll be notified

but if one specifically wants "retry if you get TimeOutError, but I don't care if I get X, Y, Z" and you specify retry_if_exception_type((TimeOutError, X, Y, Z)) then you're also retrying if you only get X/Y/Z.

The latter does bring additional pitfalls of swallowing e.g. KeyboardInterrupt, though there's plenty stuff in this library that will already do that.

though there's always the option of just manually doing

@retry(
    retry=retry_if_exception(
        lambda e: isinstance(e, ExceptionGroup) and e.subgroup(ValueError) is not None
    )
)
def test_foo():
    ...

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree (specially with the possibility of always having the choice to set manually the retry predicate)

@jakkdl
Copy link
Contributor Author

jakkdl commented Feb 17, 2025

actually, all this messiness is kinda making me think the docs should just suggest e.g.

@retry(
    retry=retry_if_exception(
        RaisesGroup(ValueError, allow_unwrapped=True).matches
    )
)
def test_fooo() -> None:
    ...

(where RaisesGroup is either from trio or from future pytest release)

although that doesn't address @LotfiRafik's use case since RaisesGroup doesn't support variable number of exceptions. Though I don't think that's out of the question

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants