Support async
methods in stateful testing
#4107
Labels
interop
how to play nicely with other packages
new-feature
entirely novel capabilities or strategies
@given()
doesn't natively support async functions, but thanks to #1343 that can be handled downstream e.g. in pytest-trio. However, we don't have a good solution for stateful testing with async methods, and it's harder to add downstream - for example,hypothesis-trio
copy-pasted all ofstateful.py
to add a fewawait
s, but is now several years of bugfixes and performance improvements behind.I'd like to allow
RuleBasedStateMachine
s to define a mix of sync and async methods for rules, invariants, and teardown.Instead of a firm design, some notes on options:
We're not aiming to run multiple rules concurrently, just support concurrency within each rule. This might mean we can
[asyncio/anyio/trio].run()
each async method separately, which greatly simplifies the implementation.__execute_fn(self, fn, *args, /, **kwargs)
; this can default toreturn fn(*args, **kwargs)
with the obvious more-complicated options to support async frameworks (branching oninspect.iscoroutinefunction(fn)
). We can ship first-party subclasses{AnyIO,Asyncio,Trio}RuleBasedStateMachine
with the appropriate implementations.On the other hand, if we need to maintain some (async) state over the whole run, that gets more complicated.
@initialize()
to recognize methods wrapped with@contextlib.(async)contextmanager
and handle them appropriately?stateful_async_backend
setting.await whatever_rule()
returns, it should be finished. No idea what we could do about that though, beyond documenting that it's not supported.trio.testing.wait_all_*
functions might be useful to let things settle down between rules.we should think about the
deadline
setting: async stateful testing makes it pretty natural to cancel tests that exceed the grace period; and with that in mind we could do this in sync tests that call back into Hypothesis interfaces too - potentially a pretty nice speedup, independent of the rest of this issue.Thoughts?
The text was updated successfully, but these errors were encountered: