You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I prefer to keep PR CI times reasonably brief, which is in tension with a desire for thorough tests and benchmarks. I try to target runtimes below 10 minutes and, since these benchmarks need to be run twice to evaluate both versions, this means we’d be limited to about 5 minutes of benchmarking to achieve.
In trying to balance these goals for benchmarking, I propose to add a separate benchmark script benchmarks/benchmark_extended.jl that will provide more thorough coverage and can be run locally via benchpkg with an explicit script location flag. Coverage in the standard CI script can then be a little shallower and wider.
The text was updated successfully, but these errors were encountered:
I prefer to keep PR CI times reasonably brief, which is in tension with a desire for thorough tests and benchmarks. I try to target runtimes below 10 minutes and, since these benchmarks need to be run twice to evaluate both versions, this means we’d be limited to about 5 minutes of benchmarking to achieve.
In trying to balance these goals for benchmarking, I propose to add a separate benchmark script
benchmarks/benchmark_extended.jl
that will provide more thorough coverage and can be run locally viabenchpkg
with an explicit script location flag. Coverage in the standard CI script can then be a little shallower and wider.The text was updated successfully, but these errors were encountered: