Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a separate benchmark script for extended local tests #125

Open
mikeingold opened this issue Oct 27, 2024 · 0 comments
Open

Add a separate benchmark script for extended local tests #125

mikeingold opened this issue Oct 27, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request testing

Comments

@mikeingold
Copy link
Collaborator

I prefer to keep PR CI times reasonably brief, which is in tension with a desire for thorough tests and benchmarks. I try to target runtimes below 10 minutes and, since these benchmarks need to be run twice to evaluate both versions, this means we’d be limited to about 5 minutes of benchmarking to achieve.

In trying to balance these goals for benchmarking, I propose to add a separate benchmark script benchmarks/benchmark_extended.jl that will provide more thorough coverage and can be run locally via benchpkg with an explicit script location flag. Coverage in the standard CI script can then be a little shallower and wider.

@mikeingold mikeingold self-assigned this Oct 27, 2024
@mikeingold mikeingold added testing enhancement New feature or request labels Oct 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request testing
Projects
None yet
Development

No branches or pull requests

1 participant