Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Misc harness improvements #8

Merged
merged 6 commits into from
May 15, 2024
Merged

Misc harness improvements #8

merged 6 commits into from
May 15, 2024

Conversation

frank-emrich
Copy link
Contributor

This PR just collects a few uncontroversial fixes and improvements of the benchmark harness:

  • For benchmarks built using make.generic.config, allow specifying additional arguments passed to make as part of the benchmark's definition (useful for defining variants of the same benchmark)
  • Use hyperfine's --comand-name option to give useful names to the shell commands it compares. Thus, instead of printing the lengthy shell commands in the Summary table produced by hyperfine, it now shows the benchmark names instead
  • Add --prepare-only flag which makes the harness perform all preparation/compilation steps, but stop before actually running benchmarks
  • Some minor cleanup about the configuration of CLI arguments
  • Actually respect the use-mimalloc CLI flag
  • Exit with non-zero exit code if there was an error
  • add cargo to the list of helper tools that we look for in $PATH and complain if they are missing

@frank-emrich frank-emrich requested a review from dhil May 15, 2024 13:53
Copy link
Member

@dhil dhil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@dhil dhil merged commit c769003 into main May 15, 2024
2 checks passed
@dhil dhil deleted the harness-improvements branch May 15, 2024 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants