Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Show only relevant benchmarks in CI summary #9

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Show only relevant benchmarks in CI summary #9

wants to merge 1 commit into from

Commits on Mar 27, 2024

  1. Show only relevant benchmarks in CI summary 

    This commit adds a new key to the benchcomp markdown table
visualization. If `delete_row_if_empty` is set to true on a table's
extra column, then for each benchmark, if the extra column evaluates to
an empty string, then the benchmark will not be displayed in the table.

The change to the CI configuration means that benchmarks whose
solver, symex, or total runtime changed by less than 10% will not be
displayed in the tables for those metrics. The number of VCCs and number
of program steps tables will also not contain benchmarks for which those
numbers didn't change, and the success table will now only show
benchmarks that are either newly passing or newly failing.
    @karkhaz
    karkhaz committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    186a13e View commit details
    Browse the repository at this point in the history