Drop operation and seek_iai benches
#388
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The `operations` bench is no longer useful, yet pulls in polars, a huge dependency. Back when we did not yet have a full reasoner implemented, verifying correctness of our joins by comparing with the polars implementation was useful. This is now covered by, among other things, our end-to-end tests. Polars is not only a huge dependency taking a lot of time to compile, but also breaks its API on minor version bumps (which is not technically a semver violation, since it's still 0.x, but requires changes to the bench code). Furthermore, the bench uses internal Nemo APIs and thus also frequently requires changes, which is often only discovered in CI, since the bench takes much too long to run regularly. Lastly, it's not pratically useable for external people, since it depends on test data that is not freely redistributable.
`iai` is unmaintained and breaks with `valgrind-3.21`.
mmarx
added
enhancement
Flake lock file updates:
• Updated input 'nixpkgs':
'github:NixOS/nixpkgs/898cb2064b6e98b8c5499f37e81adbdf2925f7c5' (2023-10-13)
→ 'github:NixOS/nixpkgs/5550a85a087c04ddcace7f892b0bdc9d8bb080c8' (2023-10-21)
• Updated input 'nixpkgs-unstable':
'github:NixOS/nixpkgs/ca012a02bf8327be9e488546faecae5e05d7d749' (2023-10-16)
→ 'github:NixOS/nixpkgs/7c9cc5a6e5d38010801741ac830a3f8fd667a7a0' (2023-10-19)
• Updated input 'rust-overlay':
'github:oxalica/rust-overlay/a2ccfb2134622b28668a274e403ba6f075ae1223' (2023-10-18)
→ 'github:oxalica/rust-overlay/a3e829c06eadf848f13d109c7648570ce37ebccd' (2023-10-22)
haha, how long has it been since someone ran that? |
matzemathics
approved these changes
Oct 23, 2023
too long, but that was basically my fault for not uncompressing the file. In any case, two hours is way too long for any single bench. |
|
Finally that should be rectified when #311 is implemented. Until then we might as well drop this chunk of code, when nobody ever runs it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
iaiis unmaintained and breaks withvalgrind-3.21. Theseek_iaibench exercises the same code as theseekbench.operationsbench pulls in polars, which is a huge dependency that regularly breaks its API on (minor) version bumps. Furthermore, the bench itself uses internal parts from thenemo-physicalAPI and thus also regularly requires changes, often only caught in CI, since the bench runs impractically long to actually be useful (still not finished after an hour on Wille – edit: failed after 1:56:00 sincetest-files/bench-data/aux-split/aux-0.csvwas not available), and the exercised code is now covered by the end-to-end tests. It also does not help that non-redistributable test data is required for the bench to successfully execute.