Profile-Guided Optimization (PGO) results for NativeDB

[zamazan4ik]

Hi!

Recently I checked Profile-Guided Optimization (PGO) improvements on multiple projects. The results are available here. According to the tests, PGO can help with achieving better performance in many cases including databases like PostgreSQL, SQLite, ClickHouse, and many others. For databases, the results can be checked here. Since all of these, I think trying to optimize NativeDB with PGO can be a good idea.

I already did some benchmarks and want to share my results.

Test environment

• Fedora 39
• Linux kernel 6.6.9
• AMD Ryzen 9 5900x
• 48 Gib RAM
• SSD Samsung 980 Pro 2 Tib
• Compiler - Rustc 1.75
• NativeDB version: the latest for now from the main branch on commit b8e156ef782828cc6992f499f0596f05ed6c2946
• Disabled Turbo boost (for more stable results across benchmark runs)

Benchmark

For benchmark purposes, I use this benchmark with cargo bench command. For PGO optimization I use cargo-pgo tool. The same benchmark suite was used for the PGO training phase built with cargo pgo bench. PGO optimized results I got with cargo pgo optimize bench.

All measurements are done multiple times to check reproducibility - the results are stable across runs.

Results

I got the following results:

• Release: https://gist.github.com/zamazan4ik/4c71703f32ca9d84aacc46a4ff262a16
• PGO optimized compared to Release: https://gist.github.com/zamazan4ik/a5289ec3fbcd50e8438fa33de85a186a
• (just for reference) PGO instrumentation compared to Release: https://gist.github.com/zamazan4ik/c096aaeb906a4d3a47103132de094196

At least according to the results above, PGO helps with achieving better overall performance with NativeDB.

Further steps

I can suggest the following action points:

• Perform more PGO benchmarks on NativeDB. If it shows improvements - add a note to the documentation about possible improvements in NativeDB performance with PGO. So NativeDB users will be aware of PGO effects on NativeDB performance and can decide to enable PGO for their NativeDB-based applications to achieve better database performance.

Testing Post-Link Optimization techniques (like LLVM BOLT) would be interesting too (Clang and Rustc already use BOLT as an addition to PGO) but I recommend starting from the usual PGO.

I would be happy to answer all your questions about PGO.