-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
bench: add cksum performance benchmark #9064
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
GNU testsuite comparison: |
|
That's a good start ! Can you add bench jobs for each one of Thanks! |
|
you have to add it here too: |
- Add benchmarks for all 17 SUPPORTED_ALGORITHMS - Add cksum to CI benchmarks workflow - Use macro to reduce code duplication - Follow existing hashsum benchmark patterns Addresses feedback from PR uutils#9064
Add cksum to the CI benchmark matrix as requested in PR uutils#9064 feedback.
|
GNU testsuite comparison: |
0214b4f to
3679818
Compare
|
GNU testsuite comparison: |
|
Seems you have errors in the bench: |
3679818 to
db04f40
Compare
|
Oops, i only ran compilation tests, should have tested actual execution 🗿 |
CodSpeed Performance ReportMerging #9064 will degrade performances by 3.54%Comparing Summary
Benchmarks breakdown
Footnotes
|
|
some benchmarks are running too fast |
|
GNU testsuite comparison: |
|
GNU testsuite comparison: |
|
GNU testsuite comparison: |
|
most algorithms running in meaningful time ranges from CodSpeed:
Problem: SHAKE128/256 still running too fast (191-192 µs) |
1cfc282 to
18dfd46
Compare
|
GNU testsuite comparison: |
Add comprehensive benchmark suite for cksum utility covering: - Default CRC algorithm performance - Different file sizes (small, medium, large) - Raw output format - Multiple file processing - Stdin reading Follows established benchmark patterns from hashsum utility. Provides baseline performance metrics for future regression detection.
- Add benchmarks for all 17 SUPPORTED_ALGORITHMS - Add cksum to CI benchmarks workflow - Use macro to reduce code duplication - Follow existing hashsum benchmark patterns Addresses feedback from PR uutils#9064
SHA2, SHA3, SHAKE128, and SHAKE256 algorithms require --length parameter. Updated benchmark macro to handle algorithms with optional length parameter. Fixes benchmark runtime errors for these algorithms.
- Increase file sizes from 10MB to 50MB for actual algorithm performance measurement - Remove meaningless cksum_stdin benchmark (was just testing startup overhead) - Increase file sizes for raw_output and multiple_files benchmarks - Now benchmarks measure actual algorithm performance, not program startup - Results: cksum_default ~15ms, cksum_crc32b ~17ms (5x more meaningful than before)
- Increase file sizes from 50MB to 100MB for all algorithm benchmarks - Increase multiple_files test from 25MB to 50MB per file - Ensures all benchmarks measure algorithm performance (ms) not startup overhead (µs) - Addresses reviewer feedback about benchmarks running too fast
- Discovered fundamental issue: SHAKE algorithms require --length parameter - But cksum implementation only supports --length with blake2b, sha2, sha3 - This creates unresolvable conflict for SHAKE benchmarking - Removed SHAKE algorithms from benchmarks entirely - All remaining benchmarks now measure meaningful algorithm performance (ms) - Addresses reviewer feedback about benchmarks running too fast
- Add proper --length parameter support for SHAKE128 and SHA256 algorithms - Fix fundamental limitation where SHAKE algorithms required --length but cksum didn't support it - Add LengthRequiredForShake error type for better error messages - Implement custom SHAKE benchmarks using direct digest calculation - SHAKE benchmarks now show meaningful timing (142-540ms) instead of startup overhead - Aligns with cryptographic best practices for XOF (extensible output functions) - Follows OpenSSL approach for SHAKE algorithm implementation SHAKE benchmarks: - cksum_shake128: 142.1-526.3 ms (median 145.4ms) - cksum_shake256: 176.2-540.8 ms (median 182.8ms) All 17 algorithms now have meaningful performance measurements.
9e3fa8c to
f4dd91d
Compare
b823dbf to
4d657d7
Compare
|
Closing due to merge conflicts. New PR will be created with clean branch. |
cksum lacks performance benchmarks while other utilities have them.
My CRC32B fix in #9026 removed crc32fast dependency but needs formal performance validation to prove the replacement is better. This adds baseline testing for default CRC algorithm across different file sizes and use cases. Enables future regression detection and provides infrastructure to validate algorithm changes.