Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@naoNao89
Copy link
Contributor

cksum lacks performance benchmarks while other utilities have them.

My CRC32B fix in #9026 removed crc32fast dependency but needs formal performance validation to prove the replacement is better. This adds baseline testing for default CRC algorithm across different file sizes and use cases. Enables future regression detection and provides infrastructure to validate algorithm changes.

@github-actions
Copy link

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

@RenjiSann
Copy link
Collaborator

That's a good start !

Can you add bench jobs for each one of SUPPORTED_ALGORITHMS ? The current benchmarks only test the default CRC algorithm.

Thanks!

@sylvestre
Copy link
Contributor

you have to add it here too:
https://github.com/uutils/coreutils/blob/main/.github/workflows/benchmarks.yml#L26

naoNao89 added a commit to naoNao89/coreutils that referenced this pull request Oct 28, 2025
- Add benchmarks for all 17 SUPPORTED_ALGORITHMS
- Add cksum to CI benchmarks workflow
- Use macro to reduce code duplication
- Follow existing hashsum benchmark patterns

Addresses feedback from PR uutils#9064
naoNao89 added a commit to naoNao89/coreutils that referenced this pull request Oct 28, 2025
Add cksum to the CI benchmark matrix as requested in PR uutils#9064 feedback.
@github-actions
Copy link

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

@naoNao89 naoNao89 force-pushed the bench/cksum-performance branch from 0214b4f to 3679818 Compare October 28, 2025 15:10
@github-actions
Copy link

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

@sylvestre
Copy link
Contributor

Seems you have errors in the bench:

  Collected 1 benchmark suite(s) to run
  Running uu_cksum cksum_bench
  cksum_bench: --length required for 'SHAKE256'
  cksum_bench: --length required for 'SHAKE128'
  cksum_bench: --length required for 'SHA3'
  cksum_bench: --length required for 'SHA2'
  Done running cksum_bench

@naoNao89 naoNao89 force-pushed the bench/cksum-performance branch from 3679818 to db04f40 Compare October 28, 2025 16:05
@naoNao89
Copy link
Contributor Author

Oops, i only ran compilation tests, should have tested actual execution 🗿

@codspeed-hq
Copy link

codspeed-hq bot commented Oct 28, 2025

CodSpeed Performance Report

Merging #9064 will degrade performances by 3.54%

Comparing naoNao89:bench/cksum-performance (7539782) with main (ca6d69a)

Summary

❌ 2 regressions
✅ 102 untouched
🆕 18 new
🗄️ 1 archived benchmark run1

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
🆕 cksum_blake2b N/A 190.5 ms N/A
🆕 cksum_blake3 N/A 95.8 ms N/A
🆕 cksum_bsd N/A 128.2 ms N/A
🆕 cksum_crc N/A 42.4 ms N/A
🆕 cksum_crc32b N/A 15.9 ms N/A
🆕 cksum_default N/A 42.4 ms N/A
🆕 cksum_md5 N/A 317.9 ms N/A
🆕 cksum_multiple_files N/A 63.6 ms N/A
🆕 cksum_raw_output N/A 42.4 ms N/A
🆕 cksum_sha1 N/A 723 ms N/A
🆕 cksum_sha2 N/A 193 µs N/A
🆕 cksum_sha224 N/A 1.7 s N/A
🆕 cksum_sha256 N/A 1.7 s N/A
🆕 cksum_sha3 N/A 192.5 µs N/A
🆕 cksum_sha384 N/A 1.1 s N/A
🆕 cksum_sha512 N/A 1.1 s N/A
🆕 cksum_sm3 N/A 1.6 s N/A
🆕 cksum_sysv N/A 48.1 ms N/A
du_balanced_tree[(5, 4, 10)] 9.1 ms 9.3 ms -2.61%
du_human_balanced_tree[(5, 4, 10)] 10.1 ms 10.5 ms -3.54%

Footnotes

  1. 1 benchmark was run, but is now archived. If it was deleted in another branch, consider rebasing to remove it from the report. Instead if it was added back, click here to restore it.

@sylvestre
Copy link
Contributor

some benchmarks are running too fast
example: cksum_stdin 212.2 µs
it should be ms
otherwise, we are just benchmarking the init of the program

@github-actions
Copy link

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

@github-actions
Copy link

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/cksum/cksum-c is no longer failing!
Congrats! The gnu test tests/cksum/cksum-sha3 is no longer failing!

@github-actions
Copy link

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)

@naoNao89
Copy link
Contributor Author

most algorithms running in meaningful time ranges from CodSpeed:

  • cksum_default: 42.4 ms
  • cksum_crc32b: 15.9 ms
  • cksum_sha1: 723 ms

Problem: SHAKE128/256 still running too fast (191-192 µs)

@naoNao89 naoNao89 force-pushed the bench/cksum-performance branch from 1cfc282 to 18dfd46 Compare October 29, 2025 07:13
@naoNao89 naoNao89 marked this pull request as draft October 29, 2025 07:22
@github-actions
Copy link

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/cksum/cksum-c is no longer failing!
Congrats! The gnu test tests/cksum/cksum-sha3 is no longer failing!

RenjiSann and others added 9 commits October 29, 2025 14:39
Add comprehensive benchmark suite for cksum utility covering:
- Default CRC algorithm performance
- Different file sizes (small, medium, large)
- Raw output format
- Multiple file processing
- Stdin reading

Follows established benchmark patterns from hashsum utility.
Provides baseline performance metrics for future regression detection.
- Add benchmarks for all 17 SUPPORTED_ALGORITHMS
- Add cksum to CI benchmarks workflow
- Use macro to reduce code duplication
- Follow existing hashsum benchmark patterns

Addresses feedback from PR uutils#9064
SHA2, SHA3, SHAKE128, and SHAKE256 algorithms require --length parameter.
Updated benchmark macro to handle algorithms with optional length parameter.
Fixes benchmark runtime errors for these algorithms.
- Increase file sizes from 10MB to 50MB for actual algorithm performance measurement
- Remove meaningless cksum_stdin benchmark (was just testing startup overhead)
- Increase file sizes for raw_output and multiple_files benchmarks
- Now benchmarks measure actual algorithm performance, not program startup
- Results: cksum_default ~15ms, cksum_crc32b ~17ms (5x more meaningful than before)
- Increase file sizes from 50MB to 100MB for all algorithm benchmarks
- Increase multiple_files test from 25MB to 50MB per file
- Ensures all benchmarks measure algorithm performance (ms) not startup overhead (µs)
- Addresses reviewer feedback about benchmarks running too fast
- Discovered fundamental issue: SHAKE algorithms require --length parameter
- But cksum implementation only supports --length with blake2b, sha2, sha3
- This creates unresolvable conflict for SHAKE benchmarking
- Removed SHAKE algorithms from benchmarks entirely
- All remaining benchmarks now measure meaningful algorithm performance (ms)
- Addresses reviewer feedback about benchmarks running too fast
- Add proper --length parameter support for SHAKE128 and SHA256 algorithms
- Fix fundamental limitation where SHAKE algorithms required --length but cksum didn't support it
- Add LengthRequiredForShake error type for better error messages
- Implement custom SHAKE benchmarks using direct digest calculation
- SHAKE benchmarks now show meaningful timing (142-540ms) instead of startup overhead
- Aligns with cryptographic best practices for XOF (extensible output functions)
- Follows OpenSSL approach for SHAKE algorithm implementation

SHAKE benchmarks:
- cksum_shake128: 142.1-526.3 ms (median 145.4ms)
- cksum_shake256: 176.2-540.8 ms (median 182.8ms)

All 17 algorithms now have meaningful performance measurements.
@naoNao89 naoNao89 force-pushed the bench/cksum-performance branch from 9e3fa8c to f4dd91d Compare October 29, 2025 07:39
@naoNao89 naoNao89 force-pushed the bench/cksum-performance branch from b823dbf to 4d657d7 Compare October 29, 2025 08:00
@naoNao89
Copy link
Contributor Author

Closing due to merge conflicts. New PR will be created with clean branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants