Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@akretz
Copy link
Contributor

@akretz akretz commented Oct 29, 2025

The base64 benchmarks are all broken at the moment and don't do anything. They all fail with base-common-no-such-file. This PR addresses the following:

  • In create_tmp_file(), a TempDir object is created. When the function returns, the directory is deleted on TempDir::drop(). Calling setup_test_file() fixes this, as it solves this problem by calling std::mem::forget() on the directory.
  • The decode tests attempt to decode a file which was generated previously by calling uumain(). However, that call to uumain() doesn't do what is intended. argv[0] is the program name and the format!(">{in_file_str}") argument is interpreted, as the input file name for base64. That file doesn't exist and the benchmark file doesn't get generated. I don't think a shell-like usage of > to redirect stdout works here. Perhaps it's a good idea to add a -o file to accomplish that? Similarly to how the macOS base64 command has this option.
  • The 5GB test files seem excessively large to me. The CI spends a few minutes just to generate these files. I have reduced the test files to 50MB, which makes a benchmark iteration run in ~75ms on my machine. That seems more appropriate for a benchmark to me.
  • The benchmarks should not spam stdout.

@codspeed-hq
Copy link

codspeed-hq bot commented Oct 29, 2025

CodSpeed Performance Report

Merging #9082 will degrade performances by 99.97%

Comparing akretz:fix-base64-bench (b521a90) with main (514291d)

Summary

❌ 5 regressions
✅ 100 untouched

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
b64_decode_ignore_garbage_synthetic 149.9 µs 457,771.2 µs -99.97%
b64_decode_synthetic 149 µs 458,862.8 µs -99.97%
b64_encode_synthetic 145.6 µs 75,093.6 µs -99.81%
du_human_balanced_tree[(5, 4, 10)] 10.1 ms 10.4 ms -2.96%
du_max_depth_balanced_tree[(6, 4, 10)] 32.4 ms 33 ms -2.02%

@github-actions
Copy link

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

@sylvestre
Copy link
Contributor

Could you please update the bench to be ms? Ie a bit longer

@akretz
Copy link
Contributor Author

akretz commented Oct 29, 2025

Could you please update the bench to be ms? Ie a bit longer

The decode benchmarks are each at about half a second now, in the same ballpark as most other benchmarks. Should it be even longer?

@RenjiSann
Copy link
Collaborator

The decode benchmarks are each at about half a second now, in the same ballpark as most other benchmarks. Should it be even longer?

From what I see, they're half a millisecond, which is indeed not enough

@akretz
Copy link
Contributor Author

akretz commented Nov 3, 2025

The decode benchmarks are each at about half a second now, in the same ballpark as most other benchmarks. Should it be even longer?

From what I see, they're half a millisecond, which is indeed not enough

Unless I have a serious brain fart here, 457,771.2 µs is 457.8 ms. Comma is the thousands separator across the pond.

@sylvestre
Copy link
Contributor

oh, crap, sorry

@sylvestre sylvestre merged commit 975e18c into uutils:main Nov 3, 2025
120 of 122 checks passed
@RenjiSann
Copy link
Collaborator

Ah yes indeed, my bad ^^"
It's interesting that Codspeed decides to write these in micro seconds instead of milliseconds here

sylvestre added a commit that referenced this pull request Nov 4, 2025
naoNao89 pushed a commit to naoNao89/coreutils that referenced this pull request Nov 9, 2025
* base64: fix tempdir getting removed

* base_common: add -o option to output to file

* base64: fix benchmark input files

Also pipe benchmark output to /dev/null in order to not spam stdout.

* base64: benchmark 50MB files instead of 5GB
naoNao89 pushed a commit to naoNao89/coreutils that referenced this pull request Nov 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants