fix(wc):GNU wc-cpu.sh #9144

mattsu2020 · 2025-11-04T11:40:49Z

Overview

Disable SIMD-accelerated paths when GLIBC_TUNABLES removes AVX/AVX512 so wc falls back to the naive counters.
Add hidden --debug flag output that reports whether hardware acceleration is active, disabled by tunables, or unavailable at runtime.
Cache SIMD policy decisions and reuse them within the fast path code to avoid repeated environment parsing.

Testing

cargo test -p uu_wc
cargo clippy -p uu_wc -- -D warnings
Spot-check wc -l with and without GLIBC_TUNABLES='glibc.cpu.hwcaps=-AVX2,-AVX512F'

codspeed-hq · 2025-11-04T11:53:33Z

CodSpeed Performance Report

Merging #9144 will not alter performance

_{Comparing mattsu2020:wc_compatibility (62673fc) with main (b9b9655)}

Summary

✅ 127 untouched
⏩ 6 skipped¹

6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

github-actions · 2025-11-04T12:01:36Z

GNU testsuite comparison:

Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

src/uu/wc/src/cpu_features.rs

Add shared CPU hardware capability detection in uucore to prevent code duplication across utilities. This provides a unified interface for detecting CPU features (AVX512, AVX2, PCLMUL, SSE2, ASIMD) and respecting GLIBC_TUNABLES environment variable. This unblocks PR uutils#9088 (cksum --debug) and PR uutils#9144 (wc --debug) by providing a common implementation that both utilities can use. Features: - CPU feature detection with caching (singleton pattern) - GLIBC_TUNABLES parsing for hwcaps restrictions - Cross-platform support (x86/x86_64, aarch64) - Comprehensive test coverage - Zero-cost abstractions using std::arch Implementation details: - Uses std::arch feature detection (no external deps for detection) - Adds cfg-if dependency for conditional compilation - Feature-gated behind "hardware" feature flag - Android excluded (no CPUID access in sandboxed environment) Related: uutils#9088, uutils#9144 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

Add shared CPU hardware capability detection in uucore to prevent code duplication across utilities. This provides a unified interface for detecting CPU features (AVX512, AVX2, PCLMUL, SSE2, ASIMD) and respecting GLIBC_TUNABLES environment variable. This unblocks PR uutils#9088 (cksum --debug) and PR uutils#9144 (wc --debug) by providing a common implementation that both utilities can use. Features: - CPU feature detection with caching (singleton pattern) - GLIBC_TUNABLES parsing for hwcaps restrictions - Cross-platform support (x86/x86_64, aarch64) - Comprehensive test coverage - Zero-cost abstractions using std::arch Implementation details: - Uses std::arch feature detection (no external deps for detection) - Adds cfg-if dependency for conditional compilation - Feature-gated behind "hardware" feature flag - Android excluded (no CPUID access in sandboxed environment) Related: uutils#9088, uutils#9144

Add shared CPU hardware capability detection in uucore to prevent code duplication across utilities. This provides a unified interface for detecting CPU features (AVX512, AVX2, PCLMUL, SSE2, ASIMD) and respecting GLIBC_TUNABLES environment variable. This unblocks PR #9088 (cksum --debug) and PR #9144 (wc --debug) by providing a common implementation that both utilities can use. Features: - CPU feature detection with caching (singleton pattern) - GLIBC_TUNABLES parsing for hwcaps restrictions - Cross-platform support (x86/x86_64, aarch64) - Comprehensive test coverage - Zero-cost abstractions using std::arch Implementation details: - Uses std::arch feature detection (no external deps for detection) - Adds cfg-if dependency for conditional compilation - Feature-gated behind "hardware" feature flag - Android excluded (no CPUID access in sandboxed environment) Related: #9088, #9144

github-actions · 2025-11-24T12:35:13Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

github-actions · 2025-11-24T12:56:39Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

Add shared CPU hardware capability detection in uucore to prevent code duplication across utilities. This provides a unified interface for detecting CPU features (AVX512, AVX2, PCLMUL, SSE2, ASIMD) and respecting GLIBC_TUNABLES environment variable. This unblocks PR uutils#9088 (cksum --debug) and PR uutils#9144 (wc --debug) by providing a common implementation that both utilities can use. Features: - CPU feature detection with caching (singleton pattern) - GLIBC_TUNABLES parsing for hwcaps restrictions - Cross-platform support (x86/x86_64, aarch64) - Comprehensive test coverage - Zero-cost abstractions using std::arch Implementation details: - Uses std::arch feature detection (no external deps for detection) - Adds cfg-if dependency for conditional compilation - Feature-gated behind "hardware" feature flag - Android excluded (no CPUID access in sandboxed environment) Related: uutils#9088, uutils#9144

RenjiSann · 2025-11-25T13:21:37Z

Can you add a test to avoid regressions in the future ?

github-actions · 2025-11-25T14:40:09Z

GNU testsuite comparison:

Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

sylvestre · 2025-11-25T14:56:56Z

job fails with:
ERROR taplo:format_files: the file is not properly formatted path="/home/runner/work/coreutils/coreutils/src/uu/wc/Cargo.toml"

github-actions · 2025-11-25T15:09:39Z

GNU testsuite comparison:

Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

Add shared CPU hardware capability detection in uucore to prevent code duplication across utilities. This provides a unified interface for detecting CPU features (AVX512, AVX2, PCLMUL, SSE2, ASIMD) and respecting GLIBC_TUNABLES environment variable. This unblocks PR uutils#9088 (cksum --debug) and PR uutils#9144 (wc --debug) by providing a common implementation that both utilities can use. Features: - CPU feature detection with caching (singleton pattern) - GLIBC_TUNABLES parsing for hwcaps restrictions - Cross-platform support (x86/x86_64, aarch64) - Comprehensive test coverage - Zero-cost abstractions using std::arch Implementation details: - Uses std::arch feature detection (no external deps for detection) - Adds cfg-if dependency for conditional compilation - Feature-gated behind "hardware" feature flag - Android excluded (no CPUID access in sandboxed environment) Related: uutils#9088, uutils#9144

github-actions · 2025-12-01T14:44:05Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

sylvestre · 2025-12-01T15:34:57Z

src/uu/wc/src/wc.rs

+        if policy.allows_simd() {
+            let enabled = policy.enabled_features();
+            if enabled.is_empty() {
+                eprintln!("wc: debug: hardware support unavailable on this CPU");


please use the translate! macro

github-actions · 2025-12-01T23:43:53Z

GNU testsuite comparison:

Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

Add shared CPU hardware capability detection in uucore to prevent code duplication across utilities. This provides a unified interface for detecting CPU features (AVX512, AVX2, PCLMUL, SSE2, ASIMD) and respecting GLIBC_TUNABLES environment variable. This unblocks PR uutils#9088 (cksum --debug) and PR uutils#9144 (wc --debug) by providing a common implementation that both utilities can use. Features: - CPU feature detection with caching (singleton pattern) - GLIBC_TUNABLES parsing for hwcaps restrictions - Cross-platform support (x86/x86_64, aarch64) - Comprehensive test coverage - Zero-cost abstractions using std::arch Implementation details: - Uses std::arch feature detection (no external deps for detection) - Adds cfg-if dependency for conditional compilation - Feature-gated behind "hardware" feature flag - Android excluded (no CPUID access in sandboxed environment) Related: uutils#9088, uutils#9144

asder8215 · 2025-12-19T18:54:59Z

This looks good to me so far

RenjiSann · 2025-12-20T12:15:39Z

The hardware API in uucore changed, so there is no simd_policy function anymore.
There is some refactoring to do in this PR

Also, could you squash the different commits in a single one for better traceability once merged ? Thanks !

- use SimdPolicy::detect with hardware feature labeling - keep SIMD behavior respecting GLIBC_TUNABLES - consolidate wc SIMD debug output and tests

- Changed multi-line SIMD feature vector creation to a single-line expression for improved readability and consistency with surrounding code. - No functional changes; only stylistic refactoring in the wc debug logic.

github-actions · 2025-12-20T12:55:08Z

GNU testsuite comparison:

Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

RenjiSann · 2025-12-20T15:17:00Z

$ ../gnu/src/wc -l --debug /dev/null          
wc: avx512 support not detected
wc: using avx2 hardware support
0 /dev/null

$ GLIBC_TUNABLES='glibc.cpu.hwcaps=-AVX512F,-AVX2' ../gnu/src/wc -l --debug /dev/null
wc: avx512 support not detected
wc: avx2 support not detected
0 /dev/null

$ GLIBC_TUNABLES='glibc.cpu.hwcaps=-AVX512F' ../gnu/src/wc -l --debug /dev/null    
wc: avx512 support not detected
wc: using avx2 hardware support
0 /dev/null

$ GLIBC_TUNABLES='glibc.cpu.hwcaps=-AVX512F' ./target/debug/wc -l --debug /dev/null      
wc: debug: hardware support disabled by GLIBC_TUNABLES (AVX512F)
0 /dev/null

When we only disable AVX512, this is misleading because we don't mention that AVX2 is still usable

Add new localization strings and logic to provide detailed debug information when SIMD support is limited by GLIBC_TUNABLES, including lists of disabled and enabled features. Refactor SIMD allowance check for better accuracy in detecting runtime support.

github-actions · 2025-12-21T14:13:50Z

GNU testsuite comparison:

Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

sylvestre · 2025-12-22T11:44:12Z

src/uu/wc/src/wc.rs

+    }
+}
+
+fn is_wc_simd_runtime_feature(feature: &HardwareFeature) -> bool {


ould be shortened to is_runtime_feature since it's in wc context

sylvestre · 2025-12-22T11:44:17Z

src/uu/wc/src/wc.rs

+    )
+}
+
+fn is_wc_simd_debug_feature(feature: &HardwareFeature) -> bool {


sylvestre · 2025-12-22T11:45:05Z

src/uu/wc/src/wc.rs

+    )
+}
+
+fn wc_simd_enabled_features(policy: &SimdPolicy) -> Vec<HardwareFeature> {


Consider combining wc_simd_enabled_features and wc_simd_disabled_features into a single function that returns both to avoid duplicate iteration

sylvestre · 2025-12-22T11:46:17Z

src/uu/wc/src/wc.rs

+
+        let runtime_disabled = !disabled_runtime_features.is_empty();
+
+        match (enabled.is_empty(), runtime_disabled, disabled.is_empty()) {


Pattern matching on tuple with 3 bool elements is hard to read :/

sylvestre · 2025-12-22T11:47:47Z

src/uu/wc/src/wc.rs

+        .collect()
+}
+
+pub(crate) fn wc_simd_allowed(policy: &SimdPolicy) -> bool {


should we cache this ?

Since it's already cached elsewhere, it's a tough call whether to cache it here.

Refactor SIMD feature detection and reporting in the wc utility by introducing a WcSimdFeatures struct to group enabled, disabled, and runtime-disabled features. This replaces multiple separate functions with a single function, improving code organization and efficiency by reducing redundant iterations over feature lists. Also rename helper functions for clarity and update debug output logic accordingly.

github-actions · 2025-12-22T12:29:19Z

GNU testsuite comparison:

Congrats! The gnu test tests/wc/wc-cpu is no longer failing!

RenjiSann · 2025-12-22T14:12:28Z

src/uu/wc/src/wc.rs

+        if enabled_empty && !runtime_disabled {
+            eprintln!("{}", translate!("wc-debug-hw-unavailable"));
+        } else if runtime_disabled {
+            eprintln!(
+                "{}",
+                translate!("wc-debug-hw-disabled-glibc", "features" => disabled.join(", "))
+            );
+        } else if !enabled_empty && disabled_empty {
+            eprintln!(
+                "{}",
+                translate!("wc-debug-hw-using", "features" => enabled.join(", "))
+            );
+        } else {
+            eprintln!(


Suggested change

if enabled_empty && !runtime_disabled {

eprintln!("{}", translate!("wc-debug-hw-unavailable"));

} else if runtime_disabled {

eprintln!(

"{}",

translate!("wc-debug-hw-disabled-glibc", "features" => disabled.join(", "))

);

} else if !enabled_empty && disabled_empty {

eprintln!(

"{}",

translate!("wc-debug-hw-using", "features" => enabled.join(", "))

);

} else {

eprintln!(

if enabled_empty && !runtime_disabled {

show_error!("{}", translate!("wc-debug-hw-unavailable"));

} else if runtime_disabled {

show_error!(

"{}",

translate!("wc-debug-hw-disabled-glibc", "features" => disabled.join(", "))

);

} else if !enabled_empty && disabled_empty {

show_error!(

"{}",

translate!("wc-debug-hw-using", "features" => enabled.join(", "))

);

} else {

show_error!(

The show_error! macro is specifically made to prepend <bin-name>: to the message

RenjiSann · 2025-12-22T14:12:53Z

src/uu/wc/locales/en-US.ftl

+wc-debug-hw-unavailable = wc: debug: hardware support unavailable on this CPU
+wc-debug-hw-using = wc: debug: using hardware support (features: { $features })
+wc-debug-hw-disabled-env = wc: debug: hardware support disabled by environment
+wc-debug-hw-disabled-glibc = wc: debug: hardware support disabled by GLIBC_TUNABLES ({ $features })
+wc-debug-hw-limited-glibc = wc: debug: hardware support limited by GLIBC_TUNABLES (disabled: { $disabled }; enabled: { $enabled })


Suggested change

wc-debug-hw-unavailable = wc: debug: hardware support unavailable on this CPU

wc-debug-hw-using = wc: debug: using hardware support (features: { $features })

wc-debug-hw-disabled-env = wc: debug: hardware support disabled by environment

wc-debug-hw-disabled-glibc = wc: debug: hardware support disabled by GLIBC_TUNABLES ({ $features })

wc-debug-hw-limited-glibc = wc: debug: hardware support limited by GLIBC_TUNABLES (disabled: { $disabled }; enabled: { $enabled })

wc-debug-hw-unavailable = debug: hardware support unavailable on this CPU

wc-debug-hw-using = debug: using hardware support (features: { $features })

wc-debug-hw-disabled-env = debug: hardware support disabled by environment

wc-debug-hw-disabled-glibc = debug: hardware support disabled by GLIBC_TUNABLES ({ $features })

wc-debug-hw-limited-glibc = debug: hardware support limited by GLIBC_TUNABLES (disabled: { $disabled }; enabled: { $enabled })

RenjiSann · 2025-12-22T14:13:21Z

src/uu/wc/locales/fr-FR.ftl

+wc-debug-hw-unavailable = wc : debug : prise en charge matérielle indisponible sur ce CPU
+wc-debug-hw-using = wc : debug : utilisation de l'accélération matérielle (fonctions : { $features })
+wc-debug-hw-disabled-env = wc : debug : prise en charge matérielle désactivée par l'environnement
+wc-debug-hw-disabled-glibc = wc : debug : prise en charge matérielle désactivée par GLIBC_TUNABLES ({ $features })
+wc-debug-hw-limited-glibc = wc : debug : prise en charge matérielle limitée par GLIBC_TUNABLES (désactivé : { $disabled } ; activé : { $enabled })


Suggested change

wc-debug-hw-unavailable = wc : debug : prise en charge matérielle indisponible sur ce CPU

wc-debug-hw-using = wc : debug : utilisation de l'accélération matérielle (fonctions : { $features })

wc-debug-hw-disabled-env = wc : debug : prise en charge matérielle désactivée par l'environnement

wc-debug-hw-disabled-glibc = wc : debug : prise en charge matérielle désactivée par GLIBC_TUNABLES ({ $features })

wc-debug-hw-limited-glibc = wc : debug : prise en charge matérielle limitée par GLIBC_TUNABLES (désactivé : { $disabled } ; activé : { $enabled })

wc-debug-hw-unavailable = debug : prise en charge matérielle indisponible sur ce CPU

wc-debug-hw-using = debug : utilisation de l'accélération matérielle (fonctions : { $features })

wc-debug-hw-disabled-env = debug : prise en charge matérielle désactivée par l'environnement

wc-debug-hw-disabled-glibc = debug : prise en charge matérielle désactivée par GLIBC_TUNABLES ({ $features })

wc-debug-hw-limited-glibc = debug : prise en charge matérielle limitée par GLIBC_TUNABLES (désactivé : { $disabled } ; activé : { $enabled })

mattsu2020 changed the title ~~fix(wc):GNUwc-cpu.sh~~ fix(wc):GNU wc-cpu.sh Nov 4, 2025

RenjiSann reviewed Nov 4, 2025

View reviewed changes

src/uu/wc/src/cpu_features.rs Outdated Show resolved Hide resolved

RenjiSann reviewed Nov 4, 2025

View reviewed changes

src/uu/wc/src/cpu_features.rs Outdated Show resolved Hide resolved

RenjiSann mentioned this pull request Nov 4, 2025

Add --debug flag to cksum #9088

Merged

naoNao89 mentioned this pull request Nov 14, 2025

feat(uucore): add shared hardware detection module #9279

Merged

sylvestre reviewed Dec 1, 2025

View reviewed changes

ChrisDryden mentioned this pull request Dec 19, 2025

wc: implement --debug flag #9725

Open

ChrisDryden mentioned this pull request Dec 19, 2025

printenv skips environment variables with invalid UTF-8 #9702

Closed

ChrisDryden mentioned this pull request Dec 19, 2025

GNU coreutils 9.9: Detailed Test report 12/19 #9729

Open

RenjiSann force-pushed the wc_compatibility branch from 1a25344 to b655c35 Compare December 20, 2025 12:14

wc: align SIMD policy integration

5a6f263

- use SimdPolicy::detect with hardware feature labeling - keep SIMD behavior respecting GLIBC_TUNABLES - consolidate wc SIMD debug output and tests

mattsu2020 force-pushed the wc_compatibility branch from 53099cd to 5a6f263 Compare December 20, 2025 12:36

mattsu2020 and others added 2 commits December 20, 2025 21:38

Merge branch 'main' into wc_compatibility

15eacb6

refactor(wc): simplify SIMD feature collection in debug output

a61c191

- Changed multi-line SIMD feature vector creation to a single-line expression for improved readability and consistency with surrounding code. - No functional changes; only stylistic refactoring in the wc debug logic.

mattsu2020 requested review from RenjiSann and sylvestre December 22, 2025 11:38

sylvestre reviewed Dec 22, 2025

View reviewed changes

RenjiSann reviewed Dec 22, 2025

View reviewed changes


		let runtime_disabled = !disabled_runtime_features.is_empty();

		match (enabled.is_empty(), runtime_disabled, disabled.is_empty()) {

Uh oh!

fix(wc):GNU wc-cpu.sh #9144

Are you sure you want to change the base?

fix(wc):GNU wc-cpu.sh #9144

Uh oh!

Conversation

mattsu2020 commented Nov 4, 2025 • edited by RenjiSann Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Testing

Uh oh!

codspeed-hq bot commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging #9144 will not alter performance

Summary

Footnotes

Uh oh!

github-actions bot commented Nov 4, 2025

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Nov 24, 2025

Uh oh!

github-actions bot commented Nov 24, 2025

Uh oh!

RenjiSann commented Nov 25, 2025

Uh oh!

github-actions bot commented Nov 25, 2025

Uh oh!

sylvestre commented Nov 25, 2025

Uh oh!

github-actions bot commented Nov 25, 2025

Uh oh!

github-actions bot commented Dec 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 1, 2025

Uh oh!

asder8215 commented Dec 19, 2025

Uh oh!

RenjiSann commented Dec 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 20, 2025

Uh oh!

RenjiSann commented Dec 20, 2025

Uh oh!

github-actions bot commented Dec 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 22, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

mattsu2020 commented Nov 4, 2025 •

edited by RenjiSann

Loading

codspeed-hq bot commented Nov 4, 2025 •

edited

Loading

RenjiSann commented Dec 20, 2025 •

edited

Loading