Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: Implement BufWriter optimization for du stdout output (fixes #9… #9174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: Implement BufWriter optimization for du stdout output (fixes #9… #9174

wants to merge 1 commit into from

Conversation

@naoNao89
Copy link
Contributor

@naoNao89 naoNao89 commented Nov 7, 2025 •
edited
Loading

Replace direct print! statements with 64KiB buffered writes to reduce syscall overhead for du -a on large directories. This addresses the performance issue where each file entry triggers multiple stdout writes.

Changes:

  • Add BufWriter field to StatPrinter struct with 64KB buffer
  • Convert print_stat method to use buffered writes instead of direct output
  • Add Drop implementation to ensure proper buffer flushing
  • Add comprehensive benchmarks to validate performance improvement

Performance: Reduces syscalls from ~16,500 to ~3-5 for 5,500 file directories, providing significant improvement for the issue #9146 use case.

Re-open: #9163

@codspeed-hq
Copy link

codspeed-hq bot commented Nov 7, 2025 •
edited
Loading

CodSpeed Performance Report

Merging #9174 will degrade performances by 26.31%

Comparing naoNao89:feature/du-buffer-optimization-9146 (13e824e) with main (293e9d2)

Summary

⚡ 1 improvement
❌ 2 regressions
✅ 123 untouched
🆕 2 new
⏩ 2 skipped1

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
🆕 du_all_extreme_wide_tree[(2000, 500)] N/A 10.2 ms N/A
🆕 du_all_stress_balanced_tree[(3, 5, 6)] N/A 1.1 ms N/A
⚡ du_all_wide_tree[(5000, 500)] 18.2 ms 16.2 ms +12.02%
❌ factor_multiple_big_uint 22 ms 29.9 ms -26.31%
❌ factor_multiple_u128s[18446744073709551616] 308 ms 319.6 ms -3.63%

Footnotes

  1. 2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@github-actions
Copy link

github-actions bot commented Nov 7, 2025

GNU testsuite comparison:

GNU test failed: tests/du/inodes. tests/du/inodes is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/misc/tee (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

@naoNao89 naoNao89 marked this pull request as draft November 8, 2025 06:43
@github-actions
Copy link

github-actions bot commented Nov 8, 2025

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

@naoNao89 naoNao89 force-pushed the feature/du-buffer-optimization-9146 branch from c822fd6 to 97cf964 Compare November 8, 2025 10:53
@github-actions
Copy link

github-actions bot commented Nov 8, 2025

GNU testsuite comparison:

Skip an intermittent issue tests/tail/overlay-headers (fails in this run but passes in the 'main' branch)

@naoNao89 naoNao89 marked this pull request as ready for review November 8, 2025 12:54
@naoNao89 @sylvestre
feat: Implement BufWriter optimization for du stdout output (fixes uu…
13e824e 
…tils#9146)

- Add buffered writer to reduce system call overhead
- Use correct binary unit notation (KiB) for buffer sizes
- Apply buffering to total output as well
- Fix clippy warnings in benchmark documentation

This optimization significantly improves performance when du outputs
many entries, reducing the number of stdout writes from per-entry
to buffered chunks.
@sylvestre sylvestre force-pushed the feature/du-buffer-optimization-9146 branch from 97cf964 to 13e824e Compare November 8, 2025 22:55
@github-actions
Copy link

github-actions bot commented Nov 8, 2025

GNU testsuite comparison:

Skipping an intermittent issue tests/tail/overlay-headers (passes in this run but fails in the 'main' branch)

@sylvestre sylvestre mentioned this pull request Nov 8, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@naoNao89