dd: optimize O_DIRECT buffer alignment to reduce syscall overhead #9104
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
9f131bd to
fecebff
Compare
CodSpeed Performance ReportMerging #9104 will not alter performanceComparing Summary
Footnotes |
Implement page-aligned buffer allocation and optimize O_DIRECT flag handling to match GNU dd behavior. Key changes: - Add allocate_aligned_buffer() for page-aligned memory allocation - Update buffer allocation to use aligned buffers - Modify handle_o_direct_write() to only remove O_DIRECT for partial blocks - Add Output::write_with_o_direct_handling() for proper O_DIRECT handling - Add comprehensive unit and integration tests Fixes uutils#6078
fecebff to
2560240
Compare
…IRECT on ARM O_DIRECT requires page-aligned buffers and writes. The conv=sync flag pads output to block size, which may not be page-aligned, causing EINVAL errors on ARM systems. The core O_DIRECT functionality is already well-tested by: - test_o_direct_with_aligned_buffer_full_blocks - test_o_direct_with_partial_final_block - test_o_direct_various_block_sizes
|
GNU testsuite comparison: |
Contributor
Author
|
I need more dopamine when stuck on a bug, so new PRs might be good :)) |
|
GNU testsuite comparison: |
|
GNU testsuite comparison: |
cre4ture
reviewed
Nov 17, 2025
| /// Stub for non-Linux platforms - just return the original error. | ||
| #[cfg(not(any(target_os = "linux", target_os = "android")))] | ||
| #[allow(dead_code)] | ||
| fn handle_o_direct_write( |
Contributor
There was a problem hiding this comment.
is this stub still needed? I guess its not called an non-linux anymore, right?
e.g. there is already a stub for "write_with_o_direct_handling" which doesn't call this helper anymore
| /// This function allocates a `Vec<u8>` with proper alignment to support O_DIRECT | ||
| /// without triggering EINVAL errors. | ||
| #[cfg(any(target_os = "linux", target_os = "android"))] | ||
| fn allocate_aligned_buffer(size: usize) -> Vec<u8> { |
Contributor
There was a problem hiding this comment.
@sylvestre is this something we could move to a more central location? Or is this the only place where we need aligned memory allocations?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #6078
page-aligned buffers + smarter O_DIRECT handling. Theory says 5x fewer syscalls. 🗿
Checklist: