block: Remove use of raw libc::pread/pwrite & implement VHDx O_DIRECT support - #8378
Conversation
rbradford
commented
Jun 12, 2026
- block: Implement FileExt for RawFile
- block: Add AlignedBuffer
- block: qcow: Port RawFile to AlignedBuffer
- block: qcow: Port backing file support to FileExt
- block: qcow: Port to AlignedBuffer
- block: qcow: Port to FileExt
- block: Add AlignedFile
- block: qcow: Port RawFile to AlignedFile
- block: vhdx: Use AlignedFile for O_DIRECT-safe I/O
aa42c62 to
bb9e099
Compare
bb9e099 to
7f420c8
Compare
@rbradford thanks for putting in the time on this. At quick glance it looks good. There's a lot in here, and the shape gives a solid base I could rebase #8335 on top of. With the initial cleanup landing, what stays on my side is at least STATX_DIOALIGN, the loopback ext4 test, and the vhdx direct_io plumbing, plus whatever else falls out once I rebase. I'll take a closer look and follow up with a proper review. Thanks again. |
|
Ideally we would move RawFile out of QCOW and consolidate with the sync raw backend support. But let's moderate ourselves. |
weltling
left a comment
There was a problem hiding this comment.
The shape is good. AlignedBuffer is generalized and now owns the alignment bookkeeping and takes any FileExt. RawFile reduced to a position tracker on top of AlignedFile, that's a good step toward the eventual RawFile consolidation you mentioned.
Thanks!
| fn dynamic_vhdx(size_mib: u64) -> Option<TempFile> { | ||
| let tf = TempFile::new().unwrap(); | ||
| let path = tf.as_path(); | ||
| let status = Command::new("qemu-img") |
There was a problem hiding this comment.
Making qemu-img as official dependency for unit tests is IMO a good idea.
Working a lot with QCOW I've been not once tempted to switch away from the write logic there as qemu-img would have made it just simpler at many places and wolud allow to clean out some non production code.
Also given integration tests already use it as a dependency, we could later bundle some utility logic there in cloud-hypervisor/tests or alike and make it reusable across all the block unit tests.
Except also we'd like to get into the business of producing cli tools, but probably not at this stage.
Thanks
| let mut abuf = AlignedBuffer::new(offset, buf.len(), self.alignment)?; | ||
| abuf.read_from(&self.file)?; // RMW: preserve head/tail padding | ||
| abuf.as_mut_slice().copy_from_slice(buf); | ||
| abuf.write_to(&self.file)?; |
There was a problem hiding this comment.
Possible behavior change worth a look here. While the full aligned region is written, an unaligned write at or past EOF would silently extend the file with zeroes up to the next alignment boundary. The behavior is actually documented as per std, just the slight difference is the file extended to the next alignment boundary past offset + buf.len(), not just to offset. No concern, just worth a note.
Thanks
FileExt-based io has to be limited to aligned cases that use bounce buffers because it's all slice based. The sync raw backend uses iovecs to guest memory and can't manifest slices safely. Isolate to qcow, this seems like a good clean up though. |
|
Implement std::os::unix::fs::FileExt for RawFile by delegating to the inner File. This enables callers holding a reference to a RawFile to use read_exact_at and write_all_at directly for non-cursor I/O (like pread, etc) without going through custom helper functions. Assisted-by: Claude:Opus-4.6 Signed-off-by: Rob Bradford <[email protected]>
The block code repeatedly open-codes O_DIRECT alignment calculations and bounce-buffer allocation at each I/O site. Add an AlignedBuffer struct that handles the alignment and allocation in one place, using FileExt (read_exact_at and write_all_at) for the I/O (so no need for custom libc wrappers). The caller creates an AlignedBuffer with an offset, length and alignment, then uses read_from and write_to for aligned I/O and as_slice and as_mut_slice to access the logical data portion within the aligned region. This is a lot like the AlignedBuf that was already existing in the QCOW2 code but is a more generalised version. Assisted-by: Claude:Opus-4.6 Signed-off-by: Rob Bradford <[email protected]>
Replace the manual alloc_zeroed/dealloc and pread64/pwrite64 with use of the new AlignedBuffer structure. Assisted-by: Claude:Opus-4.6 Signed-off-by: Rob Bradford <[email protected]>
Replace use of raw pread64/pwrite64 functions with std::os::unix::fs::FileExt for I/O without a cursor. Assisted-by: Claude:Opus-4.6 Signed-off-by: Rob Bradford <[email protected]>
Replace the qcow specific AlignedBuf along with the pread64/pwrite64 helpers with the new common AlignedBuffer implementation. Assisted-by: Claude:Opus-4.6 Signed-off-by: Rob Bradford <[email protected]>
Replace the use of the pread64/pwrite64 helpers with versions from std::os::unix::fs::FileExt. As this was the last use of these pread functions remove them and their tests. Assisted-by: Claude:Opus-4.6 Signed-off-by: Rob Bradford <[email protected]>
Provide a single home for O_DIRECT alignment and RMW behavior behind an std::os::unix::fs::FileExt implementation built on AlignedBuffer. Unaligned requests are bounced through an AlignedBuffer (applying RMW for writes) and aligned requests pass straight through to the inner File. Assisted-by: Claude:Opus-4.8 Signed-off-by: Rob Bradford <[email protected]>
Reuse the functionality in the AlignedFile wrapper for the QCOW RawFile wrapper. This makes alignment handling more transparent. Assisted-by: Claude:Opus-4.8 Signed-off-by: Rob Bradford <[email protected]>
By redirecting VHDx I/O through the AlignedFile the required RMW semantics can be achieved for writes less than the logical block size whilt reusing the same logic used for other backend implementations. Assisted-by: Claude:Opus-4.8 Signed-off-by: Rob Bradford <[email protected]>
7f420c8 to
660344a
Compare