Codestin Search App

weltling · 2026-06-04T07:47:13Z

Context

Cloud Hypervisor lets a guest disk be opened with direct=on, which asks the host to use O_DIRECT. The kernel then refuses any I/O that is not aligned to what the underlying storage requires. On a host that demands a 4K boundary, the passthrough backends previously assumed 512 bytes and returned EINVAL on the first unaligned guest I/O, making direct=on unusable. Fixed VHD could not even be opened, because reading its trailing footer sector is unaligned by construction.

What this series does

This builds on #8378, which already routes qcow2 and VHDX through AlignedFile. The series makes the raw file implementation the one place that understands O_DIRECT. AlignedFile owns the alignment, probed once from the kernel at open time, and bounces any unaligned access through a shared AlignedBuffer. RawFile wraps it, and every format layers on top, so raw, qcow2, fixed VHD, and VHDX all reach the disk through the same handler and agree on one alignment value.

VHDX keeps using AlignedFile directly rather than going through RawFile, since it does positional I/O and needs no cursor or position tracking. The formats that rely on a seeking read and write cursor go through RawFile, which adds that on top of the same handler.

The raw workers keep the fast iovec submission for aligned requests and fall back to a synchronous bounce otherwise, so there is no separate async path for the unaligned case. qcow stops carrying its own alignment and bounce buffers and defers to the shared handler, taking an extra copy in exchange for a single implementation.

Fixed VHD additionally advertises the host topology to the guest and reads its footer sector with O_DIRECT cleared so it can be opened on a 4K sector filesystem.

Tests

A new integration test exercises direct I/O data disks on a 4096 byte sector loop filesystem end to end, running an aligned dd round trip inside the guest. Wrappers cover raw, qcow2, fixed VHD, and vhdx.

Compatibility

The direct=off path is unchanged everywhere and default backend alignment stays at the historical sector size.

Fixes: #8050

Assisted-by: Claude:Opus-4.8

rbradford

Honestly - i'm kinda completely lost in all the complexity and it makes it hard for me to do a good job as maintainer. As the contributor of the change i'm sure (hopefully) that you fully understand the code you're submitting but we do have to consider the future maintenance overhead.

I thought we only supported direct=on for raw files - if not, would that make things simpler?

rbradford · 2026-06-04T10:18:29Z

    }

    /// Write owned operation data cluster-by-cluster with COW from backing file.
+    #[allow(clippy::too_many_arguments)]


I think we should use #[expect(clippy::too_many_arguments)] here so if there was ever a refactoring that reduces it will get flagged up.

Fixed, thanks.

rbradford · 2026-06-04T10:19:07Z

        backing_file: &Option<Arc<dyn BackingRead>>,
-        alignment: usize,
+        alignment: u64,
+        dio: bool,


I think it would be helpful to be consistent and use the same variable name everywhere - so this would be direct ?

Gotcha, consistently naming it direct now in all backends`.

Thanks

rbradford · 2026-06-04T10:20:37Z

+/// accounting for the filesystem, underlying block device, and any stacking
+/// such as loop or device mapper. Returns [`SECTOR_SIZE`] on any failure or
+/// when the kernel does not report a value.
+pub(crate) fn query_dio_alignment(fd: RawFd) -> u64 {


Again - it would be helpful to maintain the same terminology - makes grepping and understand the code easier. I know query_direct_alignment() is not quite as pretty but it doesn't use different terminology to the rest of the codebase.

Yep, did the renames here , too - probe_direct_alignment, query_direct_alignment and so on.

Thanks

rbradford · 2026-06-04T10:21:01Z

+        let stx = unsafe { stx.assume_init() };
+        if stx.stx_mask & STATX_DIOALIGN != 0 && stx.stx_dio_mem_align > 0 {
+            let align = cmp::max(stx.stx_dio_mem_align, stx.stx_dio_offset_align) as u64;
+            debug!("statx(STATX_DIOALIGN) returned alignment {align}");


Stray debug?

Oh yes, fixed!

rbradford · 2026-06-04T10:44:55Z

+    /// Returns the alignment in bytes that the backend requires for
+    /// offset, length, and buffer addresses.
+    ///
+    /// The default of [`SECTOR_SIZE`] reflects buffered I/O backends
+    /// that have no alignment requirement beyond the virtio sector size.
+    /// Backends that require larger alignment (typically O_DIRECT on a
+    /// 4 KiB sector device) override this to report the probed value.
+    fn alignment(&self) -> u64 {
+        SECTOR_SIZE
+    }
+
+    /// Reads exactly `buf.len()` bytes at `offset`.
+    ///
+    /// The caller is responsible for satisfying any alignment requirement
+    /// reported by [`BlockBackend::alignment`].
+    fn pread_at(&mut self, offset: u64, buf: &mut [u8]) -> io::Result<()> {
+        self.seek(SeekFrom::Start(offset))?;
+        self.read_exact(buf)
+    }
+
+    /// Writes all of `buf` at `offset`.
+    ///
+    /// The caller is responsible for satisfying any alignment requirement
+    /// reported by [`BlockBackend::alignment`].
+    fn pwrite_at(&mut self, offset: u64, buf: &[u8]) -> io::Result<()> {
+        self.seek(SeekFrom::Start(offset))?;
+        self.write_all(buf)
+    }


I'm a little confused - these trait methods are entirely unused?

Good catch. These started as a uniform positional I/O entry on the trait, the idea being that callers go through pread_at/pwrite_at and the backend decides whether to honor the alignment contract internally. In the final shape that decision moved one level down into the per backend dispatch_op path, so the trait methods never gained a caller. I dropped them, the trait keeps only alignment() as the new addition.

Thanks!

rbradford · 2026-06-04T10:48:21Z

+fn pwrite_once(fd: RawFd, buf: &[u8], offset: u64) -> io::Result<usize> {
+    let off = i64::try_from(offset).map_err(|e| io::Error::new(io::ErrorKind::InvalidInput, e))?;
+    // SAFETY: buf is a valid slice for buf.len() bytes.
+    let n = unsafe { libc::pwrite64(fd, buf.as_ptr().cast(), buf.len(), off) };
+    if n < 0 {
+        return Err(io::Error::last_os_error());
+    }
+    Ok(n as usize)
+}


Do all the call sites for this handle the short writes correctly? Why not a pwrite_all to handle that?

Fair point, I've reshaped to pwrite_all that loops until the kernel accepts the full buffer, returning WriteZero on a zero return. Also added EINTR retry to both pwrite_all and pread_full for symmetry.

Thanks!

weltling · 2026-06-04T14:37:04Z

Honestly - i'm kinda completely lost in all the complexity and it makes it hard for me to do a good job as maintainer. As the contributor of the change i'm sure (hopefully) that you fully understand the code you're submitting but we do have to consider the future maintenance overhead.

I thought we only supported direct=on for raw files - if not, would that make things simpler?

Thanks for taking the time to look. The size of the diff is indeed somewhat more than I had initially expected when articulating the linked issue. Let me explain how it landed like this.

First, direct=on is not RAW only today. The config parser accepts it and forwards to all image types. QCOW already supports it on the synchronous data path through its own AlignedBuf, aligned_pread, aligned_pwrite helpers, but QcowAsync is not wired up. At the same time RAW with direct=on is broken outright on storage that requires 4k alignment. Fixed VHD is practically the same as RAW with some extra footer handling on top. Every backend also comes in a sync and an async flavor, which adds to the surface.

Quick summary of where the series lands.

The primary goal is direct=on actually working in the underlying formats
The chosen implementation looks like the simplest one
Existing QCOW implementation is simplified and extended
RAW is fixed, and Fixed VHD profits for free
VHDX is explicitly excluded
The same aligned IO building blocks back all the format backends
The path with direct=off is practically unaffected

On complexity - I did start with a theoretically cleaner idea. A wrapper around any backend with no need to touch the backend itself. As mentioned in the description that turned out to not fit our current architecture very well.

On maintenance overhead, the shape is one shared RMW helper and every backend dispatches through it. With that in place each backend change is a small mechanical step - probe alignment and route unaligned ops through the helper. The bulk of the line count is tests plus moving around existing code like BlockBackend and QCOW. The new machinery is the smaller part, and supporting async direct=on is of course complex on its own. However, narrowing to RAW only would reduce the patch on paper but would leave divergent code in QCOW.

If there is a specific part that reads as more complex than it has to be, point me at it and I will either simplify it or explain why it has to be that shape.

I will address the inline comments next.

Thanks!

rbradford

Okay - please can you go through and review your PR.

I think a lot of this complexity comes from trying to handle unaligned O_DIRECT reads/writes asynchronously. Can't we just fallback to the synchronous path - the Linux kernel won't be generating those requests and it doesn't matter if the firmware boot is a bit slower.

rbradford · 2026-06-05T08:37:00Z

@@ -0,0 +1,27 @@
+// Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.


Your commit message doesn't say why it needs to be moved.

Initially it was intended to carry more methods. Now the necessity is probably not there, thus I've dropped the extraction, just adding alignment() in place.

Thanks

rbradford · 2026-06-05T08:38:05Z

+    /// that have no alignment requirement beyond the virtio sector size.
+    /// Backends that require larger alignment (typically O_DIRECT on a
+    /// 4 KiB sector device) override this to report the probed value.
+    fn alignment(&self) -> u64 {


This is a lot of extra text here - it's pretty obvious what a method called alignment() is trying to do. Is it AI generated - the commit is not labelled as such.

Comments trimmed. Thanks

rbradford · 2026-06-05T08:39:35Z

+    /// that have no alignment requirement beyond the virtio sector size.
+    /// Backends that require larger alignment (typically O_DIRECT on a
+    /// 4 KiB sector device) override this to report the probed value.
+    fn alignment(&self) -> u64 {


The commit message adding is talking about hypothetical stuff that was removed from a previous version of this PR.

Yep, went through all comments and fixed stale ones. Thanks

rbradford · 2026-06-05T08:40:54Z

+//
+// SPDX-License-Identifier: Apache-2.0
+
+//! Rounding math for centralized alignment handling.


CH uses en_GB. But also try to avoid terms that are different between the languages.

Gotcha. I hope I caught all those cases. PR description redone accordingly. Will keep this in mind.

Thanks

rbradford · 2026-06-05T08:41:55Z

+
+/// Returns `offset` rounded down to a multiple of `alignment`.
+///
+/// Panics in debug builds if `alignment` is zero or not a power of two.


I'm going to stop the review here - please go through and trim the excessive comments whether they be AI generated or not.

Cleaned that up now. Thanks

rbradford · 2026-06-05T08:43:19Z

+/// logical sector size. Backends still probe the actual requirement
+/// at open time. This constant is the documented fallback and the
+/// value used by tests or code that needs a compile time constant.
+pub const DEFAULT_DIRECT_IO_ALIGNMENT: u64 = 4096;


Why do we want this fallback. If O_DIRECT probe fails then we should just fail as we can't be sure what the block size needs to be.

It was actually used in tests only, I removed it now.

Thanks

rbradford · 2026-06-05T08:43:48Z

+//
+// SPDX-License-Identifier: Apache-2.0
+
+pub mod core;


I think core is really unhelpful module name just put these functions in in the mod.rs.

Done, thanks.

rbradford · 2026-06-05T09:14:32Z

I do appreciate the time you've put into this PR but this is 33 commits and so very time consuming to review and I do want us to make sure we consider "do we really need this" (e.g. async unaligned O_DIRECT - when is that really going to happen?)

Although unit tests are good - I don't think we need unit tests to check round_up/round_down are correct. Please also consider combining unit tests into the commit that introduces the code.

weltling · 2026-06-05T10:04:35Z

@rbradford Thanks muchly for the second pass.

ACK on the async unaligned path. The async tracker and the synthetic completion plumbing exist only to keep unaligned IO on the io_uring/AIO submission path. I will drop the async unaligned machinery and have the async backends fall back to the shared sync RMW helper for the unaligned case. One implementation, no async variant.

On the comments - I used AI to refine them and it went wordy. I will go through the series and cut those.

Thanks!

rbradford · 2026-06-05T10:27:52Z

+use crate::{Error, SECTOR_SIZE};
+
+/// Common interface for all block backends.
+pub trait BlockBackend: Read + Write + Seek + Send + Debug {


Does this still need Read + Write + Seek? Also do we actually need this trait - it's never passed around as dyn BlockBackend ?

Because e.g. FixedVhd has impl Read that I don't think ever actually gets called.

Fair point, the Read + Write + Seek shape is worth revisiting. I can plan a cleanup for that.

Thanks

weltling · 2026-06-05T22:06:58Z

Thanks for the review, @rbradford. Quick summary of the main points in addition to inline responses.

Series shape is down to 23 commits. Unit test commits are folded into the commits that introduce the code, and the trivial round_up / round_down tests are removed. Async unaligned O_DIRECT now goes through the same dispatch_op as sync, with the synthetic completion injected into the existing data IO.

Walked every commit and trimmed verbose comments, polished commit messages, fixed stale references and mentions of work that no longer exists, checked the conformance between locales.

On the Read + Write + Seek shape, fair point, worth revisiting. I would keep this PR scoped to the DIO fix and do the trait cleanup as a follow up.

Thanks

rbradford · 2026-06-08T15:47:23Z

-        if alignment == 0
-            && mappings.len() == 1
+        // single io_uring readv. Misaligned O_DIRECT reads are handled by
+        // dispatch_op in submit_data_operation, so DIO does not


I asked you to remove references to DIO.

Fair point, thanks for spelling it out. I had read the earlier note as scoped to identifiers, e.g. dio -> direct, query_dio_alignment -> query_direct_alignment. The same extends to comments and commit messages. I went over all the prose now, too.

Thanks!

rbradford · 2026-06-08T15:47:47Z

+    /// I/O alignment exposed to the block layer, at least SECTOR_SIZE.
+    /// When `direct` is true the same value is also the O_DIRECT alignment.
+    alignment: u64,
+    /// True when the underlying data file was opened with O_DIRECT.


I don't think we need this comment.

Removed, thanks.

rbradford · 2026-06-08T15:48:32Z

    ret == 0 && stat.st_mode & S_IFMT == S_IFBLK
 }

+/// Query the O_DIRECT memory and offset alignment requirement for an fd.


This commit message still says DIO.

Fixed, thanks.

rbradford · 2026-06-08T15:48:41Z

+/// Uses `statx(STATX_DIOALIGN)` (Linux >= 6.1) to obtain the exact memory and
+/// offset alignment the kernel requires for direct I/O on this specific fd.
+/// Unlike `fstatvfs().f_bsize`, which only returns the filesystem's preferred
+/// I/O block size, `STATX_DIOALIGN` reports the true per fd DIO constraint


Ditto here.

Fixed, thanks.

rbradford · 2026-06-08T15:49:27Z

+pub fn round_down(offset: u64, alignment: u64) -> u64 {
+    debug_assert!(alignment.is_power_of_two() && alignment > 0);
+    offset & !(alignment - 1)
+}
+
+/// Returns `offset` rounded up to `alignment`, or `None` on overflow.
+pub fn round_up(offset: u64, alignment: u64) -> Option<u64> {
+    debug_assert!(alignment.is_power_of_two() && alignment > 0);
+    let mask = alignment - 1;
+    offset.checked_add(mask).map(|v| v & !mask)
+}


Are these actually used outside of aligned_range?

Nope, they were not used elsewhere. I just inlined them.

Thanks!

rbradford · 2026-06-08T15:50:19Z

+}
+
+/// Write all of `buf` to `fd` at `offset`, retrying on short writes.
+fn pwrite_all(fd: RawFd, buf: &[u8], offset: u64) -> io::Result<()> {


_all ? I think _all is more consistent with the Rust std?

We also already have a bunch of pread/pwrite calls in block/ - maybe we should consolidate on a single set of wrappers in a preparatory commit and use that everywhere.

Yes, _all matches std write_all_at.

I've added a small prep commit that promotes pwrite_all to a shared crate::pwrite_all and drops qcow implementation.

The other pread/pwrite sites are different shapes:

libc preadv/pwritev, vectored, in raw sync worker

libc pread64, retry loop in qcow pread_exact, plus std read_at, read_exact_at, write_all_at in qcow

std read_exact and write_all, cursor based, in request.rs, vhd, vhdx, lib.rs tests

Folding any of those in needs separate thought, I would rather take it as follow up work.

Thanks

We can't have multiple functions doing the same thing - that's a maintenance nightmare. We should either decide to do everything cursor based or send everything through preadv/pwritev with _all/_exact variants building on that.

Yeah, probably time to iron out the wrinkles here. Positional looks like the best fit. It matches what every backend already calls under the hood and aligns with std FileExt. Cursor based shapes need Seek, which forces single threaded fd use and blocks the per queue worker layout raw and qcow have today.

Thanks

I filed #8353 to track this. Seems a little too big for this scope, though, what would you say?

Thanks

rbradford · 2026-06-08T15:50:57Z

+use crate::async_io::{AsyncIoOperation, OwnedIoBuffer};
+
+/// Read `buf.len()` bytes from `fd` at `offset`, zero filling past EOF.
+fn pread_full(fd: RawFd, buf: &mut [u8], offset: u64) -> io::Result<usize> {


What's the difference between _full and .. - also does this retry on short reads?

Renamed to pread_padded since _full might have been ambiguous. The helper loops on short reads, retries EINTR, and zero pads past EOF so the caller always sees a fully populated buffer. The closest std analog is read_exact_at, but that errors on EOF rather than padding, which is the opposite of what RMW needs here. _padded keeps the EOF tail zero fill explicit in the name.

Thanks

DemiMarie · 2026-06-08T19:40:32Z

the Linux kernel won't be generating those requests

Do you know if other kernels also won’t be? I’m thinking Windows and FreeBSD.

DemiMarie · 2026-06-08T19:41:34Z

Did this work fine with the Linux kernel block device implementation?

weltling · 2026-06-08T23:29:02Z

Did this work fine with the Linux kernel block device implementation?

Covered end to end by a test on a /dev/loopN with 4096 byte logical block size and ext4 on top.

weltling · 2026-06-08T23:46:57Z

@rbradford all comments addressed. PTAL when you get a chance. Thanks.

rbradford · 2026-06-16T22:04:19Z

#8378 merged now so maybe you want to revisit this?

weltling · 2026-06-16T22:34:49Z

#8378 merged now so maybe you want to revisit this?

Thanks, on it. Building on that, the work comes down mostly to the backend alignment parts and the relevant testing.

rbradford

Thanks for updating - I still find this very confusing. I think we have too many abstractions that are almost similar.

I think we need the layering to be simplified - VHD/VHDX/QEMU should all layer atop a synchronous or asynchronous raw file implementation that handles O_DIRECT on the underlying filesystem. It doesn't matter if those upper implementations have extra copies as frankly if you're using those non-raw implementations you have no expectation of performance. We need to compromise performance for simplicity and maintainability as right now this code is really hard to reason about.

rbradford · 2026-06-17T12:22:39Z

+    // SAFETY: fcntl(F_GETFL) is always safe on a valid fd.
+    let flags = unsafe { libc::fcntl(fd, libc::F_GETFL) };
+    if flags >= 0 && (flags & libc::O_DIRECT) != 0 {
+        Some(query_direct_alignment(fd))


I think the other method could be folded in here - why do we need this layer?

Folded. There is now a single probe_direct_alignment that returns Option<u64>.

Thanks

rbradford · 2026-06-17T12:23:45Z

-            }
-        }
+        let alignment = if direct_io {
+            query_direct_alignment(file.as_raw_fd()) as usize


Having a method called probe_direct_alignment and one called query_direct_alignment is pretty confusing.

Collapsed into one as per above. Thanks

weltling · 2026-06-19T17:27:49Z

Could we either a.) Replace all the cursor based stuff that forces through RawFile so it can just depend on AlignedFile allowing RawFile to be dropped or b.) Add the cursor interface to AlignedFile allowing RawFile to be dropped

Thanks for the review, and good that we are converging on a direction. Option b) looks the most promising to me, so I will explore that one. It also reduces the level of abstraction further, which is the goal here.

Thanks!

weltling · 2026-06-19T17:38:54Z

Looking better I think - but it unnerves me that this touches 25 files - this subsystem needs flattening and simplifying. Also I don't see any changes to vhdx?

On the file count, most of it is breadth. Each backend opens its own file with separate sync and async workers, so O_DIRECT touches many of them by nature. Option b will not change that much, but it drops a layer.

On VHDX, it already layers on AlignedFile from #8378, so there is nothing to change there. That is why the series does not touch it.

Thanks

DemiMarie

There are some incorrect comments. 2 of the 3 already existed. Only 1 is new.

I very much agree that the subsystem needs to be seriously simplified. It’s far too complex.

DemiMarie · 2026-06-19T21:17:50Z

-                    offset,
-                )
-            }
+            // Unaligned under O_DIRECT: fall back to the synchronous path.


This code is already synchronous.

Thanks for checking through. Yep, it's a sync worker. This comment is gone in the updated version. Thanks

DemiMarie · 2026-06-19T21:18:18Z

+                // SAFETY: the memory pointed to by `iovecs` is backed by the op,
+                // and valid for the kernel to write to by construction of
+                // AsyncIoOperation.


There is no async I/O operation involved here, so the comment is wrong. (Preexisting issue)

That's correct, but the text refers to the typename AsyncIoOperation, not async behavior. The sync worker still operates on an AsyncIoOperation value.

Thanks

DemiMarie · 2026-06-19T21:18:36Z

+                // SAFETY: the memory pointed to by `iovecs` is backed by the op,
+                // and valid for the kernel to read from by construction of
+                // AsyncIoOperation.


Same wrong comment.

Same here, the reference is the type invariant, not async execution.

Thanks

weltling · 2026-06-19T22:14:00Z

Option b) implemented now, RawFile is gone.

Thanks

rbradford

Looks great - thank you for your patience on the iterations. Let's try and keep reducing code where we can.

rbradford · 2026-06-20T08:53:16Z

+
+/// True when `op` satisfies `alignment` and can go straight to the kernel.
+pub(crate) fn operation_is_aligned(op: &AsyncIoOperation, alignment: u64) -> bool {
+    if alignment <= 1 {


Elsewhere we check == 0

Unified to == 0 for consistency. Thanks

Now you've resolved this i'm going to land this PR.

Move the statx STATX_DIOALIGN probe out of DiskTopology into a free probe_direct_alignment helper keyed on a raw fd. The helper gates on the O_DIRECT open flag and returns the kernel reported alignment only when direct I/O is in effect, and None otherwise. DiskTopology::probe keeps the same call path and result. AlignedFile::new now determines its O_DIRECT block alignment from probe_direct_alignment instead of trial reads at 512 and 4096, falling back to SECTOR_SIZE when the kernel does not report a value. This matches how the raw and fixed VHD workers determine alignment, so all backends agree on one source of truth. Signed-off-by: Anatol Belski <[email protected]>

RawFile wrapped AlignedFile only to add a seek position and the file trait impls that the qcow and vhost_user_block code expects. Fold that position and every impl onto AlignedFile so the wrapper layer goes away and callers work with a single O_DIRECT aware file type. AlignedFile now tracks a cursor and implements Read, Write, Seek, WriteZeroesAt, PunchHole, FileSync, SeekHole, BlockBackend, Clone, AsRawFd and AsFd in addition to the positional FileExt path. The direct_io flag is dropped because alignment already encodes it, where a zero alignment means the file was not opened with O_DIRECT. All RawFile uses in the qcow internals and vhost_user_block move to AlignedFile, and raw_file.rs is removed. Signed-off-by: Anatol Belski <[email protected]>

The raw sync, io_uring and AIO workers now own an AlignedFile and use it for the O_DIRECT alignment value and for the unaligned fallback. Aligned operations keep the fast preadv and pwritev iovec path straight to the kernel. When the offset or an iovec base or length is not a multiple of the probed alignment, the worker gathers the iovecs into one contiguous host buffer and runs a synchronous RMW through AlignedFile, then scatters the result back into guest memory. RawDisk constructs the AlignedFile from the disk file and the direct flag and passes it into each worker, so alignment is probed once at open time. The fixed VHD workers are threaded through the same AlignedFile based constructors using a non-direct AlignedFile to preserve current behavior. Signed-off-by: Anatol Belski <[email protected]>

Read the trailing footer sector through an AlignedFile rather than probing the device topology and reading a full logical block. The AlignedFile bounce buffer serves the trailing sector of an O_DIRECT fd whose offset is unaligned against the device block size, so the read no longer fails with EINVAL on a 4k sector backing store. Signed-off-by: Anatol Belski <[email protected]>

Thread the direct flag from the disk open options through VhdDisk into the AlignedFile the workers run on, so a fixed VHD opened with direct=on issues O_DIRECT I/O instead of buffered I/O. Alignment is probed once on that AlignedFile and reused by the sync and io_uring workers. Advertise host topology from VhdDisk::topology by probing the underlying file. On a 4096 byte sector filesystem opened with O_DIRECT this reports logical_block_size 4096 to the guest, so the guest never issues 512 byte I/O that the host kernel would reject as misaligned. Falls back to the default topology with a warning when the probe fails. Signed-off-by: Anatol Belski <[email protected]>

The qcow workers carried their own O_DIRECT alignment requirement and bounced unaligned cluster accesses through AlignedBuffer. Now that the data file is an AlignedFile that handles O_DIRECT transparently, the qcow layer can read and write through plain buffers and let AlignedFile perform the aligned bounce. Remove the alignment field and the per cluster AlignedBuffer RMW branches from both the sync and async workers. The async io_uring fast path still needs to avoid submitting unaligned guest iovecs under O_DIRECT, so gate it on is_direct rather than on a stored alignment value. Drop the QcowAsync alignment override so it reports the trait default sector size, matching QcowSync. qcow never submits guest iovecs to the kernel under O_DIRECT, so reporting a larger value only forced the request layer into an extra bounce buffer. This adds one buffer copy per unaligned O_DIRECT cluster but moves all alignment handling into a single place. The buffered path is unchanged. With qcow no longer the only caller, AlignedBuffer::read_exact_from becomes dead code, so remove it and switch its tests to read_from. Signed-off-by: Anatol Belski <[email protected]>

Add a parameterized helper that creates a 1 GiB ext4 loop filesystem with 4096 byte sectors, populates it with a small data disk in the requested format, attaches that disk with direct=on, and runs a 4096 byte aligned dd round trip with oflag=direct and iflag=direct followed by cmp. Wrappers exercise raw, qcow2, fixed VHD, and vhdx. The qcow2 and vhdx wrappers expect the guest to see the on disk LBS of 512. The others expect the host LBS of 4096. Signed-off-by: Anatol Belski <[email protected]>

weltling · 2026-06-20T10:16:23Z

Looks great - thank you for your patience on the iterations. Let's try and keep reducing code where we can.

Thanks a lot, and thanks for staying with the review through all the iterations. I will go through the patch set again and write back. If you have a spot in mind, please share.

Thanks

rbradford · 2026-06-20T11:33:15Z

Looks great - thank you for your patience on the iterations. Let's try and keep reducing code where we can.

Thanks a lot, and thanks for staying with the review through all the iterations. I will go through the patch set again and write back. If you have a spot in mind, please share.

I wasn't referring to this patch set - I think it's pretty lean - it was a more general remark.

weltling · 2026-06-20T12:31:42Z

Looks great - thank you for your patience on the iterations. Let's try and keep reducing code where we can.

Thanks a lot, and thanks for staying with the review through all the iterations. I will go through the patch set again and write back. If you have a spot in mind, please share.

I wasn't referring to this patch set - I think it's pretty lean - it was a more general remark.

Ah, got it. A lot of OOP for its own sake is sometimes too much, so reducing the code makes sense to me, especially with a good part of me written in C. Thanks for landing it!

weltling requested a review from a team as a code owner June 4, 2026 07:47

weltling force-pushed the block-aligned-io branch from 978b497 to 4a6589b Compare June 4, 2026 07:57

rbradford reviewed Jun 4, 2026

View reviewed changes

weltling force-pushed the block-aligned-io branch 2 times, most recently from 5fcd96a to 6bc7d5d Compare June 4, 2026 18:57

rbradford requested changes Jun 5, 2026

View reviewed changes

rbradford reviewed Jun 5, 2026

View reviewed changes

weltling force-pushed the block-aligned-io branch from 6bc7d5d to 49cc710 Compare June 5, 2026 21:39

weltling changed the title ~~block: Honor kernel reported DIO alignment across backends~~ block: Conform to kernel reported DIO alignment across backends Jun 5, 2026

rbradford requested changes Jun 8, 2026

View reviewed changes

weltling force-pushed the block-aligned-io branch from 49cc710 to 1136e66 Compare June 8, 2026 23:11

weltling changed the title ~~block: Conform to kernel reported DIO alignment across backends~~ # block: Conform to kernel reported direct I/O alignment across backends Jun 8, 2026

weltling force-pushed the block-aligned-io branch from 1136e66 to d9788e9 Compare June 8, 2026 23:22

weltling mentioned this pull request Jun 9, 2026

block: Consolidate I/O helpers on positional FileExt style API #8353

Closed

weltling force-pushed the block-aligned-io branch from d9788e9 to 4063b25 Compare June 12, 2026 08:29

rbradford mentioned this pull request Jun 12, 2026

block: Remove use of raw libc::pread/pwrite & implement VHDx O_DIRECT support #8378

Merged

rbradford marked this pull request as draft June 16, 2026 22:04

weltling force-pushed the block-aligned-io branch 2 times, most recently from 5028aba to 195a9a1 Compare June 17, 2026 12:05

rbradford requested changes Jun 17, 2026

View reviewed changes

rbradford changed the title ~~# block: Conform to kernel reported direct I/O alignment across backends~~ block: Improve O_DIRECT handling Jun 19, 2026

DemiMarie reviewed Jun 19, 2026

View reviewed changes

weltling force-pushed the block-aligned-io branch from 10269e6 to 9898502 Compare June 19, 2026 21:58

rbradford approved these changes Jun 20, 2026

View reviewed changes

weltling added 7 commits June 20, 2026 11:39

weltling force-pushed the block-aligned-io branch from 9898502 to b5ab072 Compare June 20, 2026 10:13

rbradford added this pull request to the merge queue Jun 20, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 20, 2026

rbradford added this pull request to the merge queue Jun 20, 2026

Merged via the queue into cloud-hypervisor:main with commit dd2f18e Jun 20, 2026
38 checks passed

weltling deleted the block-aligned-io branch June 20, 2026 12:52

This was referenced Jun 29, 2026

direct=on: guest gets 4K logical block size on reflink/CoW files, breaking 512-native cloud images (host-kernel-dependent) #8477

Open

block: Crate Refactoring - Cycle 3: Async Writes & Completion #8033

Open

rbradford added this to Cloud Hypervisor Roadmap Jul 8, 2026

github-project-automation Bot moved this to ✅ Done in Cloud Hypervisor Roadmap Jul 8, 2026

rbradford added the bug-fix Bug fix to include in release notes label Jul 8, 2026

umangPokhriyall mentioned this pull request Aug 28, 2026

The O_DIRECT read-modify-write path is not serialized against other I/O to the same block #8812

Open

		@@ -0,0 +1,27 @@
		// Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.

Uh oh!

Conversation

weltling commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

What this series does

Tests

Compatibility

Uh oh!

rbradford left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

weltling commented Jun 4, 2026

Uh oh!

rbradford left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rbradford commented Jun 5, 2026

Uh oh!

weltling commented Jun 5, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rbradford Jun 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

weltling commented Jun 4, 2026 •

edited

Loading

rbradford Jun 5, 2026 •

edited

Loading