Thanks to visit codestin.com
Credit goes to github.com

Skip to content

virtio: sparse-align initial BAR placement to avoid Windows PnP rebalance deadlock - #8205

Merged
rbradford merged 1 commit into
cloud-hypervisor:mainfrom
cocoonstack:fix/win-bar-rebalance
May 19, 2026
Merged

rbradford merged 1 commit into
cloud-hypervisor:mainfrom
cocoonstack:fix/win-bar-rebalance

Conversation

@CMGS

@CMGS CMGS commented May 13, 2026

Copy link
Copy Markdown
Contributor

Fixes #8202.

Windows 11 25H2's PnP arbiter (pci.sys) rewrites peer BAR addresses on resource rebalance and frequently picks targets that overlap another live device's BAR — see #8202 for the full Microsoft documentation references and the deadlock failure mode. With the default 512 KiB-aligned packed layout, three or four adjacent virtio device BARs at the top of MMIO64 are within Windows' preferred-address window, so the rebalance collides with non-trivial probability (~50% in our reproducer) and move_bar() correctly refuses the conflicting allocation. The guest, having committed the new address to its own state, never reconciles with the rolled-back config register and wedges early boot.

This PR takes the low-risk option #1 from the issue: bump the virtio capability BAR allocator alignment to 8 MiB so initial devices land at well-separated offsets (each 512 KiB BAR consumes 8 MiB of address slack). Windows-side BAR moves now have room to succeed without colliding with other CH-managed devices. On phys_bits=46, the alignment-induced waste is ~64 TiB of unused address space — negligible.

Verified on our testbed: Win11 25H2 boot + vm net --nics 1→2→4→2→1→0 resize chain + 0-NIC snapshot + clone-with-override + chain clone all pass deterministically.

Two commits:

  1. virtio: 8 MiB-aligned initial BAR placement — applies to fresh allocation in VirtioPciDevice::allocate_bars(). Both the Memory*BitRegion and the optional shm BAR paths are bumped to 8 MiB alignment.

  2. virtio: relax BAR alignment on restore — needed because the guest may have reprogrammed a BAR to a 512 KiB-aligned (but not 8 MiB-aligned) address while running; move_bar() uses CAPABILITY_BAR_SIZE (512 KiB) as its alignment, so any address the guest had settled on must remain valid through snapshot/restore. Without this, every restore from a snapshot of a guest that had moved any BAR fails with Allocating space for an IO BAR failed. Fresh allocation continues to use the sparse 8 MiB alignment.

This is independent of #8203 / #8204 (allocator state leak in move_bar()): even with PR #8204 keeping host-side state consistent, the rebalance attempt still fails when the target overlaps, leaving the guest with an invalid view of the BAR. Sparse layout prevents the rebalance from ever needing to land on an occupied range in the first place.

The issue mentions two alternative fixes (#2 coordinated swap, #3 QEMU-style accept-and-stomp). Both would be more invasive — happy to follow up with those if upstream prefers a deeper fix.

Signed-off-by: CMGS [email protected]

@CMGS
CMGS requested a review from a team as a code owner May 13, 2026 07:29
@CMGS
CMGS force-pushed the fix/win-bar-rebalance branch from 41709f3 to 2586f8b Compare May 13, 2026 07:36
Comment thread virtio-devices/src/transport/pci_device.rs Outdated
phip1611
phip1611 previously approved these changes May 13, 2026

@phip1611 phip1611 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice write-up and investigation of the issue! Code wise LGTM. I’m a little hesitant to approve this without input from someone with more PCI experience than me.

@phip1611
phip1611 dismissed their stale review May 13, 2026 07:49

hit wrong button - didn't want to approve...

@CMGS
CMGS force-pushed the fix/win-bar-rebalance branch 2 times, most recently from 51d9655 to 587daf8 Compare May 13, 2026 08:01
Comment thread virtio-devices/src/transport/pci_device.rs Outdated
@CMGS
CMGS force-pushed the fix/win-bar-rebalance branch 3 times, most recently from 0c18f57 to 4c2d97f Compare May 13, 2026 09:59
Comment thread virtio-devices/src/transport/pci_device.rs Outdated
Comment thread virtio-devices/src/transport/pci_device.rs Outdated
@CMGS
CMGS force-pushed the fix/win-bar-rebalance branch 2 times, most recently from 5b7b907 to 74ed43f Compare May 14, 2026 02:17
@CMGS
CMGS requested a review from rbradford May 14, 2026 17:04
@phip1611

Copy link
Copy Markdown
Member

Please rebase feature branches in CH, never update a branch using the merge method.

@CMGS

CMGS commented May 15, 2026

Copy link
Copy Markdown
Contributor Author

Please rebase feature branches in CH, never update a branch using the merge method.

rebased, thanks

@phip1611

Copy link
Copy Markdown
Member

I think something went wrong. Please take a look:

image

@CMGS
CMGS force-pushed the fix/win-bar-rebalance branch from 7b4c623 to d50a481 Compare May 15, 2026 12:10
@CMGS

CMGS commented May 15, 2026

Copy link
Copy Markdown
Contributor Author

I think something went wrong. Please take a look:

image

sorry, rebased, on the top of main now

@CMGS
CMGS force-pushed the fix/win-bar-rebalance branch from d50a481 to 8087691 Compare May 15, 2026 17:34
Comment thread virtio-devices/src/transport/pci_device.rs Outdated
@CMGS
CMGS force-pushed the fix/win-bar-rebalance branch from 8087691 to d959290 Compare May 16, 2026 04:48
@CMGS
CMGS requested a review from rbradford May 16, 2026 04:51
Comment thread virtio-devices/src/transport/pci_device.rs Outdated
@CMGS
CMGS force-pushed the fix/win-bar-rebalance branch 2 times, most recently from 02b3a06 to fe627c2 Compare May 16, 2026 16:40
@CMGS
CMGS requested a review from rbradford May 16, 2026 16:41

@rbradford rbradford left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great - please just squash the comment into the first commit. FWIW I think there is the right amount of detail in the comment message.

Windows 11 PnP rebalance rewrites peer BARs into the same range CH
packed the initial layout at, causing move_bar() failures and boot
deadlock. Pack Mmio64 BARs at 8 MiB stride. Mmio32 isn't wide enough
for the same stride, but its BARs don't participate in guest BAR
rebalancing.

On restore, pin the BAR to the snapshot address (alignment=None) so a
guest-relocated BAR with smaller alignment is accepted.

Signed-off-by: CMGS <[email protected]>
@CMGS
CMGS force-pushed the fix/win-bar-rebalance branch from fe627c2 to f8c6675 Compare May 19, 2026 02:55
@CMGS
CMGS requested a review from rbradford May 19, 2026 02:59
@rbradford
rbradford added this pull request to the merge queue May 19, 2026
Merged via the queue into cloud-hypervisor:main with commit f8f92bd May 19, 2026
41 checks passed
@CMGS
CMGS deleted the fix/win-bar-rebalance branch June 15, 2026 06:33
@github-project-automation github-project-automation Bot moved this from 🆕 New to ✅ Done in Cloud Hypervisor Roadmap Jul 8, 2026
@rbradford rbradford added the bug-fix Bug fix to include in release notes label Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix Bug fix to include in release notes

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

Windows guest PnP rebalance deadlocks boot when reprogrammed BAR overlaps another live device

3 participants