Codestin Search App

shayonj · 2026-03-06T14:59:11Z

This introduces a new memory_restore_mode option on the restore path that allows Cloud Hypervisor to populate guest memory lazily using userfaultfd instead of reading the entire snapshot file into guest RAM before the VM resumes.

Motivation

The existing restore path reads the full memory-ranges file into guest RAM before restore completes. For multi-gigabyte guests this front-loads a significant I/O cost and increases restore latency proportionally with guest memory size. This has been raised as a limitation in prior discussions around snapshot resume performance (#7623).

Firecracker solves this by passing the userfaultfd file descriptor to an external handler process over a Unix socket, giving orchestrators full control over how pages are served. That model is powerful but requires an external component and introduces deployment complexity. This PR takes a simpler approach that keeps the handler entirely inside the VMM process, which is self-contained and requires no changes to the deployment model. For now, perhaps?

Why userfaultfd instead of mmap

The natural first thought for lazy restore is to mmap(MAP_PRIVATE) the snapshot file over the guest memory region. However, that replaces the existing memory mapping, which breaks subsystems that depend on the original mapping type and backing pages, including shared memory for vhost-user, VFIO DMA pinning, and KVM memory slots. Related discussions (#7624, #5749, #6110, #4069, #7302). Userfaultfd avoids this by intercepting page faults on the existing mapping and populating pages into it via UFFDIO_COPY, preserving the original mapping type.

How it works

When memory_restore_mode=ondemand is specified on the restore command, the memory manager creates a userfaultfd descriptor after setting up guest memory but before populating it. It negotiates the kernel features required for the actual memory zone mapping types (requesting UFFD_FEATURE_MISSING_SHMEM for shared memory zones and UFFD_FEATURE_MISSING_HUGETLBFS for hugepage-backed zones) and registers each guest RAM range for missing-page fault interception. A handler thread is spawned that uses epoll to wait for page fault events on the userfaultfd. On each fault, it seeks into the snapshot file, reads the corresponding page, and resolves the fault with UFFDIO_COPY. The handler handles concurrent faults from multiple vCPUs by treating EEXIST (page already resolved by a prior fault) as a benign race and waking any remaining blocked threads with UFFDIO_WAKE. Once all pages have been served the handler exits automatically, and it is also stopped cleanly on VM shutdown via an eventfd signal.

If memory_restore_mode is omitted the existing eager copy restore path is used, so there is no change to current behavior.

Strict failure semantics

Went with the mindset that this is an explicit mode selection. If the user asks for ondemand mode and the userfaultfd cannot be created, the required kernel features are not available, or any region cannot be registered, restore fails immediately. Instead of silently falling back to eager copy, which would make behavior non-deterministic and make it harder to debug issues.

Constraints

Prefaulting (prefault=on) cannot be combined with ondemand mode because the two are contradictory. This is validated early and produces a clear error message.

Changes

The RestoreConfig struct gains a memory_restore_mode field with a MemoryRestoreMode enum (Copy or Ondemand). This is a restore-time parameter, not VM hardware configuration, so it belongs in RestoreConfig rather than in the persisted VM config. The field is plumbed through Vmm::vm_restore, Vm::new, and MemoryManager::new_from_snapshot to the point where the restore strategy is dispatched. The MemoryZone struct now carries shared and hugepages metadata from the zone configuration so the restore path can derive the exact userfaultfd features needed for the actual runtime mappings. A small userfaultfd constants module provides the ioctl numbers and protocol constants, and a separate uffd module wraps the raw ioctls into safe Rust functions. The seccomp filter is updated to allow the userfaultfd syscall and the four uffd ioctls (UFFDIO_API, UFFDIO_COPY, UFFDIO_REGISTER, UFFDIO_WAKE) under the VMM thread profile. The OpenAPI spec is updated with the new enum field.

Added integration tests for coverage in a separate commit.

shayonj · 2026-03-06T15:23:40Z

Requesting review from folks who I have seen active in previous similar conversations 🙏🏾

@rbradford

rbradford

Thanks for your contribution - just a quick review initially! Perhaps also split out the bindings and abstraction module into their own commits to make the changes that touch the common code easier to review.

(Could even go for further and move API changes out too.)

Also I don't see changes to ch-remote.

DamianB-BitFlipper · 2026-03-06T18:29:09Z

@rbradford Thanks for reviewing so promptly. We're hoping to get uffd in to CH to enable some really powerful features!

rbradford · 2026-03-06T18:34:30Z

@rbradford Thanks for reviewing so promptly. We're hoping to get uffd in to CH to enable some really powerful features!

Thank you for your contribution - i'm looking forward to seeing this feature land too.

DamianB-BitFlipper · 2026-03-06T20:05:20Z

I thoroughly reviewed the code and added my comments. Overall very happy with the code, great work @shayonj .

shayonj · 2026-03-06T20:48:10Z

Thank you for the reviews @rbradford and @DamianB-BitFlipper . I think I addressed all of them and left a few open for discussion.

(Could even go for further and move API changes out too.)

This is also addressed now. Let me know if it's not clear.

Also I don't see changes to ch-remote.

I blelieve ch-remote delegates to RestoreConfig::parse() for parsing and uses RestoreConfig::SYNTAX for its help text which is getting updated. So ch-remote restore source_url=...,memory_restore_mode=ondemand works transparently without.

I also ran a before/after comparison of a restore from snapshot on a GCE n2-standard-8 (2GB guest raw snapshot, 3 iterations each):

	Copy (eager)	Ondemand (userfaultfd)
Regular memory	7,140ms	83ms
Hugepage (2MB)	7,147ms	85ms
RSS after restore	2,048MB	7MB

Showing some promising figures.

Restore guest RAM by mapping the snapshot memory file copy-on-write over the still-unconsumed private anonymous guest mappings, before any KVM memslot, device or thread consumes them. Nothing is copied up front, pages fault in from the page cache — so VMs restored from the same snapshot share it — and guest writes stay private per VM. Because the mapping is established at region-creation time rather than being overlaid after the fact, it does not hit the mapping-identity problems that motivated removing the original 2020 CoW-restore path (a60b437) or rejecting an mmap overlay during the on-demand restore work (cloud-hypervisor#7800). Opt-in via memory_restore_mode=mmap; the default Copy path is unchanged. The overlay is confined to plain private-anonymous regions (checked per region via file_offset), re-applies the region reserve and THP policy, and rejects a snapshot file shorter than the saved ranges. Shared or hugepage RAM (global or per-zone), zones with numa/reserve/mergeable/ hotplug attributes, resizable RAM, KSM, pvmemcontrol and device passthrough fall back to the eager copy. The snapshot file must remain on disk and unchanged for the VM lifetime. Measured on a 16-core x86_64 host (512 MiB guests, same binary, only the mode switched, 3 interleaved rounds): single restore p50 54-58ms -> 22-35ms; 16 concurrent restores from one snapshot: per-restore p50 443-463ms -> 72-82ms, wall 456-478ms -> 87-108ms. Restored VMs boot and run normally. Signed-off-by: CMGS <[email protected]>

Restore guest RAM by mapping the snapshot memory file copy-on-write over the still-unconsumed private anonymous guest mappings, before any KVM memslot, device or thread consumes them. Nothing is copied up front, pages fault in from the page cache — so VMs restored from the same snapshot share it — and guest writes stay private per VM. Because the mapping is established at region-creation time rather than being overlaid after the fact, it does not hit the mapping-identity problems that motivated removing the original 2020 CoW-restore path (a60b437) or rejecting an mmap overlay during the on-demand restore work (cloud-hypervisor#7800). Opt-in via memory_restore_mode=copyonwrite; the default Copy path is unchanged. The overlay is confined to plain private-anonymous regions (checked per region via file_offset), re-applies the region reserve and THP policy, and rejects a snapshot file shorter than the saved ranges. Shared or hugepage RAM (global or per-zone), zones with numa/reserve/ mergeable/hotplug attributes, resizable RAM, KSM, pvmemcontrol and device passthrough fall back to the eager copy. The snapshot file must remain on disk and unchanged for the VM lifetime. Measured on a 16-core x86_64 host (512 MiB guests, same binary, only the mode switched, 3 interleaved rounds): single restore p50 54-58ms -> 22-35ms; 16 concurrent restores from one snapshot: per-restore p50 443-463ms -> 72-82ms, wall 456-478ms -> 87-108ms. Restored VMs boot and run normally. Signed-off-by: CMGS <[email protected]>

shayonj requested a review from a team as a code owner March 6, 2026 14:59

shayonj commented Mar 6, 2026

View reviewed changes

Comment thread vmm/src/seccomp_filters.rs Outdated

shayonj force-pushed the uffd-restore branch 4 times, most recently from 4b08c0e to 7b853b3 Compare March 6, 2026 17:03

rbradford reviewed Mar 6, 2026

View reviewed changes