Thanks to visit codestin.com
Credit goes to github.com

Skip to content

vmm: support migration of paused VMs - #8099

Merged
rbradford merged 2 commits into
cloud-hypervisor:mainfrom
phi-nguyendp:phi.nguyendp/migrate-paused-vm
May 4, 2026
Merged

rbradford merged 2 commits into
cloud-hypervisor:mainfrom
phi-nguyendp:phi.nguyendp/migrate-paused-vm

Conversation

@phi-nguyendp

@phi-nguyendp phi-nguyendp commented Apr 25, 2026

Copy link
Copy Markdown
Contributor

This extends migration to support paused VMs, preserving the paused state on the destination.

Changes:

  • Add CompletePaused protocol command that finalizes migration without resuming the VM on the destination
  • Skip the pause step during migration if the VM is already paused
  • On migration failure, only restore the running state if the VM was originally running (not paused)

Closes #7815.

@phi-nguyendp
phi-nguyendp requested a review from a team as a code owner April 25, 2026 08:01
@phi-nguyendp
phi-nguyendp force-pushed the phi.nguyendp/migrate-paused-vm branch 2 times, most recently from 383ed1e to a9e69d3 Compare April 25, 2026 11:11
@phip1611
phip1611 self-requested a review April 25, 2026 11:57
@phip1611 phip1611 linked an issue Apr 25, 2026 that may be closed by this pull request

@phip1611 phip1611 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution!

Using a dedicated Migration Protocol command to decide whether the VM should be resumed on the destination is a neat and simple solution to the underlying problem. That said, I have a couple of questions and requests.

Could you please elaborate on your use case and motivation for migrating paused VMs? I’m slightly concerned that this change may be AI-assisted without being grounded in active work on live migration. Apologies if that assumption is wrong. Given the times we live in, reviewers need to be more aware and cautious about this. If you used AI/LLMs while preparing this contribution, please make sure you follow our CONTRIBUTING.md guidelines.

Also, a change like this needs integration test coverage. Please add an integration test for it. I think you can extend the existing migration helper with a paused: bool flag.

PS: Converted to draft as non-trivial parts (such as an integration tests) are missing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You will need to update the mermaid diagram please:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

Comment thread vmm/src/lib.rs Outdated
@@ -979,11 +979,16 @@ impl Vmm {
StateReceived {
state_receive_begin,
} => match req.command() {
Command::Complete => {
cmd @ (Command::Complete | Command::CompletePaused) => {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this combined match arm doesn't make sense! For example, we still log Migration (incoming): resume:{}ms when we do not resume the VM.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's right, I separated them into two different arms

@phip1611
phip1611 marked this pull request as draft April 27, 2026 08:32
@phi-nguyendp

phi-nguyendp commented Apr 28, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for your contribution!

Using a dedicated Migration Protocol command to decide whether the VM should be resumed on the destination is a neat and simple solution to the underlying problem. That said, I have a couple of questions and requests.

Could you please elaborate on your use case and motivation for migrating paused VMs? I’m slightly concerned that this change may be AI-assisted without being grounded in active work on live migration. Apologies if that assumption is wrong. Given the times we live in, reviewers need to be more aware and cautious about this. If you used AI/LLMs while preparing this contribution, please make sure you follow our CONTRIBUTING.md guidelines.

  • No, I didn't use AI to write this code, although I used AI to write the commit message (not a native speaker).
  • When we do host maintenance, we don't want to reach out to each VM owner and ask them if we can resume/destroy their VMs before migrating them to another host. We did use the same approach for our QEMU VMs.
  • Real use case for cloud-hypervisor: not really, we do not have many cloud-hypervisor VMs yet.
  • My company started using cloud-hypervisor recently, so I want to explore how the migration feature in cloud-hypervisor works. I see this issue when reading code, and I guess it is not complicated, so I'm giving it a try.

Also, a change like this needs integration test coverage. Please add an integration test for it. I think you can extend the existing migration helper with a paused: bool flag.

I'll try to add it

PS: Converted to draft as non-trivial parts (such as an integration tests) are missing.

@phip1611

Copy link
Copy Markdown
Member

Perfect - thanks! :)

@phi-nguyendp
phi-nguyendp force-pushed the phi.nguyendp/migrate-paused-vm branch 3 times, most recently from 72d11d7 to 39e180b Compare May 1, 2026 01:23
@phi-nguyendp

Copy link
Copy Markdown
Contributor Author

Added integration tests

@rbradford
rbradford force-pushed the phi.nguyendp/migrate-paused-vm branch from 6b765ba to f0245b3 Compare May 1, 2026 13:38

@rbradford rbradford left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some tests also got accidentally deleted

Comment thread vm-migration/src/protocol.rs Outdated
@@ -103,6 +103,7 @@ use crate::bitpos_iterator::BitposIteratorExt;
/// Configured --> Configured: Memory
/// Configured --> StateReceived: State
/// StateReceived --> Completed: Complete
/// StateReceived --> PausedVMCompleted: CompletePaused

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see PausedVMCompleted mentioned anywhere else?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I misunderstood how the diagram was constructed, so I’ve updated it again.

Comment thread cloud-hypervisor/tests/integration.rs Outdated
}

#[test]
#[ignore = "See #5532"]
#[cfg(target_arch = "x86_64")]
#[cfg(not(feature = "mshv"))]
fn test_live_upgrade_ovs_dpdk_local() {
_test_live_migration_ovs_dpdk(true, true);
_test_live_migration_ovs_dpdk(true, true, true);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you intend to turn it on for paused here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed this test case.

Comment thread cloud-hypervisor/tests/integration.rs Outdated
}

#[test]
#[cfg(not(feature = "mshv"))]
fn test_live_upgrade_watchdog_local() {
_test_live_migration_watchdog(true, true);
_test_live_migration_watchdog(true, true, false);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need all these variants.

Just one live migration and one live upgrade test with paused should be sufficient. This will consume too much CI time.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’ve uploaded a new version. There will only be two test cases for live migration—one basic case and one local variant. For live update, since the integration tests rely on the released cloud-hypervisor-static binary, I assume I can only add new tests once scripts/run_integration_tests_x86_64.sh is updated to the new version.

@phi-nguyendp
phi-nguyendp force-pushed the phi.nguyendp/migrate-paused-vm branch from f0245b3 to 155346a Compare May 2, 2026 05:25
@phi-nguyendp
phi-nguyendp marked this pull request as ready for review May 4, 2026 02:23
@phip1611

phip1611 commented May 4, 2026

Copy link
Copy Markdown
Member

Will review it this week.

@phip1611 phip1611 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM! We don't do merge main into feature branch-style git histories. Instead, we use rebases. Although I think the MQ would resolve this automatically, please adapt your branch.

This extends migration to also support paused VMs, preserving the
paused state on the destination.

Changes:
- Add CompletePaused protocol command that finalizes migration without
 resuming the VM on the destination
- Skip the pause step during migration if the VM is already paused
- On migration failure, only restore the running state if
  the VM was originally running (not paused)

Signed-off-by: Nguyen Dinh Phi <[email protected]>
Adding a paused flag to live_migration() tests; when this
flag is set, the VM will be paused before migration is
performed.

Signed-off-by: Nguyen Dinh Phi <[email protected]>
@phi-nguyendp
phi-nguyendp force-pushed the phi.nguyendp/migrate-paused-vm branch from 622d34b to c29900e Compare May 4, 2026 07:35
@phi-nguyendp

Copy link
Copy Markdown
Contributor Author

Thanks, LGTM! We don't do merge main into feature branch-style git histories. Instead, we use rebases. Although I think the MQ would resolve this automatically, please adapt your branch.

I've rebased it.

@phip1611

phip1611 commented May 4, 2026

Copy link
Copy Markdown
Member

thanks! Test coverage is looking solid. Happy to have this in the Merge Queue soon

@rbradford
rbradford added this pull request to the merge queue May 4, 2026
Merged via the queue into cloud-hypervisor:main with commit c595125 May 4, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

migration: add support for migrating paused VMs

4 participants