Codestin Search App

lisongqian · 2024-08-29T08:03:49Z

This patch introduces the Inter-VM shared memory(ivshmem) device
to support sharing a memory region between multiple processes running
different guests and the host.

This patch supports the basic ivshmem functions like ivshmem-plain in QEMU[1].

[1] https://www.qemu.org/docs/master/specs/ivshmem-spec.html

Signed-off-by: Yi Wang [email protected]
Signed-off-by: Songqian Li [email protected]

lisongqian · 2024-08-30T02:26:52Z

Sorry, I accidentally deleted the branch. Reopen it now.

rbradford · 2024-09-05T14:30:37Z

Could you elaborate some more on the use case for this?

lisongqian · 2024-09-12T14:00:48Z

Could you elaborate some more on the use case for this?

Thank you for your review. I add a use case to ivshmem.md. This provides a more detailed usage description of the ivshmem device.

lisongqian · 2024-09-26T08:39:04Z

@rbradford PTAL, I'd appreciate you letting me know of any suggestions.

rbradford · 2024-09-27T16:04:18Z

Pinging @cloud-hypervisor/cloud-hypervisor-reviewers - does anybody else that thoughts about this - i'm not sure about whether this fits into the "modern cloud" usescase.

Perhaps this could be behind a feature flag?

up2wing

Some comments added, PTAL.

up2wing · 2024-09-29T05:19:14Z

Pinging @cloud-hypervisor/cloud-hypervisor-reviewers - does anybody else that thoughts about this - i'm not sure about whether this fits into the "modern cloud" usescase.

As far as I know, stratovirt uses ivshmem for some kind of sound card[1] and dpdk uses it for zero-copy data sharing. Also, people can easily share memory between guest and host without vm-exit, it's useful for some performance use case.

Perhaps this could be behind a feature flag?

Well in my opinion as this can toggle using command line or config and no harm to performance, a feature flag maybe
not needed?

openeuler-mirror/stratovirt@5f2220a

lisongqian · 2024-10-21T10:16:06Z

@up2wing Thank you for your review. All comments have fixed, PTAL.

lisongqian · 2024-10-21T10:25:19Z

Perhaps this could be behind a feature flag?

As @up2wing says, ivshmem needs to be enabled by passing a param, I don't think it matters to add a feature flag unless you consider more things? @rbradford

liuw

Sorry for being late to the party. Is there a way to test this in the CI so that it doesn't regress?

liuw · 2024-10-22T17:15:52Z

Pinging @cloud-hypervisor/cloud-hypervisor-reviewers - does anybody else that thoughts about this - i'm not sure about whether this fits into the "modern cloud" usescase.

Perhaps this could be behind a feature flag?

To me this enables a few interesting use cases.

lisongqian · 2024-10-23T09:00:08Z

Sorry for being late to the party. Is there a way to test this in the CI so that it doesn't regress?

I can add a case to test ivshmem device discovery but can't test the device function since our test kernel doesn't have a driver for ivshmem. So I'm not sure if we need to add this test case.

liuw · 2024-10-23T14:45:27Z

Sorry for being late to the party. Is there a way to test this in the CI so that it doesn't regress?

I can add a case to test ivshmem device discovery but can't test the device function since our test kernel doesn't have a driver for ivshmem. So I'm not sure if we need to add this test case.

Is there a plan to upstream the said driver?

I think a very basic test case will be to make sure the PCI device shows up correctly in the guest, with the correct BAR configuration. This can be done by checking lspci's output against some expected values.

likebreath

Thank you for persistency. Besides some comments below, one main open from me is around live-migration.

Similar to my comments about snapshot support, I am worried that the live migration support is very limited.Say does the current live-migration work when the ivhsmem device is actively being used (say write and read)? Also, how does the ivshmem device handle dirty bit tracking from the changes of other running VM when it is shared?

I don't think we have answers for all these questions, given the status on the ivshmem device itself. Also I don't think solving these problems should be the scope of this PR.

I'd suggest putting some warning messages to advise the support of snapshot and live-migration is very limited and experimental only.

lisongqian · 2025-07-16T07:55:53Z

Similar to my comments about snapshot support, I am worried that the live migration support is very limited.Say does the current live-migration work when the ivhsmem device is actively being used (say write and read)? Also, how does the ivshmem device handle dirty bit tracking from the changes of other running VM when it is shared?

The backend of ivshmem device is a file, the live migration of ivhmem device is similar to that of a block device. If this backend file is shared between VMs, it's better to use a file on NFS. But the current ivshmem device(ivshmem-plain) is mainly used to share memory between host and guest, it's recommended to use ivshmem-doorbell for sharing memory between VMs.
IMO, The live migration limitations of the ivshmem device are the same as those of shared block devices.

I'd suggest putting some warning messages to advise the support of snapshot and live-migration is very limited and experimental only.

I will add some warning messages.

likebreath · 2025-07-16T21:04:18Z

But the current ivshmem device(ivshmem-plain) is mainly used to share memory between host and guest, it's recommended to use ivshmem-doorbell for sharing memory between VMs.

That's good to clarify. Can you please include this as a part of the limitation comments? Thank you.

likebreath · 2025-07-29T16:38:51Z

@lisongqian I want to check-in and make sure you get everything you need from me to make progress on this pR. I think we are pretty close to finally close this long standing one. Let me know if you have any questions.

lisongqian · 2025-07-30T04:04:31Z

@lisongqian I want to check-in and make sure you get everything you need from me to make progress on this pR. I think we are pretty close to finally close this long standing one. Let me know if you have any questions.

Sorry for the late reply, I was a bit busy last week. PR is ready now. @likebreath

likebreath

All major opens are now closed. Some more comments below. I think we are pretty close to land this PR. Thanks again for your persistency.

lisongqian · 2025-08-07T15:06:41Z

@likebreath Thank you very much for your careful review! All comments have been replied.

lisongqian · 2025-08-08T03:44:30Z

BTW, I also improved the check conditions for the link checker.

likebreath · 2025-08-08T22:56:16Z

BTW, I also improved the check conditions for the link checker.

Thank you, can you please open a separate PR for these changes?

likebreath

Changes look goo to me. Only two comments left to make the integration test clearer.

Also another big change was just landed (#7117), you will need a rebase. Please let me know if you need help with it.

Move UserspaceMapping to vm-device to avoid redefinition since UserspaceMapping is used by both `virtio-devices` and `device` crate. Signed-off-by: Songqian Li <[email protected]>

This patch introduces the inter-vm shared memory(ivshmem) device to share a memory region between multiple processes running different guests and the host. This patch supports the basic ivshmem functions like ivshmem-plain in QEMU[1]. [1] https://www.qemu.org/docs/master/specs/ivshmem-spec.html Signed-off-by: Yi Wang <[email protected]> Signed-off-by: Songqian Li <[email protected]>

likebreath · 2025-08-12T17:29:40Z

Thank you for the fast turn around. With the latest integration test updates, it appears they caught some actual issues:

thread 'ivshmem::test_ivshmem' panicked at tests/integration.rs:2368:5:
assertion `left == right` failed
  left: "ivshmem device test data read"
 right: ""

Signed-off-by: Yi Wang <[email protected]> Signed-off-by: Songqian Li <[email protected]>

Signed-off-by: Songqian Li <[email protected]>

Signed-off-by: Bo Chen <[email protected]> Signed-off-by: Songqian Li <[email protected]>

Signed-off-by: Songqian Li <[email protected]>

likebreath

@lisongqian Thanks again for the contribution and persistency. I believe we are ready to merge this PR.

lisongqian · 2025-08-15T02:38:07Z

we

Thank you equally for your continued help. ❤️
Thanks to everyone who helped. ❤️

lisongqian requested a review from a team as a code owner August 29, 2024 08:03

lisongqian force-pushed the upstream/ivshmem branch 3 times, most recently from ad319d6 to bf8f368 Compare August 29, 2024 08:52

lisongqian closed this Aug 29, 2024

lisongqian deleted the upstream/ivshmem branch August 29, 2024 09:04

lisongqian restored the upstream/ivshmem branch August 30, 2024 02:24

lisongqian reopened this Aug 30, 2024

lisongqian force-pushed the upstream/ivshmem branch from bf8f368 to 73e7199 Compare September 5, 2024 08:05

lisongqian force-pushed the upstream/ivshmem branch from 73e7199 to 6fd68f5 Compare September 12, 2024 13:46

lisongqian force-pushed the upstream/ivshmem branch from 6fd68f5 to fc45ea0 Compare September 26, 2024 08:18

up2wing reviewed Sep 29, 2024

View reviewed changes

lisongqian force-pushed the upstream/ivshmem branch 4 times, most recently from 62941ed to a7a4efd Compare October 21, 2024 10:14

liuw reviewed Oct 22, 2024

View reviewed changes

Comment thread docs/ivshmem.md Outdated

lisongqian force-pushed the upstream/ivshmem branch 2 times, most recently from 1b699dd to b1776a0 Compare October 23, 2024 08:51

lisongqian force-pushed the upstream/ivshmem branch 5 times, most recently from 847c7cf to c41f05a Compare July 9, 2025 16:11

likebreath reviewed Jul 16, 2025

View reviewed changes

likebreath reviewed Aug 6, 2025

View reviewed changes

likebreath reviewed Aug 11, 2025

View reviewed changes

Comment thread tests/integration.rs Outdated

Comment thread tests/integration.rs Outdated

lisongqian added 2 commits August 12, 2025 11:18

virtio-devices: move userspace mapping to vm-device

5581dff

Move UserspaceMapping to vm-device to avoid redefinition since UserspaceMapping is used by both `virtio-devices` and `device` crate. Signed-off-by: Songqian Li <[email protected]>

RuoqingHe reviewed Aug 14, 2025

View reviewed changes

Comment thread vmm/src/config.rs Outdated

lisongqian added 7 commits August 14, 2025 11:53

vmm: ivshmem device support

563769f

Signed-off-by: Yi Wang <[email protected]> Signed-off-by: Songqian Li <[email protected]>

docs: add ivshmem device introduction

427e884

Signed-off-by: Songqian Li <[email protected]>

fuzz: add ivshmem device config

8f9f3ab

Signed-off-by: Songqian Li <[email protected]>

tests: add ivshmem integration test case

a65cb02

Signed-off-by: Songqian Li <[email protected]>

ci: add build and quality check for ivshmem feature

4f78250

Signed-off-by: Songqian Li <[email protected]>

devices, vmm: Handle ivshmem bar reprogramming properly

ac4aca1

Signed-off-by: Bo Chen <[email protected]> Signed-off-by: Songqian Li <[email protected]>

tests: add ivshmem live migration test case

f9220e1

Signed-off-by: Songqian Li <[email protected]>

likebreath approved these changes Aug 14, 2025

View reviewed changes

likebreath mentioned this pull request Aug 18, 2025

vm-migration: Add support for downtime limits #7033

Closed

likebreath mentioned this pull request Sep 18, 2025

nvidia-smi fails after hotplug/unplug #7328

Closed

Uh oh!

Conversation

lisongqian commented Aug 29, 2024

Uh oh!

lisongqian commented Aug 30, 2024

Uh oh!

rbradford commented Sep 5, 2024

Uh oh!

lisongqian commented Sep 12, 2024

Uh oh!

lisongqian commented Sep 26, 2024

Uh oh!

rbradford commented Sep 27, 2024

Uh oh!

up2wing left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

up2wing commented Sep 29, 2024

Uh oh!

lisongqian commented Oct 21, 2024

Uh oh!

lisongqian commented Oct 21, 2024

Uh oh!

liuw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

liuw commented Oct 22, 2024

Uh oh!

lisongqian commented Oct 23, 2024

Uh oh!

liuw commented Oct 23, 2024

Uh oh!

likebreath left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lisongqian commented Jul 16, 2025

Uh oh!

likebreath commented Jul 16, 2025

Uh oh!

likebreath commented Jul 29, 2025

Uh oh!

lisongqian commented Jul 30, 2025

Uh oh!

likebreath left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lisongqian commented Aug 7, 2025

Uh oh!

lisongqian commented Aug 8, 2025

Uh oh!

likebreath commented Aug 8, 2025

Uh oh!