Add an option to control nested virtualization - #7408

I assume there isn't an option for KVM?

We should take into consideration we may want to provide very fine-grained configurations in the medium term.

Putting this into PlatformConfig looks wrong. This is a CPU feature, not a platform feature.

I assume there isn't an option for KVM?

We should take into consideration we may want to provide very fine-grained configurations in the medium term.

Putting this into PlatformConfig looks wrong. This is a CPU feature, not a platform feature.

We can directly add to the VM configuration as features or something?

I assume there isn't an option for KVM?
We should take into consideration we may want to provide very fine-grained configurations in the medium term.
Putting this into PlatformConfig looks wrong. This is a CPU feature, not a platform feature.

We can directly add to the VM configuration as features or something?

There is a workstream to expose fine-grained control over the CPU features. I think this belongs there.

KVM can benefit from the fine-grained control.

For the record: with current Cloud Hypervisor on Linux/KVM, nesting works out of the box including state save/resume and live-migration. It should work similar with MSHV.

For the record: with current Cloud Hypervisor on Linux/KVM, nesting works out of the box including state save/resume and live-migration. It should work similar with MSHV.
On MSHV without that CPU feature enablement it would not work.

We could add a runtime configuration for kvm similar to MSHV which disables/enables the support explicitly. Then we can have similar behavior for both backends - and disabling nesting is a valid use case in case infrastructure maintainers are concerned about people breaking out of the VMX emulation for example

nit: there are multiple typos in the commit messages.

@anirudhrb Please take another look. Thx

We don't have a nested option for KVM - it's always enabled. Why do we need to make it an option for MSHV?

If you add the option to --cpu for MSHV then you need to do the same for KVM.

With MSHV we have to pass the flag during VM creations. The customers want to control that flow with CLI, that's why I raised the PR.

Do you recommend adding feature guard around the nested feature, or recommend adding an extra argument like --mshv-cpu-feaures?

We don't have a nested option for KVM - it's always enabled. Why do we need to make it an option for MSHV?
If you add the option to --cpu for MSHV then you need to do the same for KVM.

With MSHV we have to pss the flag during VM creations. THe customers wants to control that flow with CLI, that's why I raised the PR.

Do you recommend adding feature guard around the nested feature, or recommend adding an extra argument like --mshv-cpu-feaures?

You haven't explained why you can't have it turned on all the time.

We don't have a nested option for KVM - it's always enabled. Why do we need to make it an option for MSHV?
If you add the option to --cpu for MSHV then you need to do the same for KVM.

With MSHV we have to pss the flag during VM creations. THe customers wants to control that flow with CLI, that's why I raised the PR.
Do you recommend adding feature guard around the nested feature, or recommend adding an extra argument like --mshv-cpu-feaures?

You haven't explained why you can't have it turned on all the time.

Customers does not wnat to turn on by default.

We don't have a nested option for KVM - it's always enabled. Why do we need to make it an option for MSHV?
If you add the option to --cpu for MSHV then you need to do the same for KVM.

With MSHV we have to pss the flag during VM creations. THe customers wants to control that flow with CLI, that's why I raised the PR.
Do you recommend adding feature guard around the nested feature, or recommend adding an extra argument like --mshv-cpu-feaures?

You haven't explained why you can't have it turned on all the time.

We can turn it on by default with no issue but customer does not want it default rather it wants to control in run-time. Do you also refer to turn it on by default with the CpuFeatures default initialization? Also in future we will have more Cpu features on MSVH that might need to be turned on the CLI.

Okay. If you want to make it controllable then I think you should make it controllable for KVM as well - with a default of on.

Okay. If you want to make it controllable then I think you should make it controllable for KVM as well - with a default of on.

Just curious, If we add some features that is only available for MSHV, How do we do it?

Okay. If you want to make it controllable then I think you should make it controllable for KVM as well - with a default of on.

Just curious, If we add some features that is only available for MSHV, How do we do it?

Probably something like x-mshv- prefix

Why not add a parameter to --cpu, such as: --cpu nesting={on,off}? This is generic. I think in the KVM case we can also set the default to false, we probably just have to remove a few CPUid entries in that case.

with a default of on.

For the record: Some operators are concerned about the increased attack surface of the KVM VMX emulator code.

Why not add a parameter to --cpu, such as: --cpu nesting={on,off}? This is generic. I think in the KVM case we can also set the default to false, we probably just have to remove a few CPUid entries in that case.

Exactly what I was getting at!

with a default of on.

For the record: Some operators are concerned about the increased attack surface of the KVM VMX emulator code.

We've already shipped with a default of on - so we can't change that now.

Thank you for your patience on this PR! We do need to be especially careful with adding APIs and things could affect our defaults.

Thank you @rbradford for all your effort on this. Your thorough review was very helpful and some learning for me.