AArch64 initial build #1168

michael2012z · 2020-05-13T10:18:47Z

This PR is splitted from #1126.
It prepares for upcoming PR's to enable AArch64. All building issues were fixed, but no functionality was introduced. A lot of #[cfg(target_arch = "xxx")] were used in source files to separate architecture specific code. Some utility files were updated as well.

For X86, the logic of code was not changed at all. Nothing should be broken.

For ARM, the architecture specific part is still empty. Some new added ARM-specific code may looks weird, because they were added temporarily to workaround lint warnings. But they will be replaced later by other commits with real functionality.

A workflow for "Checks" was setup to cross-build for AArch64. But now the cross-build fails, which is related to Serde in kvm-bindings. The reason is that CH forked kvm-bindings to add the support of Serde, the modification was mainly for X86 and a little was modified for ARM, but it is not ready for ARM. A workaround is to revert the modification for ARM totally. A PR for this has been issued to cloud-hypervisor/kvm-bindings

rbradford · 2020-05-13T10:52:48Z

@michael2012z Yes please feel free to send a PR to the appropriate fork repos targetting the "ch" branch. Make sure to mark it for review by @rbradford and @sboeuf

michael2012z · 2020-05-19T07:33:37Z

Hi, @rbradford , @sboeuf

Regarding the build error of kvm-bindings with feature with-serde, I tried an alternative workaround in the new added cross-build workflow: update vmm/Cargo.toml file before building (https://github.com/cloud-hypervisor/cloud-hypervisor/pull/1168/files#diff-2f78b1ca0982fac40f2687f73fc9dcabR26), now we don't have to pend on the progress of cloud-hypervisor/kvm-bindings#1.

Now cross-build jobs have passed. This PR is ready for review. Would you like to take a look?

rbradford

Looking nice so far. Just some comments to help reduce some of the conditionals

rbradford · 2020-05-20T12:39:20Z

vmm/src/memory_manager.rs

 }

+#[cfg(target_arch = "aarch64")]
+pub fn get_host_cpu_phys_bits() -> u8 {


Please move get_host_cpu_phys_bits to the arch/ crate.

rbradford · 2020-05-20T12:42:28Z

vmm/src/cpu.rs

        guest_memory: GuestMemoryAtomic<GuestMemoryMmap>,
-        kvm: &Kvm,
+        #[cfg(target_arch = "x86_64")] kvm: &Kvm,
+        #[cfg(target_arch = "aarch64")] _kvm: &Kvm,


#[cfg_attr(target_arch = "aarch64", allow(dead_code)]

rbradford · 2020-05-20T12:43:58Z

vmm/src/cpu.rs

+        _inserting: bool,
+        _snapshot: Option<Snapshot>,
+    ) -> Result<()> {
+        // temporary code to mute "unused variable" warnings


Put this as unimplemented! and then use #[cfg_attr(target_arch = "aarch64", allow(dead_code)] on the fields instead,

rbradford · 2020-05-20T12:44:14Z

vmm/src/device_manager.rs

+    fn add_ioapic(&mut self) -> DeviceManagerResult<Arc<Mutex<ioapic::Ioapic>>> {
+        // temporary code to mute "unused variable" warnings
+        let _ = &self.msi_interrupt_manager;
+        panic!("impossible for aarch64");


unreachable!()

rbradford · 2020-05-20T12:45:30Z

vmm/src/memory_manager.rs

        };

+        #[cfg(target_arch = "aarch64")]
+        let start_addr = GuestAddress((mem_end.0 + 1 + (256 << 20)) & !((128 << 20) - 1));


Refactor out into a start_addr() function.

I refactored more to cover the similar calculation for the start address of device space in new().

vmm/src/seccomp_filters.rs

rbradford · 2020-05-20T12:46:39Z

vmm/src/vm.rs


-        if !kvm.check_extension(Cap::TscDeadlineTimer) {
-            return Err(Error::CapabilityMissing(Cap::TscDeadlineTimer));
+        #[cfg(target_arch = "x86_64")]


Refactor out into a function in the arch crate check_required_kvm_extensions() ?

michael2012z · 2020-05-21T07:56:41Z

Hi, @rbradford Thanks for reviewing. I updated the PR as per the comments, and rebased to latest code. Please take a look again.

rbradford

Almost ready to approve. Just want to see some commit message changes.

rbradford · 2020-05-21T09:17:44Z

resources/Dockerfile

@@ -1,5 +1,6 @@
 FROM ubuntu:18.04 as dev



For b40aea0 please prefix first line of commit message with "build:" and drop trailing full-stop.

Please remove the Gerrit Change-Id.

Done for all commits.

rbradford · 2020-05-21T09:19:11Z

arch/src/aarch64/mod.rs

 // SPDX-License-Identifier: Apache-2.0

 pub mod layout;



For 24e62df please prefix with "build:" and drop gerrit Change-Id.

rbradford · 2020-05-21T09:19:58Z

.github/workflows/cross-build.yaml

@@ -0,0 +1,32 @@
+name: Cloud Hypervisor Cross Build


For 888028a please prefix first commit message line with "build: " and drop gerrit Change-Id

Updated Dockerfile to work with multiple architectures. Updated dev_cli.sh to: 1. Build container image before AArch64 image is ready in public. 2. Adjust default feature collection on AArch64. 3. Workaround a build problem with musl on AArch64. Signed-off-by: Michael Zhao <[email protected]>

This is a preparing commit to build and test CH on AArch64. All building issues were fixed, but no functionality was introduced. For X86, the logic of code was not changed at all. For ARM, the architecture specific part is still empty. And we applied some tricks to workaround lint warnings. But such code will be replaced later by other commits with real functionality. Signed-off-by: Michael Zhao <[email protected]>

The result of the workflow can be seen in Checks tab of a PR. Two targets have been added: - stable aarch64-unknown-linux-gnu - stable aarch64-unknown-linux-musl Note: a temporary step was added before building. We used "sed" command to remove "with-serde" feature of kvm-bindings in vmm/Cargo.toml. This step should be removed in future when kvm-bindings is ready. Signed-off-by: Michael Zhao <[email protected]>

rbradford · 2020-05-21T10:32:06Z

.github/workflows/cross-build.yaml

+            target: ${{ matrix.target }}
+            override: true
+      - name: Disable "with-serde" in kvm-bindings
+        run: sed -i 's/"with-serde",\ //g' vmm/Cargo.toml


This is a reasonable solution for the time being.

sboeuf

Nice PR!

michael2012z mentioned this pull request May 14, 2020

Fix the build error with "with-serde" feature in CH branch (on AArch64) cloud-hypervisor/kvm-bindings#1

Merged

michael2012z force-pushed the aarch64-enable-build branch 5 times, most recently from 852871d to a2cb466 Compare May 19, 2020 06:42

michael2012z force-pushed the aarch64-enable-build branch from a2cb466 to 9a2215f Compare May 19, 2020 08:22

rbradford reviewed May 20, 2020

View reviewed changes

michael2012z force-pushed the aarch64-enable-build branch from 9a2215f to 888028a Compare May 21, 2020 07:33

rbradford reviewed May 21, 2020

View reviewed changes

michael2012z added 3 commits May 21, 2020 17:24

michael2012z force-pushed the aarch64-enable-build branch from 888028a to 07f3c9d Compare May 21, 2020 09:27

rbradford reviewed May 21, 2020

View reviewed changes

rbradford approved these changes May 21, 2020

View reviewed changes

rbradford merged commit 4f5c8be into cloud-hypervisor:master May 21, 2020

michael2012z mentioned this pull request May 21, 2020

AArch64 support #1125

Closed

15 tasks

sboeuf reviewed Jun 2, 2020

View reviewed changes

AArch64 initial build #1168

AArch64 initial build #1168

Uh oh!

Conversation

michael2012z commented May 13, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rbradford commented May 13, 2020

Uh oh!

michael2012z commented May 19, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rbradford left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

michael2012z commented May 21, 2020

Uh oh!

rbradford left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sboeuf left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

michael2012z commented May 13, 2020 •

edited

Loading

michael2012z commented May 19, 2020 •

edited

Loading