Switch mono to use cmake-detected objcopy #83903

am11 · 2023-03-24T21:36:42Z

We use configuretools.cmake in coreclr, native libs (for both runtimes), corehost and tests to locate tools most suitable for the selected toolchain. However for mono, we manually specify a hardcoded list of objcopy names which require extra step in each cross Dockerfile (e.g. when distro package provides cross arch binutils and when distro package doesn't provide one). This manual work is redundant and adds complexity for someone working on porting runtime to a new platform or creating a docker image with runtime build prereqs.

This PR brings mono to the same plan as rest of the native subsets by re-using configuretools.cmake in mono for objcopy detection and install_with_stripped_symbols function out of functions.cmake, which takes care of stripping the binaries without dealing with platform differences at call-sites.

ps - moving mono to fully utilize configurecompiler.cmake as rest of the native subsets is ultimate goal, but it is non-trivial to accomplish in a single PR.

am11 · 2023-03-26T06:04:03Z

cc @lambdageek, @akoeplinger

lambdageek · 2023-03-27T14:02:28Z

@directhex could you take a look - particularly with an eye toward monoaotcross

lambdageek · 2023-03-27T14:02:38Z

/azp run runtime-extra-platforms

azure-pipelines · 2023-03-27T14:03:05Z

Azure Pipelines successfully started running 1 pipeline(s).

directhex · 2023-03-27T14:05:21Z

/azp run runtime-community

azure-pipelines · 2023-03-27T14:05:31Z

Azure Pipelines successfully started running 1 pipeline(s).

directhex · 2023-03-27T15:05:41Z

Have we validated that binaries are indeed still stripped? We aren't failing the build, at least

am11 · 2023-04-07T16:18:11Z

Have we validated that binaries are indeed still stripped? We aren't failing the build, at least

@directhex, I have validated that it is stripping the symbols. It shows up in build logs like this:

  Stripping symbols from mono-aot-cross into mono-aot-cross.dbg

(similar to other subsets' builds like libs.native)

akoeplinger

A few comments but looks good otherwise

src/mono/mono.proj

akoeplinger · 2023-04-12T15:34:51Z

src/mono/mono.proj

    <ItemGroup>
      <_MonoCMakeArgs Include="-DMONO_COMPONENTS_RID=$(TargetOS)-$(TargetArchitecture)" />
+      <_MonoCMakeArgs Condition="'$(TargetArchitecture)' == 'wasm'" Include="-DCLR_CMAKE_HOST_ARCH=$(BuildArchitecture)" />
+      <_MonoCMakeArgs Condition="'$(TargetArchitecture)' != 'wasm'" Include="-DCLR_CMAKE_HOST_ARCH=$(TargetArchitecture)" />


can you add a comment explaining why we have this difference?

Added a comment. It is the case until we completely switch mono to use the common infra.

Thanks, but it is still not really clear to me why this is different on wasm vs. non-wasm?

I copied this existing condition from this same file. There is no comment on line 769 either. Without this, wasm builds fail. So the comment I've added is based on my understanding of the situation. I'm sure there is a better way to elaborate it, but depending on the audience, I guess it will still fall short until we explain everything. 😅

Wasm is always cross-compiled; meaning host is always different than target. But when we add build dimension; i.e. we are in our other cross-compiling environment (e.g. linux-x64 (the system running build.sh) -> linux-arm64 (for which the cross/aot tools are compiled in stage0) -> wasm (the target emitted by those cross/aot tools in stage 1)) then host != target != build holds and it fails wasm builds without this flip.

In coreclr, tools like crossgen2 are built for multi-tagets, so we can do things like crossgen2 --targetarch arm64 on x64 machine. That cuts off the requirement of dealing with the third dimension in build infra. I suppose mono cross/aot tools are not on such multi-target plan.

Ah sorry I didn't realize this was existing code. Looks like this was added in 81724f1.

Hey 81724f1 is some of my best work!

src/mono/mono.proj

akoeplinger · 2023-04-13T10:04:25Z

src/mono/mono.proj

    <ItemGroup>
      <_MonoCMakeArgs Include="-DMONO_COMPONENTS_RID=$(TargetOS)-$(TargetArchitecture)" />
+      <_MonoCMakeArgs Condition="'$(TargetArchitecture)' == 'wasm'" Include="-DCLR_CMAKE_HOST_ARCH=$(BuildArchitecture)" />
+      <_MonoCMakeArgs Condition="'$(TargetArchitecture)' != 'wasm'" Include="-DCLR_CMAKE_HOST_ARCH=$(TargetArchitecture)" />


Thanks, but it is still not really clear to me why this is different on wasm vs. non-wasm?

akoeplinger · 2023-04-13T10:05:21Z

src/mono/mono.proj

          SkipUnchangedFiles="true"
          Condition="'$(BuildMonoAOTCrossCompilerOnly)' != 'true'" />
-
-    <Exec Condition="'$(BuildMonoAOTCrossCompilerOnly)' != 'true' and '$(MonoGenerateOffsetsOSGroups)' == '' and ('$(TargetsOSX)' == 'true' or '$(TargetsMacCatalyst)' == 'true' or '$(TargetsiOS)' == 'true' or '$(TargetstvOS)' == 'true')" Command="install_name_tool -id @rpath/$(MonoFileName) $(RuntimeBinDir)$(MonoFileName)" />


did you verify that the rpath is correct after this change?

I confirmed this works because we pass -DMONO_SHARED_LIB_NAME=$(MonoSharedLibName) into cmake now so the filename is correct from the beginning (and thus the rpath).

TBH, I mainly relied upon the CI for this one. We switched cmake install with our wrapper install_with_stripped_symbols, which is used for all binaries for coreclr, libs and corehost.

Now that you have mentioned it, I tested it on osx-arm64 and otool gives me the same output before (main) and after (PR):

$ ./build.sh mono -c Release $ otool -l artifacts/bin/mono/osx.arm64.Release/libcoreclr.dylib [...] Load command 4 cmd LC_ID_DYLIB cmdsize 48 name @rpath/libcoreclr.dylib (offset 24) time stamp 1 Thu Jan 1 02:00:01 1970 current version 2.0.0 [...]

If the passing CI legs do not indicate that iOS / other targets are in the clear and this testing (on osx-arm64) is not enough, I can try to build for those platforms as well?

I looked through the failures on runtime-extra-platforms and none of them seem to be related :)

akoeplinger · 2023-04-13T17:43:21Z

/azp run runtime-extra-platforms

azure-pipelines · 2023-04-13T17:43:44Z

Azure Pipelines successfully started running 1 pipeline(s).

akoeplinger · 2023-04-14T10:21:13Z

Thank you!

This got broken by dotnet#83903 and wasn't noticed on the PR because we don't build .frameworks unless you pass `/p:BuildDarwinFrameworks=true`. For .frameworks we need to use a different install command and we also don't need to codesign them, it actually causes an error trying to do so: > /Users/alexander/dev/runtime/artifacts/obj/mono/maccatalyst.x64.Release/mono/mini/Mono.release.framework/Versions/C/Mono.release: code object is not signed at all > In subcomponent: /Users/alexander/dev/runtime/artifacts/obj/mono/maccatalyst.x64.Release/mono/mini/Mono.release.framework/Versions/C/Mono.release.dwarf

This got broken by #83903 and wasn't noticed on the PR because we don't build .frameworks unless you pass `/p:BuildDarwinFrameworks=true`. For .frameworks we need to use a different install command and we also don't need to codesign them, it actually causes an error trying to do so (only on Catalyst though, not on iOS/tvOS for some reason): > /Users/alexander/dev/runtime/artifacts/obj/mono/maccatalyst.x64.Release/mono/mini/Mono.release.framework/Versions/C/Mono.release: code object is not signed at all > In subcomponent: /Users/alexander/dev/runtime/artifacts/obj/mono/maccatalyst.x64.Release/mono/mini/Mono.release.framework/Versions/C/Mono.release.dwarf

vargaz · 2023-04-18T20:00:28Z

This caused mono runtime builds to be stripped at build time, i.e. running ninja causes a stripped mono-sgen to be generated even in debug configuration.

am11 · 2023-04-18T20:13:53Z

@vargaz, in ./build.sh mono+libs+clr+host, the strip mechanism used for libs, host and clr is now used for mono. If we are fine for shared object out of libs.native and other subsets to be stripped in debug configuration as well as release alike, we should be OK with that behavior in mono; right? If not, we can set KeepNativeSymbols to true for debug config.

vargaz · 2023-04-18T20:18:43Z

The problem is that the stripping is done during the build, not the install/publish etc. step, so even the local binaries under 'obj/mono/' are stripped, which shouldn't happen.

am11 · 2023-04-18T20:38:05Z

Is there any benefit in having unstripped binary under obj/ and stripped under bin/, even when there is ./build.sh mono -p:KeepNativeSymbols=true option for someone needing fat binaries? AFAIK, debuggers read symbol files with no issue in the installed location.

My point was that this is how the other native binaries built from this repo are treated. Granted it is different than what we had for mono before, but consistent with rest of the repo.

vargaz · 2023-04-18T20:43:45Z

Building directly under artifacts/obj using ninja avoids the time overhead of running build.sh.
Also, stripping files during a build is not how unix development is traditionally done, they should be stripped when they are installed/published into bin/ or the runtime pack.

ghost added community-contribution Indicates that the PR has been added by a community member area-Build-mono labels Mar 24, 2023

Switch mono to use cmake-detected objcopy

0913ff2

am11 force-pushed the feature/build/mono-objcopy branch from 0ff0fa2 to 0913ff2 Compare March 26, 2023 06:03

am11 marked this pull request as ready for review March 26, 2023 06:03

am11 requested review from thaystg, lambdageek, vargaz, SamMonoRT, lateralusX and marek-safar as code owners March 26, 2023 06:03

am11 mentioned this pull request Mar 26, 2023

Upgrade arm64 cross image to Ubuntu 22.04 / llvm-15 #83939

Closed

Fix a condition

2111ae1

build-analysis bot mentioned this pull request Mar 27, 2023

CI failure STRICT_JS doesn't work with MODULARIZE or EXPORT_ES6 #83986

Closed

am11 added 5 commits March 28, 2023 01:24

Handle wasi case

5d3fe81

Revert static lib change

e552b0f

typo

bdf5d80

revert static lib change

ab5994d

revert static lib change

3c2a67f

build-analysis bot mentioned this pull request Apr 4, 2023

CI System.Text.RegularExpressions.Tests failure #84323

Closed

am11 mentioned this pull request Apr 8, 2023

Build on CBL-mariner host with rootfs #84148

Merged

akoeplinger reviewed Apr 12, 2023

View reviewed changes

am11 added 4 commits April 12, 2023 22:02

Address CR feedback

5f0b20c

Merge dotnet/main into feature/build/mono-objcopy

51ffe9b

.

7c830a7

Merge dotnet/main again...

ada35bc

am11 closed this Apr 13, 2023

am11 reopened this Apr 13, 2023

build-analysis bot mentioned this pull request Apr 13, 2023

WasmTestOnBrowser-System.Text.Json.Tests.WorkItemExecution timing out #84434

Closed

akoeplinger approved these changes Apr 13, 2023

View reviewed changes

akoeplinger merged commit 330d46a into dotnet:main Apr 14, 2023

akoeplinger mentioned this pull request Apr 14, 2023

Fix MacCatalyst .framework build #84858

Merged

am11 deleted the feature/build/mono-objcopy branch April 18, 2023 20:06

ghost locked as resolved and limited conversation to collaborators May 19, 2023

Switch mono to use cmake-detected objcopy #83903

Switch mono to use cmake-detected objcopy #83903

Uh oh!

Conversation

am11 commented Mar 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

am11 commented Mar 26, 2023

Uh oh!

lambdageek commented Mar 27, 2023

Uh oh!

lambdageek commented Mar 27, 2023

Uh oh!

azure-pipelines bot commented Mar 27, 2023

Uh oh!

directhex commented Mar 27, 2023

Uh oh!

azure-pipelines bot commented Mar 27, 2023

Uh oh!

directhex commented Mar 27, 2023

Uh oh!

am11 commented Apr 7, 2023

Uh oh!

akoeplinger left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

am11 Apr 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akoeplinger commented Apr 13, 2023

Uh oh!

azure-pipelines bot commented Apr 13, 2023

Uh oh!

akoeplinger commented Apr 14, 2023

Uh oh!

vargaz commented Apr 18, 2023

Uh oh!

am11 commented Apr 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vargaz commented Apr 18, 2023

Uh oh!

am11 commented Apr 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vargaz commented Apr 18, 2023

Uh oh!

Uh oh!

am11 commented Mar 24, 2023 •

edited

Loading

am11 Apr 13, 2023 •

edited

Loading

am11 commented Apr 18, 2023 •

edited

Loading

am11 commented Apr 18, 2023 •

edited

Loading