Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Try dotnet/java-interop#1302 #9750

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed

Conversation

jonpryor
Copy link
Member

@jonpryor jonpryor commented Feb 4, 2025

Does It Build™?

Does It Build™?
jonpryor added a commit to dotnet/java-interop that referenced this pull request Feb 4, 2025
Context: dotnet/android#9750

dotnet/android#9750 attempts to test #1302, and it
failed to build!

    …/src/Mono.Android/obj/Release/net10.0/android-35/mcw/Java.Util.Jar.Pack200.cs(229,54): error CS1109:
    Extension methods must be defined in a top level static class; IPackerExtensions is a nested class

Indeed, what we actually had was:

    abstract partial class Pack200 {
      public partial interface IPacker { /* … */ }

      static partial class IPackerExtensions {
        public static void Pack (this IPacker packer, Stream output, Stream input)
        {
          …
        }
      }
    }

`static` classes containing extension methods must be top-level,
and `IPackerExtensions` is instead nested within `Pack200`.

Fix `generator` so that this doesn't happen.
jonpryor added a commit to dotnet/java-interop that referenced this pull request Feb 4, 2025
Context: dotnet/android#9750

dotnet/android#9750 attempts to test #1302, and it
failed to build!

    …/src/Mono.Android/obj/Release/net10.0/android-35/mcw/Java.Util.Jar.Pack200.cs(229,54): error CS1109:
    Extension methods must be defined in a top level static class; IPackerExtensions is a nested class

Indeed, what we actually had was:

    abstract partial class Pack200 {
      public partial interface IPacker { /* … */ }

      static partial class IPackerExtensions {
        public static void Pack (this IPacker packer, Stream output, Stream input)
        {
          …
        }
      }
    }

`static` classes containing extension methods must be top-level,
and `IPackerExtensions` is instead nested within `Pack200`.

Fix `generator` so that this doesn't happen.
@jonpryor jonpryor force-pushed the dev/jonp/jonp-try-ji-1302 branch from 417f20f to 25f2cd5 Compare February 4, 2025 20:33
@jonpryor jonpryor force-pushed the dev/jonp/jonp-try-ji-1302 branch from 25f2cd5 to 81c1661 Compare February 4, 2025 20:42
@jonpryor
Copy link
Member Author

jonpryor commented Feb 4, 2025

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Context: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=10952050&view=logs&j=b227c637-c968-57be-6c5e-9409267d8fb0&t=645811be-a64c-5ec6-0b87-989cd8b36278

Trying to build a project with marshal methods enabled fails:

    error XAGJS7000: System.AggregateException: One or more errors occurred. (Object reference not set to an instance of an object.) (Object reference not set to an instance of an object.) (Object reference not set to an instance of an object.) [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:  ---> System.NullReferenceException: Object reference not set to an instance of an object. [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at Xamarin.Android.Tasks.MarshalMethodsClassifier.FindMethod(TypeDefinition type, String methodName, IMethodSignatureMatcher signatureMatcher) in /Users/builder/azdo/_work/12/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Utilities/MarshalMethodsClassifier.cs:line 696 [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at Xamarin.Android.Tasks.MarshalMethodsClassifier.IsStandardHandler(TypeDefinition topType, ConnectorInfo connector, MethodDefinition registeredMethod, MethodDefinition implementedMethod, String jniName, String jniSignature) in /Users/builder/azdo/_work/12/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Utilities/MarshalMethodsClassifier.cs:line 486 [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at Xamarin.Android.Tasks.MarshalMethodsClassifier.IsDynamicallyRegistered(TypeDefinition topType, MethodDefinition registeredMethod, MethodDefinition implementedMethod, CustomAttribute registerAttribute) in /Users/builder/azdo/_work/12/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Utilities/MarshalMethodsClassifier.cs:line 455 [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at Xamarin.Android.Tasks.MarshalMethodsClassifier.ShouldBeDynamicallyRegistered(TypeDefinition topType, MethodDefinition registeredMethod, MethodDefinition implementedMethod, CustomAttribute registerAttribute) in /Users/builder/azdo/_work/12/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Utilities/MarshalMethodsClassifier.cs:line 287 [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at Java.Interop.Tools.JavaCallableWrappers.Adapters.CecilImporter.AddMethod(CallableWrapperType declaringType, TypeDefinition type, MethodDefinition registeredMethod, MethodDefinition implementedMethod, JavaCallableMethodClassifier methodClassifier, IMetadataResolver cache) in /Users/builder/azdo/_work/12/s/xamarin-android/external/Java.Interop/src/Java.Interop.Tools.JavaCallableWrappers/Java.Interop.Tools.JavaCallableWrappers.Adapters/CecilImporter.cs:line 450 [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at Java.Interop.Tools.JavaCallableWrappers.Adapters.CecilImporter.CreateType(TypeDefinition type, IMetadataResolver resolver, CallableWrapperReaderOptions options, String outerType) in /Users/builder/azdo/_work/12/s/xamarin-android/external/Java.Interop/src/Java.Interop.Tools.JavaCallableWrappers/Java.Interop.Tools.JavaCallableWrappers.Adapters/CecilImporter.cs:line 118 [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at Java.Interop.Tools.JavaCallableWrappers.Adapters.CecilImporter.CreateType(TypeDefinition type, IMetadataResolver resolver, CallableWrapperReaderOptions options) in /Users/builder/azdo/_work/12/s/xamarin-android/external/Java.Interop/src/Java.Interop.Tools.JavaCallableWrappers/Java.Interop.Tools.JavaCallableWrappers.Adapters/CecilImporter.cs:line 19 [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at Xamarin.Android.Tasks.JCWGenerator.CreateGenerator(TypeDefinition type, MarshalMethodsClassifier classifier, String monoInit, Boolean hasExportReference, String applicationJavaClass) in /Users/builder/azdo/_work/12/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Utilities/JCWGenerator.cs:line 185 [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at Xamarin.Android.Tasks.JCWGenerator.ProcessTypes(Boolean generateCode, String androidSdkPlatform, MarshalMethodsClassifier classifier, String outputPath, String applicationJavaClass) in /Users/builder/azdo/_work/12/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Utilities/JCWGenerator.cs:line 112 [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at Xamarin.Android.Tasks.JCWGenerator.Classify(String androidSdkPlatform) in /Users/builder/azdo/_work/12/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Utilities/JCWGenerator.cs:line 70 [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at Xamarin.Android.Tasks.GenerateJavaStubs.GenerateJavaSourcesAndMaybeClassifyMarshalMethods(AndroidTargetArch arch, Dictionary`2 assemblies, Dictionary`2 userAssemblies, Boolean useMarshalMethods, Boolean generateJavaCode) in /Users/builder/azdo/_work/12/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Tasks/GenerateJavaStubs.cs:line 407 [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at Xamarin.Android.Tasks.GenerateJavaStubs.<>c__DisplayClass156_0.<Run>b__0(KeyValuePair`2 kvp) in /Users/builder/azdo/_work/12/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Tasks/GenerateJavaStubs.cs:line 212 [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]
    error XAGJS7000:    at System.Threading.Tasks.Parallel.<>c__DisplayClass43_0`2.<PartitionerForEachWorker>b__1(IEnumerator& partitionState, Int64 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) [/Users/runner/work/1/s/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk/Mono.Android.NET-Tests.csproj]

I'm not quite sure *how* we get a null, but updating `ConnectorInfo`
to turn Reflection names into IL names fixes the issue locally.
@jonpryor jonpryor requested a review from grendello as a code owner February 5, 2025 18:30
Changing from IL syntax to Reflection syntax is getting invasive!

I'm starting to wonder if we should do the *opposite*, and only
fix JCWs.
jonpryor added a commit to dotnet/java-interop that referenced this pull request Feb 6, 2025
TODO: full explanation.

#1302 and dotnet/android#9750 has turned into a
bit of a boondoggle.  Maybe the better approach is to just update
`jcw-gen` to use `+` instead of `/` for nested types, instead of
*both* `jcw-gen` *and* `generator`, because the `generator` changes
just require changes *everywhere*.

Will this smaller fix work?
@jonpryor
Copy link
Member Author

jonpryor commented Feb 6, 2025

This approach became an annoying boondoggle once I tried to integrate with Android: #9750

  • The changes to [Register] were (properly!) considered an API breaking change within Mono.Android.dll, which required updating acceptable-breakages-vReference-net10.0.txt to the tune of ~600 (!) entries
  • The changes to [Register] "broke" various pieces of linker-related code, such as MarshalMethodsClassififier and the linker itself. The fact that [Register] contained "IL-syntax" type names made it much easier to lookup types with Cecil. Attempting to change to Reflection syntax required finding all the places that performed such a lookup, which took a while (assuming I even found them all)
  • The changes to [Register] aren't required anyway! As initially mentioned, all existing binding assemblies are using IL syntax, so we have to fix them anyway!

#1304 just updates Java.Interop.Tools.JavaCallableWrappers so that generator output is unchanged. [Register] will continue to use IL nested type syntax, and That's Fine™. It results in a much smaller change to everything else.

@jonpryor jonpryor closed this Feb 6, 2025
jonpryor added a commit to dotnet/java-interop that referenced this pull request Feb 6, 2025
Context: dotnet/android#9747
Context: https://discord.com/channels/732297728826277939/732297837953679412/1336353039031734352
Context: https://discord.com/channels/732297728826277939/732297837953679412/1336358257769316372
Context: #1302
Context: dotnet/android#9750

The `[Register]` attribute provides "connector method" names, and for
interface methods this will also include the name of the type which
declares the method, which itself may be in a nested type:

	namespace Android.App {
	  public partial class Application {
	    public partial interface IActivityLifecycleCallbacks : IJavaObject, IJavaPeerable {
	      [Register (
	          name: "onActivityCreated",
	          signature: "(Landroid/app/Activity;Landroid/os/Bundle;)V",
	          connector: "GetOnActivityCreated_Landroid_app_Activity_Landroid_os_Bundle_Handler:Android.App.Application/IActivityLifecycleCallbacksInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null")]
	      void OnActivityCreated (Android.App.Activity activity, Android.OS.Bundle? savedInstanceState);

	      // …
	    }
	  }
	}

The `connector` value is copied as-is into Java Callable Wrappers,
as part of the `__md_methods` value and `Runtime.register()` call.
Given the C# type:

	partial class MauiApplication : Application {
	  partial class ActivityLifecycleCallbacks : Java.Lang.Object, Application.IActivityLifecycleCallbacks {
	    public void OnActivityCreated (Activity activity, Bundle? savedInstanceState) => …
	  }
	}

then `Java.Interop.Tools.JavaCallableWrappers` will produce:

	// Java Callable Wrapper
	/* partial */ class MauiApplication_ActivityLifecycleCallbacks
	{
	  public static final String __md_methods;
	  static {
	    __md_methods =
	      // …
	      "n_onActivityCreated:(Landroid/app/Activity;Landroid/os/Bundle;)V:GetOnActivityCreated_Landroid_app_Activity_Landroid_os_Bundle_Handler:Android.App.Application/IActivityLifecycleCallbacksInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null\n" +
	      // …
	      "";
	    mono.android.Runtime.register ("Microsoft.Maui.MauiApplication+ActivityLifecycleCallbacks, Microsoft.Maui", MauiApplication_ActivityLifecycleCallbacks.class, __md_methods);
	  }
	}

The `signature` and `connector` values from the `[Register(…)]` on
the method declaration are copied as-is into `__md_methods`.
As the `connector` value contains a `/`, `__md_methods` does as well.

This has worked fine for nearly 15+ years…on Mono/MonoVM.

This *fails* on NativeAOT and CoreCLR, as:

	Type.GetType ("Android.App.Application/IActivityLifecycleCallbacksInvoker, Mono.Android, …", throwOnError:true)

fails with:

	TypeLoadException: Could not resolve type 'Android.App.Application/IActivityLifecycleCallbacksInvoker' in assembly 'Mono.Android, …'.

The reason for the failure is that when using Reflection APIs such as
`Type.GetType()`, the [`Type.AssemblyQualifiedName` grammar][0] must
be followed, and that grammar uses `+` to "Precede a nested class",
*not* `/`.  (`/` isn't even in the Reflection grammar!)

(Aside: where does `/` come from then?  It's the *IL* separator for
nested types!)

For eventual CoreCLR and NativeAOT support, then, we need to replace
`/` with `+` *somewhere* before it hits `Type.GetType()`.

There are (at least?) three places to do so:

 1. Within `JniRuntime.JniTypeManager.RegisterNativeMembers()` or
    equivalent override.

 2. Within `generator`, updating the contents of `[Register]`.

 3. Within `Java.Interop.Tools.JavaCallableWrappers`.

(1) is rejected out of hand as it would be additional work done on-
device at runtime.  Why do that if we don't need to?

(2) was attempted in #1302 & dotnet/android#9750.
It turned into a bit of a boondoggle, because there are lots of
linker steps which interpret the `connector` value on `[Register]`,
and it "just worked" that the `connector` value contained IL names,
as the linker steps deal in IL!  Trying to update `connector` to
instead contain Reflection syntax required finding all the places in
the linker that used `connector` values, which was tedious & brittle.

"Just" (2) is also inadequate, as it would require new binding
assemblies to take advantage of, so (2) *also* needed (3).

Which brings us to (3), the current approach: *don't* alter the
semantics of the `connect` value within `[Register]`, and instead
require that `Java.Interop.Tools.JavaCallableWrappers` replace all
instances of `/` with `+` within `__md_methods`.  This is needed
*anyway*, for compatibility with existing bindings, and also avoids
lots of pain that (2) encountered.

With this approach, `generator` output is unchanged, and
`jcw-gen` output for `MauiApplication.ActivityLifecycleCallbacks`
becomes:

	// Java Callable Wrapper
	/* partial */ class MauiApplication_ActivityLifecycleCallbacks
	{
	  public static final String __md_methods;
	  static {
	    __md_methods =
	      // …
	      "n_onActivityCreated:(Landroid/app/Activity;Landroid/os/Bundle;)V:GetOnActivityCreated_Landroid_app_Activity_Landroid_os_Bundle_Handler:Android.App.Application+IActivityLifecycleCallbacksInvoker, Mono.Android, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null\n" +
	      // …
	      "";
	    mono.android.Runtime.register ("Microsoft.Maui.MauiApplication+ActivityLifecycleCallbacks, Microsoft.Maui", MauiApplication_ActivityLifecycleCallbacks.class, __md_methods);
	  }
	}

[0]: https://learn.microsoft.com/en-us/dotnet/api/system.type.assemblyqualifiedname?view=net-9.0#remarks
@github-actions github-actions bot locked and limited conversation to collaborators Mar 9, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant