Enable COM interop for collectible classes #20919

janvorli · 2018-11-09T22:27:21Z

This change is relatively straightforward:

Move the ComCallWrapper cache from AppDomain to LoaderAllocator, including the lifetime management logic and update the ComCallWrapper code here and there to be LoaderAllocator bound.
Move the interop data hash table from AppDomain to LoaderAllocator
Modify DispatchMemberInfo to use handles allocated by LoaderAllocator, which lead to some refactoring of the code that accessed the handle.

Closes #20164

jkotas · 2018-11-10T00:15:30Z

Do we need any tests for this?

jkotas · 2018-11-10T00:17:23Z

src/vm/appdomain.cpp

    m_pDefaultContext = NULL;
 #ifdef FEATURE_COMINTEROP
-    m_pComCallWrapperCache = NULL;
    m_pRCWCache = NULL;


Are the RCW caches fine wrt. unloading?

They (or I should rather say it as there is only one in one app domain) don't cause any issues while unloading. It seems that the only reason why they were per AppDomain was the isolation (based on the doc I was able to find in BOTR) and they seem to be cleaned on a regular basis.

E.g. RCW has InterfaceEntry m_aInterfaceEntries[INTERFACE_ENTRY_CACHE_SIZE]; in it. This array contains MethodTable pointers. What guarantees that none of these pointers point to MethodTable that got unloaded?

Let me look into that

GCToEEInterface::AfterGcScanRoots calls AppDomain::DetachRCWs which loops over all entries in the RCW cache and marks all RCWs whose exposed objects were not promoted as detached.
They are then cleaned in RCWCleanupList::CleanupAllWrappers() which is called from SyncBlockCache::CleanupSyncBlocks() which in turn is called from Thread::DoExtraWorkForFinalizer(). This happens right before the call to SystemDomain::System()->ProcessDelayedUnloadLoaderAllocators(), so the RCWs don't outlive the related MethodTable instances.
So unless I am missing something, it seems we should be good.

jkotas · 2018-11-10T00:20:43Z

src/vm/interoputil.cpp

-        //
-        if (pClassMT->Collectible())
-        {
-            COMPlusThrow(kNotSupportedException, W("NotSupported_CollectibleCOM"));


Delete the NotSupported_CollectibleCOM message?

Thanks for spotting that.

janvorli · 2018-11-10T00:41:02Z

We already have a couple of coreclr tests for that. Interop/MarshalAPI/IUnknown and tests in Interop/COM.

davidwrighton · 2018-11-12T23:31:44Z

I'd like to see us have a subset of "interesting" tests for collectible loadercontexts that execute during normal test runs, and don't require some sort of special test pass.

davidwrighton · 2018-11-12T23:40:05Z

It feels like there may be some issue with WinRT. are we sure that no managed WinRT type could be exposed from a collectible type?

janvorli · 2018-11-13T02:07:27Z

@davidwrighton what kind of problems with WinRT do you expect if we exposed managed WinRT objects from collectible types?

src/vm/comcallablewrapper.cpp

src/vm/loaderallocator.cpp

src/vm/interoputil.cpp

janvorli · 2018-11-20T15:03:22Z

@dotnet-bot test CentOS7.1 x64 Checked Innerloop Build and Test please
@dotnet-bot test Ubuntu x64 Checked Innerloop Build and Test please

janvorli · 2018-11-20T15:43:20Z

@dotnet-bot test Windows_NT arm64 Cross Checked Innerloop Build and Test please

janvorli · 2018-11-20T19:36:48Z

@dotnet-bot test Ubuntu x64 Checked Innerloop Build and Test (Jit - TieredCompilation=0) please

janvorli · 2018-11-20T19:37:32Z

@dotnet-bot test Ubuntu x64 Checked Innerloop Build and Test (Jit - TieredCompilation=0) please

janvorli · 2018-11-21T16:58:35Z

@jkotas I believe I've addressed feedback and responded to question. I have also added two unloadability tests. Could you please take another look?

jkotas · 2018-11-21T17:08:22Z

Is it the right design to neuter all COM object references on unload?

AppDomains needed because of they have done forceful unload regardless of whether somebody was holding reference to inside of the AppDomain or not. We are not doing forceful unload here.

I would think that COM object references should be just like any other object references. As long as there are any alive, the load context should stay alive. Once all of them are gone, the load context can be unloaded.

Thoughts?

janvorli · 2018-11-21T17:18:06Z

@jkotas We don't do that. If there are COM references, they keep the LoaderAllocator alive.

jkotas · 2018-11-21T18:34:01Z

src/vm/comcallablewrapper.cpp

-    // we have a template, create a wrapper and initialize from the template
-    // alloc wrapper, aligned 32 bytes
-    if (pWrapperCache->IsDomainUnloading())
+    if (pWrapperCache->IsLoaderAllocatorUnloading())


f there are COM references, they keep the LoaderAllocator alive.

So why do we need to throw AppDomainUnloadedException here?

This is for other purpose. After the unloading is initiated, this code prevents new CCW creation. But the ones that were created before that still work.

Regular P/Invoke interop is allowed even after the unloading is initialized. Why do we want to disallow COM?

I don't have a strong opinion on this. This was the way the AppDomain behaved after the Unload was initiated on them, so I have kept it. But I don't think there would be any issue if we removed this check.

I still have doubts about this. I do not think we should be blocking any interop when the stuff is unloading.

AppDomains had to do it because of they would not be able to forcefully unload otherwise. ALCs do not have forceful unload, so they do not need it.

Ok, let's remove this check.

jkotas · 2018-11-26T08:23:59Z

I have pulled this down to do some ad-hoc testing to see how well it works. I am hitting a crashes when I load&unload a trivial program that uses COM in a loop.

Repro: https://gist.github.com/jkotas/db2536c79dd61899781406af5316037f

Result:

Assert failure(PID 17364 [0x000043d4], Thread: 4896 [0x1320]): Consistency check failed: AV in clr at this callstack:
------
CORECLR! CodeFragmentHeap::RealAllocAlignedMem + 0x79 (0x00007ffe`85237e65)
CORECLR! Precode::AllocateTemporaryEntryPoints + 0xDD (0x00007ffe`85853589)
CORECLR! MethodDescChunk::CreateTemporaryEntryPoints + 0x69 (0x00007ffe`855ba6b5)
CORECLR! MethodDescChunk::EnsureTemporaryEntryPointsCreated + 0x108 (0x00007ffe`855baa00)
CORECLR! MethodDesc::SetTemporaryEntryPoint + 0x21 (0x00007ffe`855c75e9)
CORECLR! ILStubCache::CreateNewMethodDesc + 0x574 (0x00007ffe`858ac418)
CORECLR! ILStubCache::GetStubMethodDesc + 0x434 (0x00007ffe`858ae74c)
CORECLR! NDirect::GetStubMethodDesc + 0x83B (0x00007ffe`855395d3)
CORECLR! ILStubCreatorHelper::HolderEnter + 0x32 (0x00007ffe`8553a8c2)
CORECLR! CreateInteropILStub + 0x53F (0x00007ffe`8552be57)
CORECLR! NDirect::CreateCLRToNativeILStub + 0x6EA (0x00007ffe`8552a492)

* Remove collectible check from IsTypeVisibleFromCom. That fixes three new COM interop tests * Add collectible check to GetComClassFactory when we check for unsupported interop with WinRT * Fix two contracts

Add two tests to test COM unloadability: * One for using native COM server from managed COM client * One for using managed COM objects from native client

It was missing copying the native server dll to the test folder.

janvorli · 2018-11-29T13:38:18Z

@dotnet-bot test CentOS7.1 x64 Checked Innerloop Build and Test please

janvorli · 2018-11-29T14:47:56Z

@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test please
@dotnet-bot test Ubuntu arm Cross Checked Innerloop Build and Test please
@dotnet-bot test Ubuntu arm Cross Checked crossgen_comparison Build and Test please
@dotnet-bot test Ubuntu arm Cross Checked no_tiered_compilation_innerloop Build and Test please
@dotnet-bot test Ubuntu arm Cross Checked no_tiered_compilation_innerloop Build and Test please
@dotnet-bot test Ubuntu x64 Checked CoreFX Tests

janvorli · 2018-11-29T14:52:31Z

@dotnet-bot test Ubuntu arm Cross Release crossgen_comparison Build and Test please
@dotnet-bot test Ubuntu x64 Checked Innerloop Build and Test please
@dotnet-bot test Ubuntu16.04 arm64 Cross Checked Innerloop Build and Test please
@dotnet-bot test Ubuntu16.04 arm64 Cross Checked no_tiered_compilation_innerloop Build and Test please
@dotnet-bot test Windows_NT arm Cross Checked Innerloop Build and Test please
@dotnet-bot test Windows_NT arm Cross Debug Innerloop Build please
@dotnet-bot test Windows_NT arm64 Cross Checked Innerloop Build and Test please
@dotnet-bot test Windows_NT arm64 Cross Debug Innerloop Build please
@dotnet-bot test Windows_NT x64 Checked CoreFX Tests please
@dotnet-bot test Windows_NT x64 full_opt ryujit CoreCLR Perf Tests Correctness please
@dotnet-bot test Windows_NT x64 Checked Innerloop Build and Test (Jit - TieredCompilation=0) please
@dotnet-bot test Windows_NT x86 Checked Innerloop Build and Test (Jit - TieredCompilation=0) please

janvorli · 2018-11-29T14:53:33Z

@dotnet-bot test Windows_NT x86 Release Innerloop Build and Test please

janvorli · 2018-11-29T21:31:14Z

@dotnet-bot test OSX10.12 x64 Checked Innerloop Build and Test please

The NETClientPrimitives is disabled there too.

AaronRobinsonMSFT · 2018-11-30T05:13:27Z

There is some flakyness with this on x86 and ARM. I am unable to reproduce this locally nor is it occurring on any other architectures. I will be fixing the warnings and disabling the test and fixing this tomorrow. See #21294.

AaronRobinsonMSFT · 2018-11-30T06:52:05Z

@dotnet-bot test Windows_NT arm Cross Debug Innerloop Build please
@dotnet-bot test Windows_NT x86 Checked Innerloop Build and Test (Jit - TieredCompilation=0) please

janvorli · 2018-11-30T09:23:29Z

@jkotas I've added the new IUnknownTestInALC that serves as a regression test for the ILStubCache issue and also verified that your example works fine. Do you think we can merge this change now?

jkotas · 2018-11-30T18:14:43Z

I think so. I do not see any other issues. I am sure there are more, but they are hard to find.

janvorli · 2018-12-01T00:00:18Z

@jkotas I've removed the check for the LoaderAllocator unloading and all the stuff that become obsolete due to that.

* Enable COM interop for collectible classes * Modify DispatchInfo to use LoaderAllocator handles The DispatchMemberInfo was using global handles to refer to the managed MemberInfo instances. That doesn't work with unloadability. This change modifies it to use handles allocated from LoaderAllocator. * Disable COM interop for WinRT types * Remove collectible check from IsTypeVisibleFromCom. That fixes three new COM interop tests * Add collectible check to GetComClassFactory when we check for unsupported interop with WinRT * Add COM unloadability tests Add two tests to test COM unloadability: * One for using native COM server from managed COM client * One for using managed COM objects from native client * Add unloading test for IUnknownTest * Disable NETClientPrimitivesInALC on Win ARM The NETClientPrimitives is disabled there too. Commit migrated from dotnet/coreclr@4c461d7

This is a regression from NETFX with the introduction of collectable assemblies in .NET Core 3.1 (dotnet/coreclr#20919).

This is a regression from NETFX with the introduction of collectable assemblies in .NET Core 3.1 (dotnet/coreclr#20919). Co-authored-by: Aaron Robinson <[email protected]>

janvorli added the area-VM label Nov 9, 2018

janvorli added this to the 3.0 milestone Nov 9, 2018

janvorli self-assigned this Nov 9, 2018

janvorli requested a review from jkotas November 9, 2018 22:27

jkotas requested a review from AaronRobinsonMSFT November 10, 2018 00:09

jkotas reviewed Nov 10, 2018

View reviewed changes

AaronRobinsonMSFT reviewed Nov 13, 2018

View reviewed changes

src/vm/comcallablewrapper.cpp Show resolved Hide resolved

AaronRobinsonMSFT reviewed Nov 13, 2018

View reviewed changes

src/vm/comcallablewrapper.cpp Show resolved Hide resolved

AaronRobinsonMSFT reviewed Nov 13, 2018

View reviewed changes

src/vm/comcallablewrapper.cpp Show resolved Hide resolved

AaronRobinsonMSFT reviewed Nov 13, 2018

View reviewed changes

src/vm/loaderallocator.cpp Show resolved Hide resolved

AaronRobinsonMSFT approved these changes Nov 13, 2018

View reviewed changes

janvorli force-pushed the enable-collectible-com-interop branch from ece8b8d to f56a91d Compare November 16, 2018 02:05

AaronRobinsonMSFT reviewed Nov 19, 2018

View reviewed changes

src/vm/interoputil.cpp Outdated Show resolved Hide resolved

janvorli force-pushed the enable-collectible-com-interop branch from f56a91d to 66ecc14 Compare November 20, 2018 01:21

janvorli force-pushed the enable-collectible-com-interop branch from b30aafa to 7aa7836 Compare November 21, 2018 16:57

jkotas reviewed Nov 21, 2018

View reviewed changes

janvorli added 6 commits November 29, 2018 11:21

Disable COM interop for WinRT types

06cc375

Use .def file for the new Crst

f3dd914

Couple of fixes

2909795

* Remove collectible check from IsTypeVisibleFromCom. That fixes three new COM interop tests * Add collectible check to GetComClassFactory when we check for unsupported interop with WinRT * Fix two contracts

Add COM unloadability tests

b379ecb

Add two tests to test COM unloadability: * One for using native COM server from managed COM client * One for using managed COM objects from native client

Fix newly added test

4960976

It was missing copying the native server dll to the test folder.

Add unloading test for IUnknownTest

4e53c70

janvorli force-pushed the enable-collectible-com-interop branch from 3d43ccf to 4e53c70 Compare November 29, 2018 10:23

Disable NETClientPrimitivesInALC on Win ARM

06c6367

The NETClientPrimitives is disabled there too.

Remove check of ALC unloading from CopyFromTemplate

0081ac5

jkotas approved these changes Dec 1, 2018

View reviewed changes

janvorli merged commit 4c461d7 into dotnet:master Dec 1, 2018

janvorli deleted the enable-collectible-com-interop branch December 1, 2018 09:31

tannergooding mentioned this pull request Dec 3, 2018

Update CoreClr, CoreFx, CoreSetup, ProjectNTfs, ProjectNTfsTestILC to preview-27204-02, preview.18604.2, preview-27204-02, beta-27204-00, beta-27203-01, respectively (master) dotnet/corefx#33783

Merged

AaronRobinsonMSFT mentioned this pull request Oct 27, 2022

Memory leak in COM interop (IDispatch↔IReflect) dotnet/runtime#76350

Closed

AaronRobinsonMSFT added a commit to AaronRobinsonMSFT/runtime that referenced this pull request Oct 27, 2022

Free LOADERHANDLE DispatchMemberInfo dtor

4a46a0e

This is a regression from NETFX with the introduction of collectable assemblies in .NET Core 3.1 (dotnet/coreclr#20919).

AaronRobinsonMSFT mentioned this pull request Oct 27, 2022

Free LOADERHANDLE DispatchMemberInfo dtor dotnet/runtime#77554

Merged

AaronRobinsonMSFT added a commit to dotnet/runtime that referenced this pull request Oct 27, 2022

Free LOADERHANDLE DispatchMemberInfo dtor (#77554)

9e1350e

This is a regression from NETFX with the introduction of collectable assemblies in .NET Core 3.1 (dotnet/coreclr#20919).

github-actions bot pushed a commit to dotnet/runtime that referenced this pull request Oct 27, 2022

Free LOADERHANDLE DispatchMemberInfo dtor

4253c20

This is a regression from NETFX with the introduction of collectable assemblies in .NET Core 3.1 (dotnet/coreclr#20919).

Enable COM interop for collectible classes #20919

Enable COM interop for collectible classes #20919

Uh oh!

Conversation

janvorli commented Nov 9, 2018

Uh oh!

jkotas commented Nov 10, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

janvorli commented Nov 10, 2018

Uh oh!

davidwrighton commented Nov 12, 2018

Uh oh!

davidwrighton commented Nov 12, 2018

Uh oh!

janvorli commented Nov 13, 2018

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

janvorli commented Nov 20, 2018

Uh oh!

janvorli commented Nov 20, 2018

Uh oh!

janvorli commented Nov 20, 2018

Uh oh!

janvorli commented Nov 20, 2018

Uh oh!

janvorli commented Nov 21, 2018

Uh oh!

jkotas commented Nov 21, 2018

Uh oh!

janvorli commented Nov 21, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jkotas commented Nov 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

janvorli commented Nov 29, 2018

Uh oh!

janvorli commented Nov 29, 2018

Uh oh!

janvorli commented Nov 29, 2018

Uh oh!

janvorli commented Nov 29, 2018

Uh oh!

janvorli commented Nov 29, 2018

Uh oh!

AaronRobinsonMSFT commented Nov 30, 2018

Uh oh!

AaronRobinsonMSFT commented Nov 30, 2018

Uh oh!

jkotas commented Nov 26, 2018 •

edited

Loading