Consider that retbuf arg can point to GC heap in fgCreateCallDispatcherAndGetResult() #39815

echesakov · 2020-07-23T03:41:13Z

The issue manifests themselves when

methods Callee() and Caller() have the following simple structure

S Callee()
{
  // produce and return instance of S
}

S Caller()
{
    return Callee();
}

S is a value type with non-GC fields and has such size that the result values of Callee and Caller must passed via return buffer;
call to Callee is transformed to tail call with helpers;
Caller is invoked via Reflection.

JIT will transform Caller according to https://github.com/dotnet/runtime/blob/master/docs/design/features/tailcalls-with-helpers.md to

S result;
DispatchTailCalls(&IL_STUB_CallTailCallTarget, (IntPtr)&result, _AddressOfReturnAddress());
return result;

DispatchTailCalls expects result to be a pointer on the stack.

However, during reflection call the return buffer for S value type can be placed on the GC heap

runtime/src/coreclr/src/vm/reflectioninvocation.cpp

Lines 1123 to 1143 in 3665779

    
           if (fHasRetBuffArg) 
        
           { 
        
               // We stack-allocate this ret buff, to preserve the invariant that ret-buffs are always in the 
        
               // caller's stack frame.  We'll copy into gc.retVal later. 
        
               TypeHandle retTH = gc.pSig->GetReturnTypeHandle(); 
        
               MethodTable* pMT = retTH.GetMethodTable(); 
        
               if (pMT->IsStructRequiringStackAllocRetBuf()) 
        
               { 
        
                   SIZE_T sz = pMT->GetNumInstanceFieldBytes(); 
        
                   pRetBufStackCopy = _alloca(sz); 
        
                   memset(pRetBufStackCopy, 0, sz); 
        
                   pValueClasses = new (_alloca(sizeof(ValueClassInfo))) ValueClassInfo(pRetBufStackCopy, pMT, pValueClasses); 
        
                   *((LPVOID*) (pTransitionBlock + argit.GetRetBuffArgOffset())) = pRetBufStackCopy; 
        
               } 
        
               else 
        
               { 
        
                   PVOID pRetBuff = gc.retVal->GetData(); 
        
                   *((LPVOID*) (pTransitionBlock + argit.GetRetBuffArgOffset())) = pRetBuff; 
        
               } 
        
           }

JIT makes a wrong assumption that if Caller and Callee both have return buffer arguments it can be simple passed from the former to the latter in

runtime/src/coreclr/src/jit/morph.cpp

Lines 7907 to 7920 in 3665779

    
           // Use existing retbuf if there is one. 
        
           if (origCall->HasRetBufArg()) 
        
           { 
        
               JITDUMP("Transferring retbuf\n"); 
        
               GenTree* retBufArg = origCall->gtCallArgs->GetNode(); 
        
               assert((info.compRetBuffArg != BAD_VAR_NUM) && retBufArg->OperIsLocal() && 
        
                      (retBufArg->AsLclVarCommon()->GetLclNum() == info.compRetBuffArg)); 
        
               retValArg = retBufArg; 
        
               if (origCall->gtType != TYP_VOID) 
        
               { 
        
                   retVal = gtClone(retValArg); 
        
               } 
        
           }

As a consequence, when GC is called at DispatchTailCalls the object corresponding to the return value buffer can be moved but pointers passed to DispatchTailCalls will not be update that leads to Callee using the old non-updated location when writing the return value.

Solution: JIT needs to account for IsStructRequiringStackAllocRetBuf in fgCreateCallDispatcherAndGetResult in the similar way as it is done in fgMorphCall in

runtime/src/coreclr/src/jit/morph.cpp

Lines 8908 to 8919 in 3665779

    
           if (dest->gtType == TYP_BYREF && !(dest->OperGet() == GT_ADDR && dest->AsOp()->gtOp1->OperGet() == GT_LCL_VAR)) 
        
           { 
        
               // We'll exempt helper calls from this, assuming that the helper implementation 
        
               // follows the old convention, and does whatever barrier is required. 
        
               if (call->gtCallType != CT_HELPER) 
        
               { 
        
                   structHnd = call->gtRetClsHnd; 
        
                   if (info.compCompHnd->isStructRequiringStackAllocRetBuf(structHnd) && 
        
                       !(dest->OperGet() == GT_LCL_VAR && dest->AsLclVar()->GetLclNum() == info.compRetBuffArg)) 
        
                   { 
        
                       // Force re-evaluating the argInfo as the return argument has changed. 
        
                       call->fgArgInfo = nullptr;

Thanks @erozenfeld for help with identifying the place with wrong logic in fgCreateCallDispatcherAndGetResult!

src/tests/JIT/Regression/JitBlue/Runtime_39581/Runtime_39581.ilproj

echesakov · 2020-07-28T19:34:01Z

@erozenfeld PTAL
cc @dotnet/jit-contrib

This reverts commit 8e7b5ae.

…C heap in fgCreateCallDispatcherAndGetResult in morph.cpp

src/coreclr/src/jit/morph.cpp

src/tests/JIT/Regression/JitBlue/Runtime_39581/Runtime_39581.il

src/coreclr/src/jit/morph.cpp

erozenfeld

LGTM with a few minor suggestions.

jakobbotsch · 2020-07-29T22:45:18Z

DispatchTailCalls expects result to be a pointer on the stack.

I don't think this needs to be a hard requirement, if the pointer can be a by-ref to GC heap then the mechanism could pass it through as such.

echesakov · 2020-07-29T23:20:28Z

I don't think this needs to be a hard requirement, if the pointer can be a by-ref to GC heap then the mechanism could pass it through as such.

@jakobbotsch retBufArg points to the stack in most of the cases. Even during Reflection call we might end up with the return buffer allocated on the stack (e.g., when the corresponding return value type has a GC pointer field) rather than GC heap.

I am not sure if making DispatchTailCalls and TailCallCallTargetStub aware that retVal can point to GC heap (i.e. passing this parameter as byref instead of IntPtr) will be more beneficial. GC will need to check that these byref pointers point to GC heap anyway and in most of the cases they will be pointing to the stack.

echesakov · 2020-07-29T23:25:04Z

src/coreclr/src/jit/morph.cpp

        if (origCall->gtType != TYP_VOID)
        {
-            retVal = gtClone(retValArg);


@jakobbotsch Do you remember why this handling was needed here? I understand that on System-V the method that uses return buffer must return an address to the return buffer in rax at the end. But do we use the value in the caller somehow?

I don't know if JIT uses the returned pointer, but even if not it is presumably necessary for reverse pinvoke.
This is already what the JIT does for functions with ret buffers so I just did the same.

echesakov · 2020-07-30T03:53:46Z

All GCStress legs except win-arm64 are green with the change. The ones that failed in win-arm64 are due to #38518

jakobbotsch · 2020-07-30T08:57:20Z

@jakobbotsch retBufArg points to the stack in most of the cases. Even during Reflection call we might end up with the return buffer allocated on the stack (e.g., when the corresponding return value type has a GC pointer field) rather than GC heap.

I am not sure if making DispatchTailCalls and TailCallCallTargetStub aware that retVal can point to GC heap (i.e. passing this parameter as byref instead of IntPtr) will be more beneficial. GC will need to check that these byref pointers point to GC heap anyway and in most of the cases they will be pointing to the stack.

I don't think one more interior pointer should move the needle much and it should simplify the JIT's handling of this.
But the current approach is also fine to me.

…erAndGetResult() (dotnet#39815) Caller return buffer argument can point to GC heap while DispatchTailCalls expects the return value argument to point to the stack. We should use a temporary stack allocated return buffer to hold the value during the dispatcher call and copy the value back to the caller return buffer after that.

echesakov added the NO-REVIEW Experimental/testing PR, do NOT review it label Jul 23, 2020

echesakov mentioned this pull request Jul 23, 2020

[arm64] GCStress=C + JitStress=2 test failures in Vector tests #39581

Closed

danmoseley added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jul 23, 2020

echesakov force-pushed the Runtime_39581 branch from dc20381 to 9a197aa Compare July 24, 2020 20:12

jaredpar mentioned this pull request Jul 24, 2020

OSX machines are de-provisioned during CI / PR runs leading to failures #34472

Closed

BruceForstall reviewed Jul 24, 2020

View reviewed changes

src/tests/JIT/Regression/JitBlue/Runtime_39581/Runtime_39581.ilproj Outdated Show resolved Hide resolved

echesakov force-pushed the Runtime_39581 branch from a08c35c to 3743bd8 Compare July 28, 2020 19:32

echesakov removed the NO-REVIEW Experimental/testing PR, do NOT review it label Jul 28, 2020

echesakov changed the title ~~Add regression test for #39581~~ Consider that retbuf arg can point to GC heap in fgCreateCallDispatcherAndGetResult() Jul 28, 2020

This was linked to issues Jul 28, 2020

[arm64] GCStress=C + JitStress=2 test failures in Vector tests #39581

Closed

GCStress=C + JitStress=2 test failures in Vector tests #39579

Closed

GCStress=C + JitStress=1 test failures in Vector tests #39576

Closed

echesakov added 3 commits July 28, 2020 13:46

Add regression test for dotnet#39581

81252a1

Revert "Disable Vector tests failing under GCStress (dotnet#39580)"

9172d38

This reverts commit 8e7b5ae.

Copy to temporary buffer on stack if a struct can be allocated on a G…

bb6998c

…C heap in fgCreateCallDispatcherAndGetResult in morph.cpp

echesakov force-pushed the Runtime_39581 branch from 3743bd8 to bb6998c Compare July 28, 2020 20:46

erozenfeld requested changes Jul 29, 2020

View reviewed changes

erozenfeld reviewed Jul 29, 2020

View reviewed changes

src/coreclr/src/jit/morph.cpp Outdated Show resolved Hide resolved

erozenfeld approved these changes Jul 29, 2020

View reviewed changes

echesakov added 2 commits July 29, 2020 15:36

Address feedback in Runtime_39581.il

771d80a

Address feedback in morph.cpp

7dcc320

echesakov commented Jul 29, 2020

View reviewed changes

Merge branch 'master' into Runtime_39581

7f8b608

echesakov merged commit 1e23a06 into dotnet:master Jul 30, 2020

echesakov deleted the Runtime_39581 branch July 30, 2020 17:06

echesakov mentioned this pull request Aug 11, 2020

tail. call got stack overflow error !? #40581

Closed

karelz added this to the 5.0.0 milestone Aug 18, 2020

ghost locked as resolved and limited conversation to collaborators Dec 8, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Consider that retbuf arg can point to GC heap in fgCreateCallDispatcherAndGetResult() #39815

Consider that retbuf arg can point to GC heap in fgCreateCallDispatcherAndGetResult() #39815

Uh oh!

echesakov commented Jul 23, 2020 •

edited

Loading

Uh oh!

Uh oh!

echesakov commented Jul 28, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

erozenfeld left a comment

Uh oh!

jakobbotsch commented Jul 29, 2020

Uh oh!

echesakov commented Jul 29, 2020

Uh oh!

echesakov Jul 29, 2020

Uh oh!

jakobbotsch Jul 30, 2020

Uh oh!

echesakov commented Jul 30, 2020

Uh oh!

jakobbotsch commented Jul 30, 2020

Uh oh!

Uh oh!

	if (fHasRetBuffArg)
	{
	// We stack-allocate this ret buff, to preserve the invariant that ret-buffs are always in the
	// caller's stack frame. We'll copy into gc.retVal later.
	TypeHandle retTH = gc.pSig->GetReturnTypeHandle();
	MethodTable* pMT = retTH.GetMethodTable();
	if (pMT->IsStructRequiringStackAllocRetBuf())
	{
	SIZE_T sz = pMT->GetNumInstanceFieldBytes();
	pRetBufStackCopy = _alloca(sz);
	memset(pRetBufStackCopy, 0, sz);

	pValueClasses = new (_alloca(sizeof(ValueClassInfo))) ValueClassInfo(pRetBufStackCopy, pMT, pValueClasses);
	((LPVOID) (pTransitionBlock + argit.GetRetBuffArgOffset())) = pRetBufStackCopy;
	}
	else
	{
	PVOID pRetBuff = gc.retVal->GetData();
	((LPVOID) (pTransitionBlock + argit.GetRetBuffArgOffset())) = pRetBuff;
	}
	}

	// Use existing retbuf if there is one.
	if (origCall->HasRetBufArg())
	{
	JITDUMP("Transferring retbuf\n");
	GenTree* retBufArg = origCall->gtCallArgs->GetNode();
	assert((info.compRetBuffArg != BAD_VAR_NUM) && retBufArg->OperIsLocal() &&
	(retBufArg->AsLclVarCommon()->GetLclNum() == info.compRetBuffArg));

	retValArg = retBufArg;
	if (origCall->gtType != TYP_VOID)
	{
	retVal = gtClone(retValArg);
	}
	}

	if (dest->gtType == TYP_BYREF && !(dest->OperGet() == GT_ADDR && dest->AsOp()->gtOp1->OperGet() == GT_LCL_VAR))
	{
	// We'll exempt helper calls from this, assuming that the helper implementation
	// follows the old convention, and does whatever barrier is required.
	if (call->gtCallType != CT_HELPER)
	{
	structHnd = call->gtRetClsHnd;
	if (info.compCompHnd->isStructRequiringStackAllocRetBuf(structHnd) &&
	!(dest->OperGet() == GT_LCL_VAR && dest->AsLclVar()->GetLclNum() == info.compRetBuffArg))
	{
	// Force re-evaluating the argInfo as the return argument has changed.
	call->fgArgInfo = nullptr;

Consider that retbuf arg can point to GC heap in fgCreateCallDispatcherAndGetResult() #39815

Consider that retbuf arg can point to GC heap in fgCreateCallDispatcherAndGetResult() #39815

Uh oh!

Conversation

echesakov commented Jul 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

echesakov commented Jul 28, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

erozenfeld left a comment

Choose a reason for hiding this comment

Uh oh!

jakobbotsch commented Jul 29, 2020

Uh oh!

echesakov commented Jul 29, 2020

Uh oh!

echesakov Jul 29, 2020

Choose a reason for hiding this comment

Uh oh!

jakobbotsch Jul 30, 2020

Choose a reason for hiding this comment

Uh oh!

echesakov commented Jul 30, 2020

Uh oh!

jakobbotsch commented Jul 30, 2020

Uh oh!

Uh oh!

echesakov commented Jul 23, 2020 •

edited

Loading