[runtime] Rework how the native-to-managed eh callback is called. #5837

vargaz · 2017-10-20T15:08:42Z

Instead of catching all exceptions in the native-to-managed wrapper and calling the callback from the catch clause, call it directly from the EH code. The
previous approach caught all exceptions, which messed up debugging, since the debugger wouldn't break on uncaught exceptions raised from native-to-managed
callbacks.

luhenry · 2017-10-20T16:52:14Z

mono/metadata/marshal.c

We need to remove the finally clause here, as there are 2 possible cases:

there is an exception (and we assume there is no ftnptr_eh_callback): we are going to switch GC unsafe -> safe here so we end up unwinding in GC safe mode which is not allowed.

there is no exception: it works properly

You mean remove the clause but keep the detach call right ?

luhenry · 2017-10-20T16:54:49Z

mono/mini/mini-exceptions.c

We have to call mono_threads_detach_coop here, otherwise the ftnptr_eh_callback will unwind in GC unsafe mode and that can lead to suspend locks (the runtime tries to suspend a thread but it's in GC unsafe mode and it doesn't hit any safe point)

rolfbjarne · 2017-10-26T15:05:07Z

What happens if there are no other managed frames further up the stack? Will mono abort with an unhandled exception message?

luhenry · 2017-10-30T14:28:26Z

@rolfbjarne if there is no managed frames up the stack, there will be an unhandled exception, and that is no different than today.

rolfbjarne · 2017-10-30T18:10:57Z

@luhenry there might be a difference if there's a ftnptr_eh_callback, in which case the managed exception might be converted to a native exception, which native code might catch.

It looks like this PR will cause mono to abort the process before calling ftnptr_eh_callback in that scenario, since it will detect the "no managed frames up the stack" condition first.

luhenry · 2017-12-06T13:55:05Z

@rolfbjarne how should we treat exceptions which: 1. are not handled by a managed frame (because there is "no managed frames up the stack" for example), and 2. are not handled by a native frame? Should we have the native unhandled exception mechanism take precedence over the managed one (ie: report a native unhandled exception over a managed unhandled exception)?

Instead of catching all exceptions in the native-to-managed wrapper and calling the callback from the catch clause, call it directly from the EH code. The previous approach caught all exceptions, which messed up debugging, since the debugger wouldn't break on uncaught exceptions raised from native-to-managed callbacks.

rolfbjarne · 2018-01-09T09:30:15Z

@luhenry

If a managed exception is not handled by a managed frame, and there's no ftnptr_eh_callback callback (and nobody else wants the exception, for instance mono_runtime_invoke can request any managed exceptions, and in this case the mono runtime should consider the exception "handled"), then the mono runtime should start the unhandled managed exception logic:

If there's a debugger attached, notify the debugger.
If no debugger (or the debugger said to continue executing), it should start the unhandled managed exception logic:

mono/mono/metadata/exception.c

Line 1082 in 17a2fba

mono_invoke_unhandled_exception_hook (MonoObject *exc)

. This will terminate the process.

luhenry · 2018-01-09T14:20:40Z

@rolfbjarne ok for the case you presented, but my question is about the case where: 1. the managed exception is not handled by a managed frame, 2. there IS a ftnptr_eh_callback (so the managed exception is wrapped into a native exception and it is passed to the callback), and 3. the native exception is not handled by a native frame?

We need to do the check in the first pass, as pointed out by #5837 (comment)

lambdageek · 2018-03-20T19:29:13Z

@luhenry it looks like @vargaz updated the PR to call out to the hook in the first pass. Is there anything else that needs to change? (I'm ready to push to resolve the merge conflict)

lambdageek · 2018-03-20T19:37:40Z

@monojenkins build coop

luhenry · 2018-03-20T19:57:39Z

I don't remember all of it, but looks like it's ok from what the comments and looking at the code. @rolfbjarne could you confirm it looks good to you too?

lambdageek · 2018-03-20T22:18:05Z

~~Linux x64 Interpreter looks relavant - pinvoke3 asserts: * Assertion at interp/interp.c:1661, condition `frame.ex == NULL' not met~~

Edit: Apparently that's a known/expected interp failure. See #7739

rolfbjarne · 2018-03-20T22:40:02Z

@luhenry yes, from what I can tell (and remember) this looks good now

lambdageek · 2018-03-21T15:03:37Z

@monojenkins build failed

…hrow for PRs

lambdageek · 2018-03-21T15:22:53Z

Merged master into this PR one more time and enabled all the tests that were disabled under coop on PRs.

This should be good to go after CI and a review.

lambdageek · 2018-03-21T21:21:32Z

If we run the installed handler in the first pass, we don't run finally clauses.

lambdageek · 2018-03-21T21:58:04Z

So I think we need to refine the first pass result. Instead of just a boolean we should return some enum:

handled - caught by a managed frame (with a matching filter if one was present).
unhandled-delegate-to-native - there's a native to managed wrapper and a ftnptr_eh_callback is installed.
unhandled - first pass unwound all the way and there was no wrapper or there was no callback installed.

Then the second pass will treat unhandled-delegate-to-native as "handled" (ie, start unwinding, run finally clauses, etc) until we reach the wrapper again at which point we call the callback.

I'm assuming that first pass will need to stop if it sees the wrapper and there's a callback installed.

Does this make sense?

Also: does ftnptr_eh_callback ever return to its caller? I'm assuming no - its always going to be some kind of native throw, right? so we wouldn't ever need to keep unwinding after the wrapper frame?

…econd pass. The constraints are: 1. We must allow managed exceptions to be caught and finally blocks must run 2. We must not call the unmanaged exception handler if the first unwinding pass detects that there is a native-to-managed wrapper and a callback is installed that will handle managed exceptions (by unwinding the native code and either re-raising the managed exception or otherwise dealing with it). So we change the return value of handle_exception_first_pass to be: - MONO_FIRST_PASS_HANDLED - caught by a managed frame (with a matching filter if one was present). - MONO_FIRST_PASS_CALLBACK_TO_NATIVE - there's a native-to-managed wrapper and a ftnptr_eh_callback is installed. - MONO_FIRST_PASS_UNHANDLED - first pass unwound all the way and there was no wrapper or there was no callback installed. In mono_handle_exception_internal, we treat MONO_FIRST_PASS_CALLBACK_TO_NATIVE as handled and unwind the managed stack, invoking finally clauses as we go, until we reach the managed-to-native frame, at which point we invoke the callback.

lambdageek · 2018-03-22T23:49:20Z

Updated:
I added the three kinds of first pass results from my comment, above.

Also added a test case using setjmp/longjmp for the native callbacks.

lambdageek · 2018-03-23T01:28:51Z

The new test needs some love to work in the FullAOT configuration.

lambdageek · 2018-03-23T20:21:24Z

Linux ARMv5, Linux ARMv7 and Linux i386 failures are all #7578.

lambdageek · 2018-03-24T15:53:23Z

@monojenkins bulid failed

lambdageek · 2018-03-24T15:53:58Z

@luhenry One more review?

luhenry · 2018-03-24T23:09:13Z

mono/mini/mini-exceptions.c

 		}
+
+		if (method->wrapper_type == MONO_WRAPPER_NATIVE_TO_MANAGED && ftnptr_eh_callback) {
+			result = MONO_FIRST_PASS_CALLBACK_TO_NATIVE;


I don't know the code enough but shouldn't we return here? It seems to me that, since we found a native-to-managed frame, we don't want to continue looking for a clause that could handle the exception. We don't want to call the callback just yet either because we want to run the finalizers, but the first pass should be finished here IIUC.

@vargaz @alexanderkyte @kumpera could you please confirm? Thank you.

luhenry

You could go for returning TRUE in handle_exception_first_pass and simply keep checks for the wrspper_type in both first and second pass (the first pass would just stop trying to find a catch clause, the second would call the callback)

luhenry · 2018-03-24T23:11:58Z

mono/mini/mini-exceptions.c

 			if (unhandled)
 				mono_debugger_agent_handle_exception ((MonoException *)obj, ctx, NULL, NULL);
-			else
+			else if (res != MONO_FIRST_PASS_CALLBACK_TO_NATIVE)


If we got here, res is MONO_FIRST_PASS_UNHANDLED so this check seems redundant.

I don't think so - this is all in the else branch of if (res == MONO_FIRST_PASS_UNHANDLED), so it could be handled or passed to native.

Oh yes, misread the code.

luhenry · 2018-03-24T23:27:16Z

mono/mini/mini-exceptions.c

 			free_stack = 0xffffff;
 		}

+		if (method->wrapper_type == MONO_WRAPPER_NATIVE_TO_MANAGED && ftnptr_eh_callback) {


MONO_FIRST_PASS_CALLBACK_TO_NATIVE doesn't seem necessary since returning TRUE from handle_exception_first_pass and checking for the wrapper_type again has the same effect.

I think the difference might be in how/when we signal the debugger... we don't want MONO_FIRST_PASS_CALLBACK_TO_NATIVE to tell the debugger that we're handling the exception since the callback is kind of internal to the embedder.

But obviously I haven't written a test for that part, so I'm just guessing.

Yep, I based this comment on https://github.com/mono/mono/pull/5837/files#r176922871 which was wrong, so please ignore this review.

luhenry · 2018-03-26T14:19:57Z

mono/mini/mini-exceptions.c

 			free_stack = 0xffffff;
 		}

+		if (method->wrapper_type == MONO_WRAPPER_NATIVE_TO_MANAGED && ftnptr_eh_callback) {


Yep, I based this comment on https://github.com/mono/mono/pull/5837/files#r176922871 which was wrong, so please ignore this review.

lambdageek · 2018-03-26T17:06:57Z

@monojenkins build failed

…no#5837) * [runtime] Rework how the native-to-managed eh callback is called. Instead of catching all exceptions in the native-to-managed wrapper and calling the callback from the catch clause, call it directly from the EH code. The previous approach caught all exceptions, which messed up debugging, since the debugger wouldn't break on uncaught exceptions raised from native-to-managed callbacks. * [coop] Enable tests bug-58782-plain-throw and bug-58782-capture-and-throw for PRs * [mini] Installed native-to-managed exception handling callback from second pass. The constraints are: 1. We must allow managed exceptions to be caught and finally blocks must run 2. We must not call the unmanaged exception handler if the first unwinding pass detects that there is a native-to-managed wrapper and a callback is installed that will handle managed exceptions (by unwinding the native code and either re-raising the managed exception or otherwise dealing with it). So we change the return value of handle_exception_first_pass to be: - MONO_FIRST_PASS_HANDLED - caught by a managed frame (with a matching filter if one was present). - MONO_FIRST_PASS_CALLBACK_TO_NATIVE - there's a native-to-managed wrapper and a ftnptr_eh_callback is installed. - MONO_FIRST_PASS_UNHANDLED - first pass unwound all the way and there was no wrapper or there was no callback installed. In mono_handle_exception_internal, we treat MONO_FIRST_PASS_CALLBACK_TO_NATIVE as handled and unwind the managed stack, invoking finally clauses as we go, until we reach the managed-to-native frame, at which point we invoke the callback. * [test] Test for mono_install_ftnptr_eh_callback * [test] AOT-friendly test for mono_install_ftnptr_eh_callback

…no/mono#5837) * [runtime] Rework how the native-to-managed eh callback is called. Instead of catching all exceptions in the native-to-managed wrapper and calling the callback from the catch clause, call it directly from the EH code. The previous approach caught all exceptions, which messed up debugging, since the debugger wouldn't break on uncaught exceptions raised from native-to-managed callbacks. * [coop] Enable tests bug-58782-plain-throw and bug-58782-capture-and-throw for PRs * [mini] Installed native-to-managed exception handling callback from second pass. The constraints are: 1. We must allow managed exceptions to be caught and finally blocks must run 2. We must not call the unmanaged exception handler if the first unwinding pass detects that there is a native-to-managed wrapper and a callback is installed that will handle managed exceptions (by unwinding the native code and either re-raising the managed exception or otherwise dealing with it). So we change the return value of handle_exception_first_pass to be: - MONO_FIRST_PASS_HANDLED - caught by a managed frame (with a matching filter if one was present). - MONO_FIRST_PASS_CALLBACK_TO_NATIVE - there's a native-to-managed wrapper and a ftnptr_eh_callback is installed. - MONO_FIRST_PASS_UNHANDLED - first pass unwound all the way and there was no wrapper or there was no callback installed. In mono_handle_exception_internal, we treat MONO_FIRST_PASS_CALLBACK_TO_NATIVE as handled and unwind the managed stack, invoking finally clauses as we go, until we reach the managed-to-native frame, at which point we invoke the callback. * [test] Test for mono_install_ftnptr_eh_callback * [test] AOT-friendly test for mono_install_ftnptr_eh_callback Commit migrated from mono/mono@b20d5ce

vargaz requested review from kumpera and lewurm as code owners October 20, 2017 15:08

dnfclas added the cla-already-signed label Oct 20, 2017

vargaz requested a review from luhenry October 20, 2017 15:08

luhenry suggested changes Oct 20, 2017

View reviewed changes

vargaz force-pushed the 57648-2 branch from 5e7b8b6 to 4f8ba81 Compare October 26, 2017 00:00

vargaz force-pushed the 57648-2 branch from 4f8ba81 to ce305e0 Compare October 28, 2017 06:06

luhenry previously approved these changes Oct 30, 2017

View reviewed changes

ghost removed the cla-already-signed label Nov 16, 2017

luhenry self-assigned this Dec 6, 2017

vargaz force-pushed the 57648-2 branch from ce305e0 to e1735dc Compare January 8, 2018 16:18

marek-safar added the help wanted label Feb 13, 2018

Merge branch 'master' into vargaz-57648-2

3fdef26

marek-safar mentioned this pull request Mar 20, 2018

Hybrid Suspend #6921

Closed

23 tasks

lambdageek added 2 commits March 21, 2018 11:18

Merge branch 'master' into vargaz-57648-2

fe6852c

[coop] Enable tests bug-58782-plain-throw and bug-58782-capture-and-t…

bf33699

…hrow for PRs

luhenry removed the help wanted label Mar 21, 2018

lambdageek added the do-not-merge label Mar 21, 2018

lambdageek added 2 commits March 22, 2018 19:42

[test] Test for mono_install_ftnptr_eh_callback

06da2cf

lambdageek removed the do-not-merge label Mar 22, 2018

lambdageek self-assigned this Mar 22, 2018

lambdageek force-pushed the 57648-2 branch from 5c09c7b to 0c56422 Compare March 23, 2018 15:21

[test] AOT-friendly test for mono_install_ftnptr_eh_callback

a20760c

lambdageek force-pushed the 57648-2 branch from 0c56422 to a20760c Compare March 23, 2018 15:22

luhenry reviewed Mar 24, 2018

View reviewed changes

luhenry approved these changes Mar 26, 2018

View reviewed changes

lambdageek merged commit b20d5ce into mono:master Mar 26, 2018

[runtime] Rework how the native-to-managed eh callback is called. #5837

[runtime] Rework how the native-to-managed eh callback is called. #5837

Uh oh!

Conversation

vargaz commented Oct 20, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rolfbjarne commented Oct 26, 2017

Uh oh!

luhenry commented Oct 30, 2017

Uh oh!

rolfbjarne commented Oct 30, 2017

Uh oh!

luhenry commented Dec 6, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rolfbjarne commented Jan 9, 2018

Uh oh!

luhenry commented Jan 9, 2018

Uh oh!

lambdageek commented Mar 20, 2018

Uh oh!

lambdageek commented Mar 20, 2018

Uh oh!

luhenry commented Mar 20, 2018

Uh oh!

lambdageek commented Mar 20, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rolfbjarne commented Mar 20, 2018

Uh oh!

lambdageek commented Mar 21, 2018

Uh oh!

lambdageek commented Mar 21, 2018

Uh oh!

lambdageek commented Mar 21, 2018

Uh oh!

lambdageek commented Mar 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lambdageek commented Mar 22, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lambdageek commented Mar 23, 2018

Uh oh!

lambdageek commented Mar 23, 2018

Uh oh!

lambdageek commented Mar 24, 2018

Uh oh!

lambdageek commented Mar 24, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

luhenry left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lambdageek commented Mar 26, 2018

Uh oh!

Reviewers

luhenry commented Dec 6, 2017 •

edited

Loading

lambdageek commented Mar 20, 2018 •

edited

Loading

lambdageek commented Mar 21, 2018 •

edited

Loading

lambdageek commented Mar 22, 2018 •

edited

Loading