Initial assesement of the generated code suggests that we will now benefit from a large range of optimizations which result in smaller code size (on average) and reduced stack space usage. However, these measurements may be offset by the GC info size since more methods are now generated as fully interruptible. This needs to be carefully inspected.

Largest regression in size is for the methods that use MethodImplOptions.Synchronized because they now get expanded to use finally blocks with Monitor.Exit like on other platforms.

cc @dotnet/jit-contrib @jkotas @janvorli

@filipnavara Looking forward to this work. It will move us to a place where we can remove the Helper Method Frames in the IL_Throw and IL_Rethrow jithelpers.

See #95695.

Performance testing as of c440a50 shows that we still have bottlenecks in processing of deep stacks. This is not necessarily surprising because the code in EECodeManager for x86 was going to great length to optimize stack unwinding and reduce unnecessary GC info decoding.

using System.Diagnostics;
using System.Runtime.CompilerServices;

internal class Program
{
    static int exceptionsHandled = 0;

    [MethodImpl(MethodImplOptions.NoInlining)]
    private static void CallMe(int i)
    {
        if (i == 0)
        {
            throw new NotImplementedException();
        }

        CallMe(i - 1);
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    private static void CatchMe()
    {
        try
        {
            CallMe(10); // DEPTH: 10, 100, etc.
        }
        catch (NotImplementedException)
        {
            Interlocked.Increment(ref exceptionsHandled);
        }
    }

    private static void ThreadEntrypoint()
    {
        while (true)
        {
            CatchMe();
        }
    }

    private static void Main(string[] args)
    {
        int savedExceptionsHandled = 0;
        for (int i = 0; i < 10; i++)
        {
            new Thread(ThreadEntrypoint).Start();
        }
        Thread.Sleep(1000);
        savedExceptionsHandled = exceptionsHandled;
        Console.WriteLine($"Exceptions handled: {savedExceptionsHandled}");
        Environment.Exit(0);
    }
}

The test code throws an exception at various depths of stack (depending on configuration) and then catches it. This is run in 10 parallel threads and we measure how many exceptions we processed within a second. It's not using a benchmark framework so it includes some warm up and code tiering. I used a Ryzen 9950X machine to run the test, runtime and libraries were built in Release configuration. The table below includes the best value out of 5 consecutive runs, the variation between runs were minimal.

(Update: Added the numbers with PROCESS_EXPLICIT_FRAME_BEFORE_MANAGED_FRAME disabled which is closer to the legacy x86 code paths. This is just meant to measure the impact of always unwinding an extra frame and taking some less optimized code paths. It doesn't work or pass the runtime tests as is.)

This is not necessarily surprising because the code in EECodeManager for x86 was going to great length to optimize stack unwinding and reduce unnecessary GC info decoding.

My current plan is to try adding cache for decoded GC info into EECodeInfo (instead of the existing CodeManState structure; for X86 only).

Update: On its own adding the cache to EECodeInfo has very modest improvement (~5%). It would yeild much bigger improvements if it was paired with PROCESS_EXPLICIT_FRAME_BEFORE_MANAGED_FRAME but making that configuration work is a substantial effort. One thing that did stood out in the profiles are frequent calls to EECodeManager::GetFunctionSize, particularly from JitTokenToMethodRegionInfo which in turn was called from a call chain originating in EECodeInfo::Init. This essentially means that the GC info for each method was almost always decoded at least twice. We can either refactor this to populate the cache early or rewrite GetFunctionSize to decode just the very first word in the x86 GC info. The later option yield another 10% savings.

Update 2: Profiles show that another 10%-15% of the regression is caused by this line:

Update 3: Finally getting to the bottom of this. Seems like the StackTraceArray reallocations cause GC to kick in which stops the world. This is particularly visible in the multi-threaded benchmark. However, at least part of the speed regression exists in single threaded test as well but it's difficult to capture with a profiler since the ETW events start to skew the timings.

I am still working on analyzing the performance so I can paint a complete picture.

I've done a couple of micro-optimizations which help in reducing the stack walks overhead by around 30% with FEATURE_EH_FUNCLETS builds. The biggest one is submitted as PR #114170. I've also tried to micro-optimize few hot code paths (EECodeManager::IsFunclet 466dd7d, context copies d033b7b).

Analysis with a sampling profiler turned out to be difficult. Firstly, the new EH is faster at shallow call stacks and slower at deep call stacks. The implication being that the extra overhead is likely in the stack walking (and AppendExceptionStackFrame). I focused on profiling a code that throws an exception in 100-level deep call stack and then catches it. At the high level I see that some things take 30% more time but at the low level the building blocks are usually so fast that the running time is much smaller than the period of the sampling profiler and it's thus hard to measure accurately.

However, here are few observations:

• Copying the CONTEXT structure from pCalleeContext to pCallerContext has measurable overhead. The structure is huge but we only track the integer+control registers so this is quite wasteful.
• A runtime compiled WITHOUT FEATURE_EH_FUNCLETS but with REGDISPLAY structure changed to match the version with FEATURE_EH_FUNCLETS incurs significant overhead in some scenarios. While I didn't see much difference in the single threaded scenarios, in the multithreaded ones the performance was nearly twice as slow (using the "exceptions handled / second" metric for 10 threads continually throwing and catching exceptions over 100-deep call stack). I didn't study it in detail but I know that StackTraceInfo::AppendElement triggers a garbage collection quite often in this test case and overhead in stack walking contributes significantly to the stop-the-world time, so it's plausible that this could explain the regression.
• With PR Replace CodeManState with cache in EECodeInfo for x86. #114170 we do roughly the same number of GC info decodings on FEATURE_EH_FUNCLETS runtime as the baseline one. Same goes for the number of stack unwind calls. This means we didn't introduce any structural error but unfortunately the individual calls are still so fast that I have trouble accurately measuring whether they are slower.
• Making SfiNext + AppendExceptionStackFrame an FCall makes miniscule difference (~1% give or take) so it's unlikely that P/Invoke transitions are significant contributor to the cost.

Finally I got a profiler tool appropriate for the job. With Intel PIN it's possible to get relatively accurate instruction counts spent in different methods and diff it against the baseline: perf_diff.txt. It needs a bit of a manual analysis since the code paths between the two runtime builds are quite different.

Looks like the best bang for the buck optimizations to try are the following:

• Use NOTIFY_ON_U2M_TRANSITIONS flag in SfiInit and invert the logic in SfiNext to detect unwinding into unmanaged frames. (Use NOTIFY_ON_U2M_TRANSITIONS in SfiInit/SfiNext #114496)
• Get PROCESS_EXPLICIT_FRAME_BEFORE_MANAGED_FRAME working with new EH. There's quite a few places that just assume caller context in REGDISPLAY is valid without checking.

I've managed to get 2 PRs out addressing performance issues.

PR #114580 adds a cache around the behavior of EECodeInfo::IsFunclet. I'm not entirely convinced adding ad hoc caches like this is worth it, but it is a nice performance win that should accrue to all platforms, although especially the Windows platforms.

PR #114582 adds a set of micro-optimizations found by examining the code generated for the Windows X86 platform. Some of these optimizations are isolated to the funclets work, but more of them are general performance improvements to this logic. Again, the optimizations are most important for Windows X86, but should have positive impacts on other platforms as well.

With both of these optimizations, I see 15%ish performance improvements, and with them combined I see performance getting within about 10% of the baseline .NET 9 scenario on Windows X86. If we are able to take both of them, I don't see a reason to hold this feature back based on performance. One detail to be aware of is that since we are changing logic significantly, the current set of profile guided optimization data for Windows X86 will be out of date when we turn this feature on, and it will take weeks to months before we have new data to fix that issue.

The issues that I plan to look at next week are the debugging scenarios, which unfortunately are mostly all only testable via internal test suites. I hope to find some time next week to take a look at them. Notably, I have significant concerns around the behavior of mixed mode stacks/throwing and catching across managed/native boundaries, and such. The contract between the runtime/debugger has historically been fairly fragile in this area, and changing EH models entirely may cause a large problem.

In addition, we'll need to bump the R2R version numbers so that old, non-funclet code won't be accepted by the new runtime, and the issues are the exception handler registrations need to be understood.

I've managed to get 2 PRs out addressing performance issues.

Firstly, huge thanks for looking into the performance issues. There's some small overlap with the optimizations I was testing locally but the two PRs really move the needle. Along with #114496 which was merged yesterday we're looking at roughly +15% instruction count on funclets vs baseline (.NET 10 main) for deep stack exception propagation. There are some smaller micro-optimizations which can reduce that even further, probably worth pursuing but not essential. Notably, the performance actually increases significanly for exception propagation in shallow stack traces. Once all the PRs are reviewed and merged I can run the benchmarks and share some graphs / tables. Notably, #14170 also improved the scenario by a small bit on non-funclet runtime, so comparing to .NET 9 we are now within ~10%, as stated above. I agree with the sentiment that this seems like an acceptable trade-off since we get improved performance and code quality in many other areas and scenarios, including more common exception handling use cases.

The issues that I plan to look at next week are the debugging scenarios, which unfortunately are mostly all only testable via internal test suites.

Help on this is much appreciated! I only did very limited tests with netcoredbg to ensure I'm not completely breaking the world, but that doesn't cover any mixed native/managed exceptions.

In addition, we'll need to bump the R2R version numbers...

Right, I'll bolt that on to the commit in #113576 so we don't forget about it.

Microbenchmarks from dotnet/performance

Tested with commit 199ae88 (main) and the x86funclets branch rebased on top.

Funclets are faster on all of them. None of the benchmarks excercise the pathological paths with deep stack traces. The "deep" stack trace benchmarks are only 10-level deep.

Deep stack trace benchmark

    static int exceptionsHandled = 0;

    [MethodImpl(MethodImplOptions.NoInlining)]
    private static void CallMe(int i, NotImplementedException inEx)
    {
        if (i == 0)
        {
            throw inEx;
        }

        CallMe(i - 1, inEx);
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    private static void CatchMe(NotImplementedException inEx, int depth = 100)
    {
        try
        {
            CallMe(depth, inEx);
        }
        catch (NotImplementedException)
        {
            Interlocked.Increment(ref exceptionsHandled);
        }
    }

    [Params(1, 10, 100, 1000)]
    public int Depth;

    [Benchmark]
    public void CatchMe1000x100()
    {
        CatchMe(new NotImplementedException(), Depth);
    }    

Main:

Funclets:

Handled exceptions/second metric

This is essentially the same non-scientific benchmark shared in #113985 (comment). We run 10 threads, each of them is throwing an exception at deep stack trace and catching it at the top of the stack trace.

This is showing pretty much the worst case scenario (at depth 100) and also the improvements for the more common scenarios. It also shows that we don't introduce any significant bottleneck by taking some additional lock.

using System.Diagnostics;
using System.Runtime.CompilerServices;

public class Program
{
    static int exceptionsHandled = 0;

    [MethodImpl(MethodImplOptions.NoInlining)]
    private static void CallMe(int i, NotImplementedException inEx)
    {
        if (i == 0)
        {
            throw inEx;
        }

        CallMe(i - 1, inEx);
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    private static void CatchMe(NotImplementedException inEx, int depth = 100)
    {
        try
        {
            CallMe(depth, inEx);
        }
        catch (NotImplementedException)
        {
            Interlocked.Increment(ref exceptionsHandled);
        }
    }

    private static void ThreadEntrypoint()
    {
        while (true)
        {
            NotImplementedException inEx = new NotImplementedException();
            CatchMe(inEx, 100);
        }
    }

    public static void Main(string[] args)
    {
        exceptionsHandled = 0; 
    
        Stopwatch sw = Stopwatch.StartNew();
        int savedExceptionsHandled = 0;
        int maxThreads = action == "st" ? 1 : 10;
        for (int i = 0; i < maxThreads; i++)
        {
            new Thread(ThreadEntrypoint).Start();
        }
        Thread.Sleep(1000);
        savedExceptionsHandled = Volatile.Read(ref exceptionsHandled);
        sw.Stop();
        Console.WriteLine($"Exceptions handled: {savedExceptionsHandled}");
        Console.WriteLine($"Time: {sw.Elapsed}");
        Console.WriteLine($"Normalized: {(int)(savedExceptionsHandled * 1000 / sw.ElapsedMilliseconds)}");
        Environment.Exit(0);
    }
}

Depth: 100

Main: 386,344 ex/sec
.NET 9: 378,816 ex/sec
Funclets: 343,814 ex/sec

Depth: 50

Main: 599,575 ex/sec
Funclets: 646,922 ex/sec

Depth: 10

Main: 931,671 ex/sec
Funclets: 1,645,489 ex/sec

The issues that I plan to look at next week are the debugging scenarios, which unfortunately are mostly all only testable via internal test suites. I hope to find some time next week to take a look at them. Notably, I have significant concerns around the behavior of mixed mode stacks/throwing and catching across managed/native boundaries, and such. The contract between the runtime/debugger has historically been fairly fragile in this area, and changing EH models entirely may cause a large problem.

@davidwrighton, would it be possible to get this validated in the current preview time-frame? It will allow us to remove a few more HMFs from jithelpers.