For all regular test runs and for most debugger-attached test runs, Fixie.TestAdapter runs the test assembly in a separate process for proper isolation and for natural/uneventful loading of dependent assemblies. All Fixie test assemblies are, in fact, console applications that run the tests within. In debugger-attached runs, Fixie can call the VSTest method LaunchProcessWithDebuggerAttached, which is similar to launching a new Process yourself, but with the benefit that it is attached to the very same debugging session that the Test Adapter is using. The overall effect is that breakpoints in your tests get hit as you would expect, without having to encounter the confusing global debugger attachment dialog.

When I first added support for third-party test runners other than Visual Studio Test Explorer, I found LaunchProcessWithDebuggerAttached to be inconsistently supported, and I mistakenly thought this meant the method was using subtle internal details of Visual Studio itself to perform the cross-process attachment, and that its failure under other runners was just a matter of them not being able to call into Visual Studio internals.

Thinking that this was simply a strict limitation, a bit of magic provided within Visual Studio, I introduced a workaround for when LaunchProcessWithDebuggerAttached wasn't expected to succeed: similar to the Fixie v1 use of old .NET Framework AppDomains, we've had a custom AssemblyLoadContext meant to provide comparable in-process assembly loading isolation. It appeared to work for several example Solutions, but any end user of Fixie who wasn't using Visual Studio could easily encounter surprising behavior or exceptions thrown when trying to debug tests.

It is conceivable that the custom AssemblyLoadContext could be improved upon, but this is far too subtle and complex to diagnose and fix with any confidence, so much so that it threatened the long term viability of the project as .NET is now used cross-platform and as more developers consider alternative IDEs. Keeping the AssemblyLoadContext workaround would let it forever remain an intractable bug magnet. Even if it approached correct, we would never know. Even if it reached the highest possible correctness, running within an arbitrary third party's executable like this would still expose us to any number of surprising effects based on what assembly loading trickery that process does before loading Fixie.TestAdapter. Additionally, since VSTest's interfaces are not async, while tests we ultimately call into may be async, this in-process approach was forcing us into an undesirable "sync over async" pattern. In-process runs also break the ability to detect and report StackOverflowExceptions. All in all, this was a dead end.

Unlike my original understanding, LaunchProcessWithDebuggerAttached in fact simply makes a request back to the test runner (VSTest's test host or comparable third-party runner), passing along information similar to ProcessStartInfo (executable, command line arguments, environment variables...). It does so using a message protocol defined by the VSTest team, the same message-passing mechanism as every other interaction with the test runner ("Run These 5 Tests", "This Test Failed", "This Test Passed", etc), and some third-party runners simply do not honor the request. By trying to avoid calling it, Fixie has only really hidden the bugs and feature gaps of third party runners, while creating a source of end user pain and intractable bug reports to this repo.

As a first step toward ensuring absolutely-reliable assembly loading during debugger-attached test runs, even for incomplete third-party runners, here we just drop all in-process running attempts. Subsequent PRs will further improve the situation by better reporting the issue to the user and guiding them to a more sound workaround to keep their own work unblocked.