Include default RSP files for core compilers #80993

jjonescz · 2025-11-03T12:53:52Z

Follow up on #79911.

Fixes another inconsistency between the core and fx compilers: if the <Csc> task is used manually on a C# file which has using System.Linq; (for example), the fx compiler handles that fine (because there is csc.rsp deployed with it which has a set of default references) whereas the core compiler fails. A real-world use-case can be seen in https://dev.azure.com/devdiv/DevDiv/_git/VS/pullrequest/684436.

The fix is to include this default csc.rsp in scenarios where we invoke the core compiler on fx msbuild (to avoid a break there). We continue to not include that csc.rsp in scenarios where just the core csc is being used (again, to avoid a break in those scenarios).

@jaredpar fyi

AlekseyTs · 2025-11-06T14:57:45Z

src/Compilers/Core/MSBuildTask/Sdk/Microsoft.Build.Tasks.CodeAnalysis.Sdk.csproj

  </ItemGroup>

+  <ItemGroup>
+    <None Include="$(MSBuildThisFileDirectory)..\..\..\CSharp\csc\csc.rsp" CopyToOutputDirectory="PreserveNewest" />


"PreserveNewest"

Not sure how this option works. It feels like we should always overwrite unless exact match

This is inspired by other projects that include csc.rsp:

roslyn/src/Compilers/CSharp/csc/AnyCpu/csc.csproj

Line 14 in 3898dec

<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>

It works by not copying the file if it's not modified, so the builds are more performant

This is inspired by other projects that include csc.rsp

That is great, but I still would like to understand how exactly this option works. Other projects could have made a mistake. Since these files are not build artifacts, it doesn't feel appropriate to me to copy files only when they are newer. BTW, what exactly does it mean for a file to be newer for the purpose of this operation?

The file is copied to the output directory only if the source file has newer timestamp than the target file (the one in the output directory of the project). I think this is the common default for many MSBuild operations.

It's documented here: https://learn.microsoft.com/en-us/visualstudio/msbuild/common-msbuild-project-items?view=vs-2022#none

CopyToOutputDirectory Optional string. Determines whether to copy the file to the output directory. Values are: 1. Never 2. Always 3. PreserveNewest 4. IfDifferent Defaults to Never if DefineExplicitDefaults is set to true; otherwise, defaults to the empty string.

Looks like we could use IfDifferent (but that's a newer option: dotnet/docs#44059); or Always.

AlekseyTs · 2025-11-06T15:00:46Z

src/Compilers/Core/MSBuildTaskTests/TestUtilities/IntegrationTestBase.cs

                        <UsingTask TaskName="Microsoft.CodeAnalysis.BuildTasks.Csc" AssemblyFile="{_buildTaskDll}" />
                        <Target Name="CustomTarget">
-                            <Csc Sources="File.cs" UseSharedCompilation="{useSharedCompilation}" DisableSdkPath="{disableSdkPath}" />
+                            <Csc Sources="File.cs" UseSharedCompilation="{useSharedCompilation}" DisableSdkPath="{disableSdkPath}" NoConfig="{noConfig}" />


PairwiseData

What does this do and why the change is necessary?

It avoids exponential amount of test runs since I added new input. See https://aarnott.github.io/Xunit.Combinatorial/docs/combinatorial-vs-pairwise.html. The pairwise ones should be the interesting ones (I've verified the test still fails without the product change for example).

I do not find 8 cases too much and would be more comfortable with covering full matrix. I am uncomfortable with a situation when an "external" tool decides what combinations our tests cover and that combinations possibly changing between versions of that tool. My recommendation is to use full matrix or explicitly specifying combinations covered by the test.

AlekseyTs · 2025-11-06T15:02:24Z

        Assert.Contains("error CS", result.Output);

Could we use more precise validation like we have for VB? #Closed

Refers to: src/Compilers/Core/MSBuildTaskTests/TestUtilities/IntegrationTestBase.cs:221 in 35e173a. [](commit_id = 35e173a, deletion_comment = False)

AlekseyTs · 2025-11-06T15:09:34Z

src/Compilers/Core/MSBuildTask/Csc.cs

+                    var rspFile = Path.Combine(Path.GetDirectoryName(typeof(ManagedCompiler).Assembly.Location)!, "csc.rsp");
+                    if (File.Exists(rspFile))
+                    {
+                        commandLine.AppendSwitchIfNotNull("@", rspFile);


"@"

It doesn't look like we are testing scenario when an .rsp file is already specified #Closed

AlekseyTs · 2025-11-06T15:09:56Z

Done with review pass (commit 5)

AlekseyTs · 2025-11-07T14:33:48Z

src/Compilers/Core/SdkTaskTests/CscTests.cs

        };

-        AssertEx.Equal($"/sdkpath:{RuntimeEnvironment.GetRuntimeDirectory()} /out:test.exe test.cs", csc.GenerateResponseFileContents());
+        AssertEx.Equal($"/sdkpath:{RuntimeEnvironment.GetRuntimeDirectory()} @{RspFilePath} /out:test.exe test.cs", csc.GenerateResponseFileContents());


csc.GenerateResponseFileContents()

Consider covering "noconfig" behavior as well

AlekseyTs · 2025-11-07T14:34:10Z

src/Compilers/Core/SdkTaskTests/VbcTests.cs

        };

-        AssertEx.Equal($"/sdkpath:{RuntimeEnvironment.GetRuntimeDirectory()} /optionstrict:custom /out:test.exe test.vb", vbc.GenerateResponseFileContents());
+        AssertEx.Equal($"/sdkpath:{RuntimeEnvironment.GetRuntimeDirectory()} @{RspFilePath} /optionstrict:custom /out:test.exe test.vb", vbc.GenerateResponseFileContents());


GenerateResponseFileContents

Consider covering "noconfig" behavior as well

AlekseyTs · 2025-11-07T14:35:30Z

Done with review pass (commit 6)

jaredpar · 2025-11-11T15:12:01Z

src/Compilers/Core/MSBuildTask/Csc.cs

        protected override void AddResponseFileCommands(CommandLineBuilderExtension commandLine)
        {
-            // Pass sdkpath if we are invoking core compiler from framework to preserve the behavior that framework compiler would have.
+            // Pass sdkpath and csc.rsp if we are invoking core compiler from framework to preserve the behavior that framework compiler would have.


How is the defaut RSP file specified in the framework based build?

It is deployed alongside csc.exe. The compiler then checks the file exists and includes it automatically:

roslyn/src/Compilers/Core/Portable/CommandLine/CommonCompiler.cs

Lines 130 to 133 in a065eae

if (!SuppressDefaultResponseFile(args) && File.Exists(responseFile))

{

allArgs = new[] { "@" + responseFile }.Concat(allArgs);

}

I first tried to do that for the core compiler too but that would be a breaking change (could cause errors if the DLLs specified in the response file do not exist).

Overall I get what this PR is trying to accomplish and why we need to do it. What I'm trying to square in my mind is how we limit it to the cases that need it and think through the cases going forward.

Short term you're attaching this logic to coming from the bridge task which makes sense. And really the only way I can see us getting this in now is doing exactly this. In the long term though I'm not sure that is the right approach. Love or hate the raw <Csc> in targets files customers do it. As customers migrate from msbuild to dotnet build I'm guessing the need for raw <Csc> in targets will remain. Hence over the long term I think we will need to support this when running under dotnet build. That is harder because there is no way, today at least, to find the sdk reference path (see teams chat). That means we can't do a no args compile in dotnet build.

Basically I think in the long term we'll need support for the dotnet build infra to provide path to the appropriate Microsoft.NetCore.App.Ref package. That would then translate to /sdkpath and we could have a simple default .rsp for the core compiler. Then <Csc> would work there. This is a change across a few components so we probably should start tracking that.

Now that I typed a lot ... back to what to do about this PR

I'm thinking that we should probably only do this when:

We are in the bridge task

There is no target framework specified. At the point there is a TF then the default RSP is probably not right which is why we always pass /noconfig

The NoConfig is not passed (yes a double negative)

@rainersigwald, @baronfel this is why I've been aksing about finding the sdk path inside of dotnet build

2. There is no target framework specified. At the point there is a TF then the default RSP is probably not right which is why we always pass /noconfig

I'm not sure what you mean by this. The Csc task doesn't see the current TargetFramework (and adding a new parameter for it could introduce more breaks as we seen recently). And the SDK passes NoConfig=true when TF is specified, so it sounds like (3) supersedes (2)?

Regarding (1) and (3), those are what this PR indeed checks. Specifically, only the bridge task will include the default csc.rsp if it exists and NoConfig==false.

The Csc task doesn't see the current TargetFramework

It's one of the existing arguments. See here

And the SDK passes NoConfig=true when TF is specified, so it sounds like (3) supersedes (2)?

Where is that done? If that is already the case then I'd feel better.

It's one of the existing arguments. See here

Ah interesting, thanks!

Where is that done? If that is already the case then I'd feel better.

Well, the SDK passes NoConfig=true always here.

jjonescz · 2025-11-12T15:49:42Z

@dotnet/roslyn-compiler for a second review, thanks

jaredpar · 2025-11-12T22:11:07Z

@RikkiGibson, @JoeRobich PTAL

Include default RSP files for core compilers

cb1c532

github-actions bot added the Area-Compilers label Nov 3, 2025

jjonescz added 4 commits November 3, 2025 14:06

Include rsp files in toolset package

92d3e66

Avoid a breaking change

a3c20e6

Update unit tests

9db4d4f

Only include RSP if NoConfig=false

35e173a

jjonescz marked this pull request as ready for review November 5, 2025 18:04

jjonescz requested review from a team as code owners November 5, 2025 18:04

dotnet deleted a comment from azure-pipelines bot Nov 6, 2025

AlekseyTs reviewed Nov 6, 2025

View reviewed changes

Improve tests

4ae086d

jjonescz requested review from a team, AlekseyTs and jaredpar November 7, 2025 09:11

Use CombinatorialData instead of PairwiseData

e31f948

AlekseyTs reviewed Nov 7, 2025

View reviewed changes

Extend unit tests

a3348be

jjonescz requested a review from AlekseyTs November 10, 2025 14:40

jaredpar reviewed Nov 11, 2025

View reviewed changes

jaredpar approved these changes Nov 11, 2025

View reviewed changes

JoeRobich approved these changes Nov 14, 2025

View reviewed changes

jjonescz merged commit 86ebcfe into dotnet:main Nov 14, 2025
29 checks passed

jjonescz deleted the default-rsp branch November 14, 2025 08:42

dotnet-policy-service bot added this to the Next milestone Nov 14, 2025

dotnet-bot mentioned this pull request Nov 15, 2025

[Automated] PRs inserted in VS build main-11215.03 #81262

Closed

dotnet-maestro bot mentioned this pull request Dec 18, 2025

[main] Source code updates from dotnet/roslyn dotnet/dotnet#3957

Open

	if (!SuppressDefaultResponseFile(args) && File.Exists(responseFile))
	{
	allArgs = new[] { "@" + responseFile }.Concat(allArgs);
	}

Include default RSP files for core compilers #80993

Include default RSP files for core compilers #80993

Uh oh!

Conversation

jjonescz commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjonescz Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlekseyTs commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlekseyTs Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlekseyTs commented Nov 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlekseyTs commented Nov 7, 2025

Uh oh!

jaredpar Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjonescz commented Nov 12, 2025

Uh oh!

jaredpar commented Nov 12, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jjonescz commented Nov 3, 2025 •

edited

Loading

jjonescz Nov 6, 2025 •

edited

Loading

AlekseyTs commented Nov 6, 2025 •

edited

Loading

AlekseyTs Nov 6, 2025 •

edited

Loading

jaredpar Nov 11, 2025 •

edited

Loading