Improve MSBuild argument parsing and tracking, and use that to set RestoreProperties that optimize Restore for large projects #49526

baronfel · 2025-06-22T22:41:52Z

The premise of this PR is that we would like to set some properties during Restore-time only that disable certain kinds of default globbing behaviors - those for Compile, EmbeddedResource, and None items. #49415 has the details of why we would want to do this.

MSBuild has a facility for passing properties that only apply during a Restore: --restoreproperty. The caveat to this is that as soon as a single --restoreproperty is passed, the engine no longer uses any of the --property values passed to that same invocation. This means that we need a way to:

track any user-provided --restoreproperty values
track any user-provided --property values
determine if an implicit restore is requested
when performing an implicit restore, mirror any --property values into --restoreproperty values as well, so that the Restore has a similar execution/evaluation environment as its associated build.

To do this, we

define a S.CL Option for the --restoreproperty structure.
Create a new structure to track all MSBuild-forwarded arguments
Force all MSBuild-forwarding pathways to work in terms of this new structure
At the last point we know that an implicit restore is requested or not, we mirror the existing properties (if any) into restoreproperties
The MSbuildForwardingAppWithoutLogging transforms the new MSbuild args structure into the string array of arguments to forward to MSBuild.

With this new structure in place, we can set the three restoreproperties that control the three item types in question, and we can do so in a way that respects any user-provided properties/restoreproperties that may override our defaults as well.

Every other change in this PR is a result of this approach. Over time, we can model more parts of the MSBuild CLI (targets, binlogs, logger arguments, etc) and expand the scope of the structured MSBuild arguments object, to provide more confidence/control.

I did model properties and targets in this new approach as well - so we have most of the common use cases accurately modeled and understood now.

Copilot

Pull Request Overview

This PR implements a new mechanism for handling restore properties by introducing a structured MSBuild arguments object (MSBuildArgs) and converting all MSBuild property–forwarding from the legacy dash-prefix (‑property:) style to the new double‐dash (“--property:”) convention. Key changes include refactoring command-line parsing and forwarding logic across multiple commands, updating tests to match the new syntax, and adding support for restore‐only properties that disable default item globbing.

Reviewed Changes

Copilot reviewed 46 out of 47 changed files in this pull request and generated 2 comments.

File	Description
test/dotnet.Tests/CommandTests/*	Updated tests to assert ApplicationArgs and revised MSBuild property syntax
src/Cli/dotnet/* and src/Cli/Microsoft.DotNet.Cli.Utils/*	Refactored MSBuild forwarding and option parsing to use the new MSBuildArgs structure and “--property:” style
src/Cli/Microsoft.DotNet.Cli.Utils/MSBuildArgs.cs	Introduced a structured model for accumulating MSBuild arguments with proper merging of global and restore properties

Comments suppressed due to low confidence (1)

src/Cli/Microsoft.DotNet.Cli.Utils/MSBuildArgs.cs:115

It would be beneficial to add a comment clarifying that duplicate keys from additional restore properties will overwrite existing ones, to aid future maintainability and user understanding.

    public void ApplyPropertiesToRestore()

src/Cli/Microsoft.DotNet.Cli.Utils/MSBuildForwardingAppWithoutLogging.cs

src/Cli/dotnet/Commands/Build/BuildCommand.cs

src/Cli/Microsoft.DotNet.Cli.Utils/MSBuildArgs.cs

src/Cli/Microsoft.DotNet.Cli.Utils/Microsoft.DotNet.Cli.Utils.csproj

src/Cli/dotnet/Commands/MSBuild/MSBuildForwardingApp.cs

src/Cli/Microsoft.DotNet.Cli.Utils/MSBuildArgs.cs

baronfel · 2025-06-28T16:30:51Z

There's only one interesting failing test - Microsoft.DotNet.Tests.PackagedCommandTests.CanInvokeToolWhosePackageNameIsDifferentFromDllName. This test is intended to show that we can invoke tools that are pulled in via the mostly-deprecated DotNetCliToolReference mechanism. The tool that's brought in is targeting netcoreapp2.2, which apparently we no longer provision on the build hosts?

I'm inclined to remove this test.

src/Cli/dotnet/Commands/Build/BuildCommand.cs

src/Cli/dotnet/Commands/Run/VirtualProjectBuildingCommand.cs

src/Cli/dotnet/Commands/CommandFactory.cs

MiYanni

Since this PR is so large, I've ignored most of my super minor nits I'd normally point out (extra/missing spaces, extra newlines, comments starting with lowercase letters, odd formatting, use of !, etc.). The core changes seem solid.

src/Cli/dotnet/CommandFactory/CommandResolution/ProjectToolsCommandResolver.cs

src/Cli/dotnet/Commands/NuGet/NuGetCommandParser.cs

MiYanni · 2025-07-01T17:26:58Z

src/Cli/dotnet/Commands/CommandFactory.cs

+        System.CommandLine.Command command,
+        Argument<string[]> catchAllUserInputArgument,
+        Func<MSBuildArgs, string, VirtualProjectBuildingCommand> configureVirtualCommand,
+        Func<MSBuildArgs, string?, CommandBase> createPhysicalCommand,
+        IEnumerable<Option> optionsToUseWhenParsingMSBuildFlags,
+        ParseResult parseResult,
+        string? msbuildPath = null)


This is a little funky. Maybe a class/record to hold this stuff just as a DTO to send to this command?

src/Cli/dotnet/Commands/Restore/RestoringCommand.cs

src/Cli/dotnet/Commands/Run/RunCommand.cs

src/Cli/dotnet/ReleasePropertyProjectLocator.cs

MiYanni · 2025-07-01T20:11:57Z

src/Cli/dotnet/dotnet.csproj

@@ -95,4 +95,15 @@
    <Using Include="System.Collections.Generic.Dictionary%3CMicrosoft.NET.Sdk.WorkloadManifestReader.WorkloadId, Microsoft.NET.Sdk.WorkloadManifestReader.WorkloadDefinition%3E" Alias="WorkloadCollection" />
  </ItemGroup>

+  <!-- NOTE: only works for fast-path code changes -->
+  <Target Name="CopyToRedistFastPath" AfterTargets="AfterBuild">


This is curious. Does this fix the issue where rebuilding the CLI I wasn't always getting new updates? I'd end up needing to do a full new build to get certain things to work that I've changed.

YES! But only for the subset of things that are actually contained in the dotnet.dll. This is totally a hack. If you change Targets, or something in the Cli.Utils.dll, etc - no benefit. Even so it's massively sped up my cycle times.

Nice, I've long wanted to add something like this. I've been running manually this command every time I built dotnet CLI :)

cp .\artifacts\bin\dotnet\Debug\net10.0\dotnet.dll .\artifacts\bin\redist\Debug\dotnet\sdk\10.0.100-dev\

test/dotnet-watch.Tests/CommandLine/CommandLineOptionsTests.cs

test/dotnet.Tests/CommandTests/MSBuild/GivenDotnetOsArchOptions.cs

MiYanni · 2025-07-01T20:33:14Z

test/dotnet.Tests/GivenParserDirectives.cs

@@ -9,32 +9,6 @@ public GivenParserDirectives(ITestOutputHelper log) : base(log)
        {
        }

-        [Fact]
-        public void ItCanInvokeDiagramDirective()


Are these tests just no longer valid or...?

The diagram directive is a S.CL builtin - and it's not user-facing. It's not a relevant thing to test IMO.

baronfel · 2025-07-02T00:34:32Z

@jjonescz do you have a sec to take a look at Microsoft.DotNet.Cli.Run.Tests.RunFileTests.NoRestore_01 and see if I messed something up on this branch? When I debug this test it seems to work, but when I run the test it fails. It feels like a timing issue?

The output logs for the test are:

Executing 'dotnet run --no-restore Program.cs':
》__________________________________________________
》Project "E:\Code\dotnet-sdk\artifacts\tmp\Debug\NoRestore_01\Program.csproj" (Build target(s)):
》
》E:\Code\dotnet-sdk\artifacts\bin\redist\Debug\dotnet\sdk\10.0.100-dev\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(266,5): error NETSDK1004: Assets file 'C:\Users\chusk\AppData\Local\Temp\dotnet\runfile\Program-44d8093e140b3ceefcb5c0b5d4e6b101fc1ef03cc861347d6b511999f0a8ff6c\obj\project.assets.json' not found. Run a NuGet package restore to generate this file.
》Done building project "Program.csproj" -- FAILED.
❌
❌The build failed. Fix the build errors and run again.
Command 'dotnet run --no-restore Program.cs' exited with exit code 1.
Executing 'dotnet restore Program.cs':
》__________________________________________________
》Project "E:\Code\dotnet-sdk\artifacts\tmp\Debug\NoRestore_01\Program.csproj" (Restore target(s)):
》
》Restored E:\Code\dotnet-sdk\artifacts\tmp\Debug\NoRestore_01\Program.csproj (in 349 ms).
Command 'dotnet restore Program.cs' exited with exit code 0.
Executing 'dotnet run --no-restore Program.cs /bl:{}':
》Warning: Binary log option was specified but build will be skipped because output is up to date, specify '--no-cache' to force build.
❌Unhandled exception: An error occurred trying to start process 'C:\Users\chusk\AppData\Local\Temp\dotnet\runfile\Program-44d8093e140b3ceefcb5c0b5d4e6b101fc1ef03cc861347d6b511999f0a8ff6c\bin\debug\Program.exe' with working directory 'E:\Code\dotnet-sdk\artifacts\tmp\Debug\NoRestore_01'. The system cannot find the file specified.
Command 'dotnet run --no-restore Program.cs /bl:{}' exited with exit code 1.

and what it seems like is that somehow a 'completed' build is tracked without the actual build occurring - which leads to the invocation of the supposedly-built binary failing.

… for .NET Framework the same way we do for other projects that consume it.

It's important that per-command default behaviors come first in the list of args to parse so that user-provided values can override them.

…arget arg parsing

…se commands

src/Cli/dotnet/Commands/Restore/RestoreCommand.cs

Co-authored-by: Jan Jones <[email protected]>

github-actions bot added the Area-CLI label Jun 22, 2025

build-analysis bot mentioned this pull request Jun 23, 2025

Containers integration tests are flaky when the network fails to load the registry #49502

Open

baronfel mentioned this pull request Jun 23, 2025

SDK should disable default item globbing when asking MSBuild to perform implicit Restores #49415

Closed

baronfel force-pushed the handle-restore-properties branch 3 times, most recently from a3c3554 to 1f8c0ac Compare June 27, 2025 12:38

baronfel marked this pull request as ready for review June 27, 2025 15:23

Copilot AI review requested due to automatic review settings June 27, 2025 15:23

Copilot AI reviewed Jun 27, 2025

View reviewed changes

src/Cli/Microsoft.DotNet.Cli.Utils/MSBuildForwardingAppWithoutLogging.cs Show resolved Hide resolved

src/Cli/dotnet/Commands/Build/BuildCommand.cs Outdated Show resolved Hide resolved

jjonescz reviewed Jun 27, 2025

View reviewed changes

This was referenced Jun 27, 2025

The active test run was aborted. Reason: Test host process crashed dotnet/dnceng#451

Open

dotnet-watch tests failing #40006

Open

baronfel requested review from a team and removed request for a team June 27, 2025 23:10

baronfel requested review from tmat, arunchndr and a team as code owners June 28, 2025 01:39

baronfel force-pushed the handle-restore-properties branch from 7372306 to f12508f Compare June 28, 2025 16:32

baronfel changed the title ~~Handle restore properties and set 3 that disable default items~~ Improve MSBuild argument parsing and tracking, and use that to set RestoreProperties that optimize Restore for large projects Jun 29, 2025

jjonescz reviewed Jun 30, 2025

View reviewed changes

src/Cli/dotnet/Commands/Build/BuildCommand.cs Outdated Show resolved Hide resolved

src/Cli/dotnet/Commands/Run/VirtualProjectBuildingCommand.cs Show resolved Hide resolved

baronfel force-pushed the handle-restore-properties branch from b03968d to 02712c5 Compare July 1, 2025 13:33

jjonescz reviewed Jul 1, 2025

View reviewed changes

src/Cli/dotnet/Commands/CommandFactory.cs Outdated Show resolved Hide resolved

MiYanni approved these changes Jul 1, 2025

View reviewed changes

baronfel added 5 commits July 1, 2025 20:43

checkpoint: separate restores are doing the right thing

36672e6

Update test expectations to expect restoreproperties

4672318

start plumbing through more strict msbuild property handling

08979af

Good checkpoint - only a smattering of tests left

4c7eb3d

add restoreproperty to make cli-schema test baseline correct

3e480d6

baronfel added 22 commits July 1, 2025 20:43

align visibility of new CLI argument structure

f9b2de0

Fix S.C.I unification error on the .NET Framework build by pinning it…

0c32a70

… for .NET Framework the same way we do for other projects that consume it.

Fix clean command argument forwarding

abd6f0f

It's important that per-command default behaviors come first in the list of args to parse so that user-provided values can override them.

Deduplicate nologo to make tests easier to write

5a77e92

Align run command application argument handling to previous logic

de93d1e

Update test baselines

8ebca56

Make proper command parsers for Properties and Targets

8f43c20

address Jan's PR feedback

81afddf

FrozenDictionary => ReadOnlyDictionary

ae9881b

Actually forward the 'target' option values

625035e

Adjust to verbosity and target changes for build command parser

e3b4394

Publish parser consistency

a72e424

Restore test consistency too

690fca5

Fix store command

48e1701

Make watch not send implicit arguments (they'll just be re-implied)

8bc43a6

Fix the last trailing failing tests

7c5ed61

Add test for file-based-app custom targets, and fix bug in repeated t…

c670351

…arget arg parsing

Unify the way we create virtual project commands vs physical commands

f8c7d84

Remember to apply the 'publish and pack as release' properties to tho…

5b69cd0

…se commands

Address code review comments

730037b

Add accidental missing build summary arg to build command parser

e35a4d9

Update test baseline

2a5ce81

baronfel force-pushed the handle-restore-properties branch from c257d1e to 2a5ce81 Compare July 2, 2025 01:43

jjonescz reviewed Jul 2, 2025

View reviewed changes

src/Cli/dotnet/Commands/Restore/RestoreCommand.cs Outdated Show resolved Hide resolved

Update src/Cli/dotnet/Commands/Restore/RestoreCommand.cs

5463e5a

Co-authored-by: Jan Jones <[email protected]>

baronfel merged commit 480b58f into dotnet:main Jul 2, 2025
26 checks passed

baronfel deleted the handle-restore-properties branch July 2, 2025 13:48

Improve MSBuild argument parsing and tracking, and use that to set RestoreProperties that optimize Restore for large projects #49526

Improve MSBuild argument parsing and tracking, and use that to set RestoreProperties that optimize Restore for large projects #49526

Uh oh!

Conversation

baronfel commented Jun 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

baronfel commented Jun 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MiYanni left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

MiYanni Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MiYanni Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

baronfel Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

jjonescz Jul 2, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

MiYanni Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

baronfel Jul 1, 2025

Choose a reason for hiding this comment

Uh oh!

baronfel commented Jul 2, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

baronfel commented Jun 22, 2025 •

edited

Loading