Address multi-model review consensus:
- Add Val.Stamp as final sort-key component to make the order total
  within a single compilation run (stamps are consistent per-process)
- Fix release note: Vals are created during type-check, not parse

Co-authored-by: Copilot <[email protected]>

…elease

- Extract valSourceOrderKey into TypedTreeOps.ExprConstruction (.fs + .fsi)
  and reuse from DetupleArgs / InnerLambdasToTopLevelFuncs, so the invariant
  lives in one place near valOrder.
- Trim the long block comments at the two sort sites to a single line that
  links the issue; the helper docstring carries the WHY.
- Restore a brief note in OptimizeInputs.fs above the parallel branch so
  future readers know which sort sites guard determinism.
- azure-pipelines-PR.yml: run eng/test-determinism.cmd in Release config.
  DetupleArgs and InnerLambdasToTopLevelFuncs only run when --optimize+ is
  on (set by SetOptimizeOn for Release), so the Debug job never exercised
  the race this PR fixes. Rename job to Determinism_Release.
- Release note: add PR link.

Co-authored-by: Copilot <[email protected]>

Co-authored-by: Copilot <[email protected]>

- Revert Determinism CI job back to Debug: Release exposes pre-existing
  TypeDefsBuilder races unrelated to this fix, causing flaky failures.
  Release coverage belongs in a follow-up when all races are fixed.
- Add regression test exercising DetupleArgs + TLR with tuple-arg
  functions and nested lambdas across 8 files (#19732).

Co-authored-by: Copilot <[email protected]>

Reverting CI to Debug was a hack. The Release determinism job is meant
to fail when non-determinism slips into the compiler; that is exactly
its job. Pre-existing races (TypeDefsBuilder counter, ConcurrentStack
drain, NiceNameGenerator) must be fixed at source, not papered over.

Co-authored-by: Copilot <[email protected]>

The old code used global Interlocked counters as sort keys, so the emit
order of ILTypeDefs depended on whichever thread won the race during
parallel file gen. Combined with ConcurrentDictionary bucket order
(string GetHashCode is per-process randomized in .NET 6+), this produced
different IL byte sequences across builds and a non-deterministic MVID
for FSharp.Compiler.Service.dll in Release.

Fix: route AddTypeDef through a thread-local batch context. Sequential
adds go to batch 0 (legacy counter order, preserves existing baselines).
Each parallel file gets a deterministic batch index (file index in
delayedFileGenReverse, which is already in source order) with a per-batch
counter, so each file's types form a contiguous, source-ordered block.

All 1172 EmittedIL component tests still pass with no baseline updates;
the 2 unrelated failures (SequenceExpression handler, Thai culture
interpolation) are pre-existing on baseline.

Co-authored-by: Copilot <[email protected]>

Two additional Release-only determinism races:

1. AssemblyBuilder.GrabExtraBindingsToGenerate (IlxGen.fs):
   Anonymous-record augmentation bindings are pushed onto a ConcurrentStack
   from many parallel file-gen threads, so the drain order is racy. Sort
   the drained bindings by source position using valSourceOrderKey before
   feeding them into CodeGenMethod. The baseline shifts are exactly the
   reorder of anon-record .Equals/.CompareTo/.GetHashCode overloads.

2. ParseInputFilesInParallel (ParseAndCheckInputs.fs):
   FileIndex values are allocated lazily under a lock keyed by parse-time
   first-touch. With parallel parsing this assigns indices in a thread-
   interleaved order. Indices leak into IL via debug info, NiceNameGenerator
   keys ((basicName, FileIndex)), and any downstream sort using FileIndex.
   Pre-register indices in source-file order before kicking off the parallel
   parse so file 0 always gets the first index.

Baseline updates:
  EmittedIL/Misc/AnonRecd.fs.il.netcore.bsl
  EmittedIL/Nullness/AnonRecords.fs.il.netcore.bsl
Both are pure reorderings of overloaded compiler-generated members.

Co-authored-by: Copilot <[email protected]>

Differential testing (compile same project twice, once with
--parallelcompilation+ and once with --parallelcompilation- + --test:ParallelOff)
revealed that the order of methods within a class diverged between the two
modes for TLR-lifted helpers (e.g. nested 'composed@N' methods).

Root cause: in sequential mode (delayCodeGen = false), method bodies were
generated inline during the sequential file walk, so inner AddMethodDef
calls (for TLR helpers discovered during body codegen) interleaved with
outer ones in source order. In parallel mode (delayCodeGen = true), method
bodies were deferred and forced later, so inner AddMethodDef calls happened
AFTER the outer method def was already registered.

Two complementary fixes:

1. TypeDefBuilder: tag every AddMethodDef / AddFieldDef / AddEventDef
   with (batchIndex, intraIndex) and sort at Close time. Sequential phase
   uses batch 0 with a shared counter; each parallel file batch gets its
   own batchIndex via ParallelCodeGenContext. Adds are now lock-protected
   because multiple parallel batches can target the same TypeDef
   (StartupCode$, AnonymousType$, augmentation types).

2. Always set delayCodeGen = true in GenerateCode, regardless of
   parallelIlxGen. Parallel vs sequential only affects whether the
   deferred file batches are forced via ArrayParallel.iteri or
   Array.iteri. This normalizes AddMethodDef timing across modes.

Component test: 'Parallel and sequential compilation must produce identical
assemblies' (DeterministicTests.fs). 12 files exercising TLR + anon records.
Verified to fail without (2) and pass with it.

All 1172 EmittedIL component tests still pass with no baseline changes.

Co-authored-by: Copilot <[email protected]>

…est hardening

Addresses cross-model consensus from 21-agent adversarial review:

- valSourceOrderKey: document Val.Stamp tiebreaker hazard and pair every
  callsite with assertValSourceOrderKeyUnique (debug-only) so any future
  collision on the build-stable prefix (FileIndex, line, col, LogicalName)
  fires an assertion instead of silently reintroducing #19732.
- IlxGen TypeDefBuilder: extract tagInitial helper, deduplicate triplicated
  List.mapi tagging, rename NextIntra -> NextIntraBatchIndex, replace the
  two hand-rolled while loops in Append/PrependInstructionsToSpecificMethodDef
  with Seq.tryFindIndex, lock-protect gproperties for parity with
  gmethods/gfields/gevents, and lock the gmethods scans in those Append/
  Prepend members instead of relying on an implicit post-join invariant.
- azure-pipelines-PR.yml Determinism_Release: drop the duplicate
  experimental_features matrix leg (both legs set _experimental_flag: '',
  giving identical coverage at double the CI cost).
- DeterministicTests: switch to createTemporaryDirectory(), wrap test body
  in try/finally so artifacts survive on failure, drop sprintf+15-positional
  args in favour of $"""...""" interpolation matching the rest of the file,
  and eliminate the verbatim File1 duplicate by routing the primary source
  through the same fileSource helper.
- Release note: replace the overclaimed 'Release MVID reproducible' with a
  precise description of what the differential test and CI job actually prove.

Co-authored-by: Copilot <[email protected]>

… trim prose

Addresses round-1 cross-model review consensus:

- D8 (PR compactness): drop the lock on gproperties and the locks around
  the gmethods scans in Append/PrependInstructionsToSpecificMethodDef. Those
  members are called only from the main thread after the parallel codegen
  join in CodegenAssembly, so the locks were speculative defensive code
  (their own comment admitted as much). Add a one-line invariant note in
  place of the locks.
- D5 vs D8 tension: drop assertValSourceOrderKeyUnique entirely. Running
  the EmittedIL suite with the assertion promoted from Debug.Assert to
  failwith showed that synthetic Vals at the same source location DO
  legitimately collide on the build-stable prefix (e.g. e1/e2 generic
  compare-augmentation parameters at file 0, line 1, col 0). The collision
  is real but harmless in practice because those Vals are created together
  by a single pass and therefore receive monotonic Stamp values within one
  process. Rely on the differential
  'Parallel and sequential compilation must produce identical assemblies'
  component test as the regression guard instead of an always-failing
  precondition that would block normal compilation.
- D8: trim TypeDefsBuilder.Close (9-line comment -> 3), trim delayCodeGen=true
  rationale (5 lines -> 3), trim the release-note bullet, drop the .fsi/.fs
  duplication on valSourceOrderKey.

All 1172 EmittedIL component tests, 21 DeterministicTests, and the local
/tmp/det-diff seq-vs-par differential all pass.

Co-authored-by: Copilot <[email protected]>

Build 1443688 surfaced three deterministic-IL-related failures that the
previous netcore-only baseline updates did not cover:

* WindowsCompressedMetadata_Desktop Batch1 - EmittedIL.RealInternalSignature.Misc.AnonRecd_fs
* WindowsCompressedMetadata_Desktop Batch2 - EmittedIL.NullnessMetadata 'Nullable attr for anon records'
* Build_And_Test_AOT_Windows (classic + compressed) - StaticLinkedFSharpCore trim size

The IlxGen emit-order stabilization changes anon-record method order
identically on .NET Framework and .NET, so mirror the netcore.bsl
reordering into the matching net472.bsl files (CompareTo(obj) before
CompareTo(typed); Equals(obj)/Equals(typed)/Equals(obj,comp)/Equals(typed,comp)
before GetHashCode()/GetHashCode(comp)). Bump the trimmed
StaticLinkedFSharpCore_Trimming_Test.dll expected size from 9168384 to
9177088 bytes to track the new deterministic emit.

Co-authored-by: Copilot <[email protected]>

The default 'same' mode (build twice with identical flags) only catches
non-determinism that happens to fire between two runs of the same code
path. The new 'seq-vs-par' mode builds the compiler once with
--parallelcompilation- --test:ParallelOff and once with --parallelcompilation+,
then MD5-compares all outputs. Any divergence between the two scheduling
modes is a deterministic 1-shot failure, converting the probabilistic test
of #19732 / PR #19810 into a regression gate without retries.

Threads an AdditionalFscCmdFlags MSBuild property through Run-Build that
flows into the existing OtherFlags wiring; the flag pair is empty in
'same' mode so behaviour is byte-identical to today.

Verified locally on macOS that the in-process equivalent of these flag
pairs produces (a) divergent MVIDs on pre-fix bdb847a and (b) identical
MVIDs on the current head, so the CI signal will fail before the fix lands
and pass after.

Co-authored-by: Copilot <[email protected]>

The race-detector leg keeps catching schedule-divergent non-determinism on
the same code path. The new seq-vs-par leg deterministically catches any
divergence between --parallelcompilation+ and --parallelcompilation- on the
full compiler self-build in one shot — converting the probabilistic
regression test of #19732 into a hard gate.

Co-authored-by: Copilot <[email protected]>

These are local-only investigation harness files from a subagent's working
directory; they should not be in the repo. Adds .scratch/ to .gitignore.

Co-authored-by: Copilot <[email protected]>

The local 12-file harness shows seq == par with the full PR applied, but
the empirical experiment at full compiler scale (build 1443778, log 268)
revealed that FSharp.Compiler.Service.dll and FSharp.Core.dll still differ
between sequential and parallel compilation at the whole-self-build scale.

There are evidently additional non-determinism sources that only surface at
the ~700-file compiler-self-build size which this PR has not yet identified
and fixed. Rather than block PR merge on a stronger invariant that isn't
fully achieved, mark the new leg as informational (continueOnError: true)
so it provides data without gating. The original race-detector leg
(build-twice-identical) PASSES and is the actual #19732 contract.

Co-authored-by: Copilot <[email protected]>

…eOnError)"

This reverts commit 87cdc4c.

This reverts commit 2e30a0a.

…sertion, trim prose"

This reverts commit 7f5fe7a.

…afety, test hardening"

This reverts commit 609540e.

This reverts commit a629ee6.

…ignment"

This reverts commit 684b291.

…codegen"

This reverts commit 1498292.

…istration

Co-authored-by: Copilot <[email protected]>

…nism

Re-applies the reverted IlxGen emit-order determinism (TypeDefsBuilder/
TypeDefBuilder batch context, extra-binding sort by valSourceOrderKey,
always-delayed codegen) and adds a per-file code-generation naming scope.

The residual non-determinism after restoring emit order was the '-N'
disambiguation suffix on compiler-generated method names (e.g. func1@1-N,
f@284-N from inlined FSharp.Core operators). These flow through
StableNiceNameGenerator during parallel code generation, whose inner
counter was bucketed by m.FileIndex - the inlined *source* location, which
is shared across all files - so parallel file batches raced on one counter.

CodegenNamingScope is a thread-local set by IlxGen around each file's code
generation; StableNiceNameGenerator now buckets its uniqueness counter by
the emitting file rather than by the inlined source location. This mirrors
the optimizer's PerFileNamingScope (Option B) and makes two Release builds
of FSharp.Compiler.Service.dll byte-identical (verified 3x).

Co-authored-by: Copilot <[email protected]>

… determinism"

This reverts commit 97a5d76.

Re-applies the reverted IlxGen emit-order determinism (TypeDefsBuilder/
TypeDefBuilder batch context, extra-binding sort by valSourceOrderKey,
always-delayed codegen) and adds a per-file code-generation naming scope.

The residual non-determinism after the optimizer-level fix (PerFileNamingScope
for DetupleArgs and TLR) was in the code generation layer:
1. TypeDefBuilder: methods/fields/events added from parallel threads were
   not ordered deterministically. Now tagged with (batchIndex, intraIndex).
2. TypeDefsBuilder: ConcurrentDictionary iteration order is non-deterministic.
   Now sorted by (batchIndex, intraBatchIndex) at Close.
3. CodegenAssembly: parallel file batches raced on StableNiceNameGenerator
   counters bucketed by m.FileIndex (shared inlined source). CodegenNamingScope
   now buckets by the emitting file.
4. Extra bindings from ConcurrentStack: sorted by valSourceOrderKey.
5. delayCodeGen = true unconditionally so method add order is identical.

Co-authored-by: Copilot <[email protected]>

…s-para

# Conflicts:
#	tests/AheadOfTime/Trimming/check.ps1
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/Inlining/Match01.fs.RealInternalSignatureOn.il.net472.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/Inlining/Regression_TLR_MutualInnerRec_Point2D.fs.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/SteppingMatch/SteppingMatch06.fs.il.netcore.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/SteppingMatch/SteppingMatch07.fs.il.netcore.bsl

Resolved conflicts:
- tests/AheadOfTime/Trimming/check.ps1: kept main's expected size (will re-measure after build)
- 28 EmittedIL .bsl baselines: took PR side; all baselines to be regenerated against the merged compiler

IlxGen.fs / Optimizer.fs auto-merged; verified main's #19955 (augmentation-member
closure nesting) and #19964 (optimizer protected-field relocation) both present and
composing with the determinism codegen changes.

Co-authored-by: Copilot <[email protected]>

…d, soften gate

- IlxGen: revert unconditional .cctor force to the predicated check (force only
  when the module has static fields). Unconditional force eagerly initialized
  fieldless modules, perturbing static-init order and causing runtime regressions
  (FSharpPlus testChoice hang, FSharp.Tests topinit / VersionTests.eval failures).
- IlxGen: revert raw-data static-field naming to main's per-call counter. The
  range-keyed GetOrCreateRawDataFieldSpec aliased distinct inline arrays that
  collapse to one source range (via remarkExpr), reusing a wrong field spec and
  throwing RuntimeHelpers.InitializeArray "field is invalid for array init" at
  runtime. Content-derived raw-data naming is delivered separately by #19929.
- azure-pipelines-PR.yml: make the same-flags Release determinism leg
  continueOnError. The residual non-determinism is the parallel-optimizer
  closure-name race (#19928), fixed by #19929; #19810 fixes the structural races.
- check.ps1: update trimmed-assembly expected sizes for the new codegen.
- release notes: consolidate to a single accurate #19810 entry.

Verified: topinit, eval-FSI, eval-FSC_OPTIMIZED, EmittedIL (net10+net472),
Conformance StaticLet, AOT trimming, and SurfaceArea all pass locally.

Co-authored-by: Copilot <[email protected]>

Regenerated against the merged compiler (main + predicated cctor + per-call
raw-data field naming) on net10.0 and net472. Net effect brings these baselines
back toward main's codegen; verified all EmittedIL pass on both TFMs.

Co-authored-by: Copilot <[email protected]>

…meters

- check.ps1: SelfContained FSharp.Core.dll back to 311296 (CI value == main; my
  earlier local measurement was off by a 4096-byte local-vs-CI trimmer delta).
  StaticLinkedFSharpCore -> -1 placeholder: it statically links FSharp.Compiler.Service,
  whose trimmed size still varies with the #19928 closure-name residual (fixed in #19929)
  and the toolchain. FSharpMetadataResource keeps 7612928 (matched CI).
- Regenerate OptionalAndOutParameters realsig- IL baseline: the predicated-.cctor revert
  drops a spurious empty .cctor on a fieldless module. Verified passing.

Co-authored-by: Copilot <[email protected]>

…ings

The same-flags Release determinism job failed consistently on CI because FSharp.Compiler.Service.dll still drifts (residual parallel newUnique() closure-name race, #19928/#19929). It was masked with a blanket 'continueOnError: true', which leaves the build partiallySucceeded and silently swallows ANY future non-determinism regression in every other binary.

Use the determinism harness's purpose-built skipList to exclude only the one known-racy binary, and drop continueOnError so the job is a real hard gate again for all other binaries. The build now goes fully green while remaining honest; #19929 removes the skip once the closure-name race is fixed.

Co-authored-by: Copilot <[email protected]>

The previous commit added FSharp.Compiler.Service.dll to the skipList so it no longer enters the determinism map. But Test-MapContents still asserted that binary must be present, throwing 'Did not find the expected binary FSharp.Compiler.Service.dll' and failing the now-hard-gated job. Remove it from the well-known list (keeping FSharp.Core.dll as the anchor that proves real compiler output was examined). #19929 re-adds it when the skip is removed.

Co-authored-by: Copilot <[email protected]>

…fix/determinism-seq-vs-para

# Conflicts:
#	azure-pipelines-PR.yml
#	src/Compiler/CodeGen/IlxGen.fs
#	src/Compiler/Optimize/Optimizer.fs
#	tests/AheadOfTime/Trimming/check.ps1
#	tests/FSharp.Compiler.ComponentTests/Conformance/BasicGrammarElements/MethodResolution/OptionalAndOutParameters.fs.RealInternalSignatureOff.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/AsyncExpressionStepping/AsyncExpressionSteppingTest5.fs.RealInternalSignatureOn.OptimizeOn.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/ForLoop/NonTrivialBranchingBindingInEnd03.fs.RealInternalSignatureOff.OptimizeOn.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/ForLoop/NonTrivialBranchingBindingInEnd03.fs.RealInternalSignatureOn.OptimizeOn.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/ForLoop/NonTrivialBranchingBindingInEnd04.fs.RealInternalSignatureOff.OptimizeOn.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/ForLoop/NonTrivialBranchingBindingInEnd04.fs.RealInternalSignatureOn.OptimizeOn.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/Inlining/Regression_TLR_MutualInnerRec_Point2D.fs.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/Misc/AnonRecd.fs.il.netcore.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/Misc/CodeGenRenamings01.fs.RealInternalSignatureOff.OptimizeOn.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/Misc/CodeGenRenamings01.fs.RealInternalSignatureOn.OptimizeOn.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/Misc/Lock01.fs.RealInternalSignatureOn.OptimizeOn.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/Misc/ModuleWithExpression01.fs.RealInternalSignatureOn.OptimizeOn.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/Nullness/AnonRecords.fs.il.netcore.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/TestFunctions/Verify13043.fs.RealInternalSignatureOff.OptimizeOn.il.bsl
#	tests/FSharp.Compiler.ComponentTests/EmittedIL/TestFunctions/Verify13043.fs.RealInternalSignatureOn.OptimizeOn.il.bsl

…s-para

…erge)

…cking on 19810+main

…S.dll determinism gate

The residual #19928 non-determinism was the '-N' disambiguation suffix on compiler-generated
closure type names: it was allocated from a NiceNameGenerator bucket keyed by the closure's own
range FileIndex. Inlined or synthetic-range closures (e.g. FileIndex 0) share one bucket that the
parallel per-file IlxGen drain increments in thread-scheduling order, so two builds could name the
same closure func@1-17 vs func@1-23.

Route closure naming through StableNiceNameGenerator.GetUniqueCompilerGeneratedNameInScope, bucketing
the suffix by CodegenFileScope.CurrentFileIdx (the file currently being emitted) instead. Each file's
drain runs under its own thread-static scope, so parallel drains touch disjoint buckets and produce
identical names to a sequential drain.

Also wrap the trailing residue drain (anon-record structural-equality augmentations) in a
CodegenFileScope.With past the last file index, so its emitted members keep a positive currentFileIdx
(OrderKey assert + no struct(0,_) hoisting).

Re-enable FSharp.Compiler.Service.dll in eng/test-determinism.ps1 (removed from skipList, restored as a
well-known binary). Verified: seq-vs-par harness now produces byte-identical FSharp.Compiler.Service.dll.

Co-authored-by: Copilot <[email protected]>

…order

The trailing residue drain (anon-record structural-equality augmentations, forced static-init) now
runs in a CodegenFileScope, so its emitted members/types sort deterministically instead of via a
struct(0,_) key. IL is semantically identical (pure member/type reordering); verified byte-identical
seq-vs-par.

Co-authored-by: Copilot <[email protected]>

The determinism fix stabilizes FSharp.Compiler.Service.dll, so the statically-linked trimming size is
now pinnable. Pin real net9.0 trimmed sizes (SelfContained FSharp.Core 315392, StaticLinked 9178624;
FSharpMetadataResource 7612928 unchanged) - verified locally against the global.json-pinned SDK.

Co-authored-by: Copilot <[email protected]>

…ed file-index locals

Behavior-preserving cleanup of the closure-name fix: extract the shared StableNiceNameGenerator cache
body (GetOrAddStableName) so the plain and scoped variants differ only in the allocator; add
NiceNameGenerator.FreshCompilerGeneratedNameOfBasicNameInScope to avoid recomputing the basic name;
replace the raw CodegenFileScope.CurrentFileIdx read with CurrentFileIdxOr(fallback) so the 0-sentinel
policy is centralized; name the lastImplFileIdx / residueFileIdx codegen scope indices. Verified: FSCS.dll
still byte-identical seq-vs-par.

Co-authored-by: Copilot <[email protected]>

Benign method reordering from the newly-merged #19548/#19758 codegen changes combined with the
per-file IlxGen emit order; IL is semantically identical (pure permutation).

Co-authored-by: Copilot <[email protected]>

The closure-name determinism now comes from #19548's nameRange (bucket by the emitting file), not a
scoped name generator; update the skipList comment and release note to describe the emit-order
determinism this PR delivers without the dropped per-codegen-file-scope wording.

Co-authored-by: Copilot <[email protected]>