Tagging subscribers to this area: @dotnet/area-system-globalization
See info in area-owners.md if you want to be subscribed.

@xtqqczze Thanks for your contribution. At the moment, we are only accepting critical changes for the .NET 9.0 release. Nice-to-have fixes will need to wait until we fork the main branch to accept changes for future releases. Since this PR is already a draft, we can keep it open for a while until the fork. Alternatively, you can close it and reopen it later after the fork. I wanted to set the expectation in case you notice no traction on the PR.

CC @ericstj @stephentoub

@tarekgh This fixes a probable bug.

@xtqqczze I'll take a look. Thanks!

@xtqqczze could you please explain more why you think there is a bug?

InvariantCreateSortKeyOrdinalIgnoreCase is called guaranteeing the condition Debug.Assert(sortKey.Length >= source.Length * sizeof(char));.

@xtqqczze could you please explain more why you think there is a bug?

@tarekgh When writing to the span, we overwrite the high surrogate with the low surrogate:

This is unexpected, at least.

Can you share a test that fails because of the issue?

When writing to the span, we overwrite the high surrogate with the low surrogate:

I see. I was confused with the line Span<byte> tmp = sortKey.Slice(0, 2 * sizeof(ushort)); // help with bounds check elimination which I don't think we need it.

Can you share a test that fails because of the issue?

This is a good idea to add a test for this case.

Here is the code can be used in the test case:

            CultureInfo culture = CultureInfo.InvariantCulture;
            var sortKey = culture.CompareInfo.GetSortKey("\uD800\uDC00", CompareOptions.IgnoreCase);

            foreach (var b in sortKey.KeyData)
            {
                Console.Write($"{b:X2} "); // should produce something like D8-00-DC-00
            }

We need to add the test to https://github.com/dotnet/runtime/tree/main/src/libraries/System.Runtime/tests/System.Globalization.Tests/Invariant

This is technically a breaking change to GetSortKey(ReadOnlySpan, Span, CompareOptions) in the Bucket 3: Unlikely Grey Area category.

@MihuBot

This is technically a breaking change to GetSortKey(ReadOnlySpan, Span, CompareOptions) in the Bucket 3: Unlikely Grey Area category.

I am not worried about that. The issue is specific for Invariant Globalization mode with Surrogate pairs. Sort keys usually interesting with the full globalization support.

/ba-g failures are known