-
Notifications
You must be signed in to change notification settings - Fork 4.2k
Remove some code now that we've updated to a newer System.IO.Hashing package version #79667
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Remove some code now that we've updated to a newer System.IO.Hashing package version #79667
Conversation
…package version This version exposes the XxHash128 class, which our Checksum class uses extensively. This and four other files that were copied into roslyn for it's support are no longer needed.
| length -= 4; | ||
| hash1 = (RuntimeBitOperations.RotateLeft(hash1, 5) + hash1) ^ (p0 | NormalizeToLowercase); | ||
| hash2 = (RuntimeBitOperations.RotateLeft(hash2, 5) + hash2) ^ (p1 | NormalizeToLowercase); | ||
| hash1 = (RotateLeft(hash1, 5) + hash1) ^ (p0 | NormalizeToLowercase); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wouldn't it be better to use System.Numerics.BitOperations.RotateLeft? It's a JIT intrinsic on dotnet.
We can define static extension polyfill for net472
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have problems with the netstandard build when I tried this:
Go to definition on it in that context yields the following from the System.Collections.Immutable package:
namespace System.Numerics
{
#if !NET
internal static class BitOperations
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static uint RotateLeft(uint value, int offset) => (value << offset) | (value >> (32 - offset));
}
#endif
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Dur, just read your comment again and went ahead and did that in commit 2
|
|
||
| NotAscii: | ||
| return s_comparer.GetHashCode(obj); | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd add BitOperations to MS.CA.Contracts project. Similar to https://github.com/dotnet/roslyn/blob/main/src/Dependencies/Contracts/ValueTaskExtensions.cs.
That way we can call the dotnet version directly and net472 polyfill will be applied as needed.
… if it would alleviate the insertion issue
| --> | ||
| <PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Condition="'$(DotNetBuildSourceOnly)' != 'true'" PrivateAssets="ContentFiles" /> | ||
| <PackageReference Include="System.Collections.Immutable" /> | ||
| <PackageReference Include="System.IO.Hashing" /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, adding a new reference to compiler packages might cause problems. cc @jaredpar
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this is an issue at all, I'm definitely fine closing out this PR as this was just an attempt to remove some hashing code we had copied from the runtime, as I'm attempting to copy some of this code over to razor.
There are VS hurdles to clear if this change were to go forward too, as the test insertion is complaining about optimization data not found for System.IO.Hashing.dll
@tmat indicated he has some context with a similar earlier attempt, so it might be nice to get his input on the difficulties he encountered and whether he thinks this is worth pursuing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At the very least we should make sure the new DLL is part of the toolset compiler package but I guess it is if it's being inserted into VS. There might be other infra considerations I'm missing though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding a new DLL to the compiler dependency is not cheap. Want to avoid this unless there are significant gains.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The gains were already attained. Switching to System.IO.Hashing pays off debt as discussed here: #70808
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you forward me the approval?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really hope we take this. The original copying of code was such an unpleasant hack to work around issues. It was really only meant to be temporary (as can even be seen in the conversation on my PR where we introduced this :)).
I definitely want us to be in the place where if the runtime creates and owns high perf algorithms like this, we can just naturally use them instead of effectively rolling our own :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we have the approval I'm okay with taking it for dev18. Just want to make sure we have it before we merge. Haven't seen it yet.
Let me know if was sent and I just missed it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tmat -- Can you forward any conversations you've previously had about this approval to this group?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ToddGrun forwarded the exception request to this group. Think we can close on that being an issue.
|
I think features/extensions adds some usages of hashing, please make sure this change is not going to break that branch |
|
@ToddGrun Looking at the second test insertion and I see "No optimization data found" for System.IO.Hashing. I believe a |
This version exposes the XxHash128 class, which our Checksum class uses extensively. This and four other files that were copied into roslyn for it's support are no longer needed.
Test insertion pipeline run 1: https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_build/results?buildId=12064988&view=results
-- failed miserably
Test inseriton pipeline run 2: https://devdiv.visualstudio.com/DefaultCollection/DevDiv/_build/results?buildId=12066383&view=results