Thanks to visit codestin.com
Credit goes to github.com

Skip to content

add TryGetValue_False_FrozenDictionary benchmark #3432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Nov 29, 2023
Merged

add TryGetValue_False_FrozenDictionary benchmark #3432

merged 1 commit into from
Nov 29, 2023

Conversation

adamsitnik
Copy link
Member

No description provided.

@adamsitnik adamsitnik mentioned this pull request Oct 20, 2023
Merged
andrewjsaid
andrewjsaid reviewed Oct 21, 2023
View reviewed changes
src/benchmarks/micro/libraries/System.Collections/Frozen/Perf_FrozenDictionary_String.cs
? ValuesGenerator.ArrayOfUniqueStrings(10, minLength: 25, maxLength: 25)
: ValuesGenerator.ArrayOfUniqueValues<string>(Count);

? ValuesGenerator.ArrayOfUniqueStrings(10 * 2, minLength: 25, maxLength: 25)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider adding a new case for 20 items designed to benchmark the length optimizations.

                // bimodal distribution of lengths with some search strings outside bounds of the keys
                : Count == 20 ? ValuesGenerator.ArrayOfUniqueStrings(10, minLength: 10, maxLength: 12)
                        .Concat(ValuesGenerator.ArrayOfUniqueStrings(10, minLength: 18, maxLength: 20))
                        .Concat(ValuesGenerator.ArrayOfUniqueStrings(10 * 2, minLength: 5, maxLength: 25)).ToArray()

I ran the benchmarks against 3 versions of .NET runtime.
main - main branch as of yesterday
nolength - I removed the existing if ((uint)(key.Length - _minimumLength) <= (uint)_maximumLengthDiff) check for comparison's sake
pr92546 - dotnet/runtime#92546 which adds a new length filter

Here are the results:

// * Summary *

BenchmarkDotNet v0.13.7-nightly.20230717.35, Windows 11 (10.0.22621.2428/22H2/2022Update/SunValley2)
AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores
.NET SDK 8.0.100-rc.1.23463.5
  [Host]     : .NET 8.0.0 (8.0.23.41904), X64 RyuJIT AVX2
  Job-DLBUBA : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
  Job-LYUJAP : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2
  Job-EJCRGQ : .NET 9.0.0 (42.42.42.42424), X64 RyuJIT AVX2

PowerPlanMode=00000000-0000-0000-0000-000000000000  Arguments=/p:EnableUnsafeBinaryFormatterSerialization=true  IterationTime=250.0000 ms  
MaxIterationCount=20  MinIterationCount=15  WarmupCount=1  

|                             Method |        Job |                         ...   Toolchain | Count |          Mean |        Error |       StdDev |        Median |           Min |           Max | Ratio | RatioSD | Allocated | Alloc Ratio |
|----------------------------------- |----------- |-------------------------...------------ |------ |--------------:|-------------:|-------------:|--------------:|--------------:|--------------:|------:|--------:|----------:|------------:|
| TryGetValue_False_FrozenDictionary | Job-DLBUBA |     \dotnet-runtime-main...\corerun.exe |    10 |      22.72 ns |     0.177 ns |     0.165 ns |      22.78 ns |      22.31 ns |      22.89 ns |  1.00 |    0.00 |         - |          NA |
| TryGetValue_False_FrozenDictionary | Job-LYUJAP | \dotnet-runtime-nolength...\corerun.exe |    10 |      22.04 ns |     0.746 ns |     0.829 ns |      21.67 ns |      21.03 ns |      23.72 ns |  0.97 |    0.04 |         - |          NA |
| TryGetValue_False_FrozenDictionary | Job-EJCRGQ |  \dotnet-runtime-pr92546...\corerun.exe |    10 |      24.14 ns |     0.305 ns |     0.285 ns |      24.19 ns |      23.61 ns |      24.53 ns |  1.06 |    0.02 |         - |          NA |
|                                    |            |                         ...             |       |               |              |              |               |               |               |       |         |           |             |
| TryGetValue_False_FrozenDictionary | Job-DLBUBA |     \dotnet-runtime-main...\corerun.exe |    20 |      37.76 ns |     0.761 ns |     0.747 ns |      37.59 ns |      36.78 ns |      39.03 ns |  1.00 |    0.00 |         - |          NA |
| TryGetValue_False_FrozenDictionary | Job-LYUJAP | \dotnet-runtime-nolength...\corerun.exe |    20 |      79.91 ns |     1.291 ns |     1.207 ns |      79.65 ns |      78.42 ns |      82.54 ns |  2.12 |    0.06 |         - |          NA |
| TryGetValue_False_FrozenDictionary | Job-EJCRGQ |  \dotnet-runtime-pr92546...\corerun.exe |    20 |      21.93 ns |     0.449 ns |     0.441 ns |      21.80 ns |      21.51 ns |      23.26 ns |  0.58 |    0.02 |         - |          NA |
|                                    |            |                         ...             |       |               |              |              |               |               |               |       |         |           |             |
| TryGetValue_False_FrozenDictionary | Job-DLBUBA |     \dotnet-runtime-main...\corerun.exe |   100 |     716.16 ns |     5.014 ns |     4.187 ns |     716.11 ns |     710.24 ns |     725.50 ns |  1.00 |    0.00 |         - |          NA |
| TryGetValue_False_FrozenDictionary | Job-LYUJAP | \dotnet-runtime-nolength...\corerun.exe |   100 |     686.85 ns |     4.755 ns |     4.448 ns |     686.33 ns |     678.03 ns |     694.17 ns |  0.96 |    0.01 |         - |          NA |
| TryGetValue_False_FrozenDictionary | Job-EJCRGQ |  \dotnet-runtime-pr92546...\corerun.exe |   100 |     561.36 ns |     3.986 ns |     3.728 ns |     562.77 ns |     555.22 ns |     567.82 ns |  0.78 |    0.01 |         - |          NA |
|                                    |            |                         ...             |       |               |              |              |               |               |               |       |         |           |             |
| TryGetValue_False_FrozenDictionary | Job-DLBUBA |     \dotnet-runtime-main...\corerun.exe |  1000 |   7,837.84 ns |    49.749 ns |    46.535 ns |   7,815.30 ns |   7,781.38 ns |   7,899.49 ns |  1.00 |    0.00 |         - |          NA |
| TryGetValue_False_FrozenDictionary | Job-LYUJAP | \dotnet-runtime-nolength...\corerun.exe |  1000 |   7,701.66 ns |     7.215 ns |     6.749 ns |   7,700.41 ns |   7,692.06 ns |   7,714.71 ns |  0.98 |    0.01 |         - |          NA |
| TryGetValue_False_FrozenDictionary | Job-EJCRGQ |  \dotnet-runtime-pr92546...\corerun.exe |  1000 |   7,936.08 ns |    49.304 ns |    46.119 ns |   7,957.89 ns |   7,838.63 ns |   7,987.06 ns |  1.01 |    0.01 |         - |          NA |
|                                    |            |                         ...             |       |               |              |              |               |               |               |       |         |           |             |
| TryGetValue_False_FrozenDictionary | Job-DLBUBA |     \dotnet-runtime-main...\corerun.exe | 10000 | 140,558.00 ns | 4,053.557 ns | 4,668.082 ns | 138,462.17 ns | 135,838.37 ns | 148,890.43 ns |  1.00 |    0.00 |         - |          NA |
| TryGetValue_False_FrozenDictionary | Job-LYUJAP | \dotnet-runtime-nolength...\corerun.exe | 10000 | 136,547.30 ns | 1,650.734 ns | 1,544.097 ns | 136,678.12 ns | 134,415.09 ns | 139,203.45 ns |  0.97 |    0.04 |         - |          NA |
| TryGetValue_False_FrozenDictionary | Job-EJCRGQ |  \dotnet-runtime-pr92546...\corerun.exe | 10000 | 139,220.39 ns |   850.721 ns |   795.765 ns | 139,317.75 ns | 137,574.00 ns | 140,382.59 ns |  0.99 |    0.04 |         - |          NA |

I believe the new Count = 20 scenario case showcases both the existing optimization (without PR92546), with the ratio going from 2.12 (optimization removed) -> 1 (main), as well as the proposed optimization in PR92546 with the ratio improving from 1 to 0.58.

Although the effects of PR92546 are also visible in the Count=100 scenario, I would argue that testing strings with lengths outside the bounds of the key strings is an important benchmark to keep.

@andrewjsaid andrewjsaid mentioned this pull request Oct 26, 2023
Closed
@cincuranet
Copy link
Contributor

Merging this as is. I see dotnet/runtime#92546 still open. @andrewjsaid if/when it's merged, feel free to update the benchmarks.

@cincuranet cincuranet merged commit 21bc5c2 into dotnet:main Nov 29, 2023
@cincuranet cincuranet self-assigned this Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stephentoub stephentoub Awaiting requested review from stephentoub

@cincuranet cincuranet Awaiting requested review from cincuranet

1 more reviewer

@andrewjsaid andrewjsaid andrewjsaid left review comments

Reviewers whose approvals may not affect merge requirements
Assignees

@cincuranet cincuranet

Labels
adds new benchmarks
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@adamsitnik @cincuranet @andrewjsaid