[Perf] Windows/x64: 8 Improvements on 7/26/2023 8:42:30 AM #20123
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 8f5364695b107f37af899aba3ad1b9f98d577c33 |
| Compare | bc23bf4ee16cbd1999105b1916e1f2f76baef6a7 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsFalse<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 1.30 ms | 847.22 μs | 0.65 | 0.07 | False | 14431490.384615384 | 4170515.046296296 | 0.28898713404834836) |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<String>*'Payloads
System.Collections.ContainsFalse<String>.LinkedList(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 14.102772859803416 (T) = (0 -909230.6425438597) / Math.Sqrt((5703088272.316532 / (299)) + (3056614491.7718887 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.33320424313185676 = (1363581.9262174116 - 909230.6425438597) / 1363581.9262174116 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 8f5364695b107f37af899aba3ad1b9f98d577c33 |
| Compare | bc23bf4ee16cbd1999105b1916e1f2f76baef6a7 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsTrue<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 646.86 μs | 341.85 μs | 0.53 | 0.06 | False | ||||
| 465.07 μs | 380.63 μs | 0.82 | 0.31 | False | 5064828.431372549 | 3989939.024390244 | 0.7877737772272349) | |
| 469.96 μs | 379.35 μs | 0.81 | 0.29 | False | 4930269.607843137 | 3985365.8536585364 | 0.8083464334929198) |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<String>*'Payloads
System.Collections.ContainsTrue<String>.LinkedList(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 6.682852975219135 (T) = (0 -430099.9712998102) / Math.Sqrt((1510628508.4353876 / (299)) + (5611587026.255397 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.3682973840922415 = (680858.3033675669 - 430099.9712998102) / 680858.3033675669 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsTrue<String>.ICollection(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/2/2023 3:18:41 PM, 5/8/2023 11:50:12 PM, 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 46.73002853406124 (T) = (0 -381656.41193621757) / Math.Sqrt((1483882675.4461195 / (299)) + (645110.2745741954 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.2170133551528324 = (487436.6816444407 - 381656.41193621757) / 487436.6816444407 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Collections.ContainsTrue<String>.List(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/20/2023 3:57:43 PM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 49.02263383896598 (T) = (0 -378862.0748299319) / Math.Sqrt((1393587890.2252169 / (299)) + (542302.597142735 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.22161703129505506 = (486729.6563030838 - 378862.0748299319) / 486729.6563030838 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 8f5364695b107f37af899aba3ad1b9f98d577c33 |
| Compare | bc23bf4ee16cbd1999105b1916e1f2f76baef6a7 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 27.53 μs | 17.07 μs | 0.62 | 0.10 | False | ||||
| 21.83 μs | 16.11 μs | 0.74 | 0.08 | False | 186179.92992992993 | 151509.0811965812 | 0.8137777323989898) | |
| 22.22 μs | 16.74 μs | 0.75 | 0.09 | False | 186002.21893491125 | 152183.1735889244 | 0.8181793446355534) |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>*'Payloads
System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: Reflection)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 61.6026785778903 (T) = (0 -17163.8784852896) / Math.Sqrt((2706088.9831080446 / (299)) + (7159.892059686274 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.27203194138639775 = (23577.790649191113 - 17163.8784852896) / 23577.790649191113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: Reflection)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 43.69757712420155 (T) = (0 -16334.760052486203) / Math.Sqrt((2768246.9959000223 / (299)) + (54818.39297125186 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.28844683942321453 = (22956.485836202646 - 16334.760052486203) / 22956.485836202646 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToStream(Mode: Reflection)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 64.1860646458521 (T) = (0 -16702.012425888588) / Math.Sqrt((2591709.309662356 / (299)) + (1250.2878402458773 / (3))) is greater than 1.9679030112610798 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (3) - 2, .975) and 0.2680918030972151 = (22819.81879225629 - 16702.012425888588) / 22819.81879225629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 8f5364695b107f37af899aba3ad1b9f98d577c33 |
| Compare | bc23bf4ee16cbd1999105b1916e1f2f76baef6a7 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.CreateAddAndRemove<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 45.40 μs | 43.06 μs | 0.95 | 0.02 | False | 486214.14441147377 | 468651.9607843137 | 0.963879735238024) |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove<String>*'Payloads
System.Collections.CreateAddAndRemove<String>.List(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 6/27/2023 3:20:01 PM, 6/28/2023 2:04:56 PM, 7/3/2023 3:46:39 PM, 7/5/2023 3:16:33 PM, 7/26/2023 12:02:55 AM, 7/27/2023 9:58:25 PM falls between 7/18/2023 9:38:16 PM and 7/27/2023 9:58:25 PM.
IsImprovementStdDev: Marked as improvement because 22.616290104077788 (T) = (0 -42714.386861755374) / Math.Sqrt((1680233.3537578683 / (299)) + (71451.11042241787 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.07504759396304289 = (46180.09162738336 - 42714.386861755374) / 46180.09162738336 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository