[Perf] Changes at 5/30/2022 12:57:05 PM #5802
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.15 |
| Baseline | b8acc97aba7662204c2a6de8f6090fc115a41c9f |
| Compare | c0227b4b3e17b55681f17c4f8de4123028a44819 |
| Diff | Diff |
Regressions in System.Collections.IterateForEach<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Dictionary - Duration of single invocation | 1.05 μs | 1.28 μs | 1.22 | 0.04 | False | |||||
| HashSet - Duration of single invocation | 1.04 μs | 1.16 μs | 1.12 | 0.03 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach<Int32>*'Payloads
Histogram
System.Collections.IterateForEach<Int32>.Dictionary(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.284542926953524 > 1.1017687944946062.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 5/30/2022 10:25:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -35.36155138350178 (T) = (0 -1254.769704001009) / Math.Sqrt((406.9267741673331 / (20)) + (491.9059763474576 / (51))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (51) - 2, .025) and -0.1825025097111002 = (1061.1137766697548 - 1254.769704001009) / 1061.1137766697548 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.IterateForEach<Int32>.HashSet(Size: 512)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1633441713291062 > 1.0883283849336312.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -39.0770331296631 (T) = (0 -1166.4684261517625) / Math.Sqrt((58.887003643881826 / (21)) + (397.8402354671762 / (50))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (50) - 2, .025) and -0.12346166012489358 = (1038.2805818420968 - 1166.4684261517625) / 1038.2805818420968 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository