[Perf] Windows/arm64: 2 Regressions on 10/11/2022 6:23:40 AM #8904
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 43f7699377fff108c74e19bea9f7ecf66286952f |
| Compare | 2b07c8c46523b816b1eac5c8af092e52b20a69d2 |
| Diff | Diff |
Regressions in System.Tests.Perf_UInt32
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| TryFormat - Duration of single invocation | 2.33 ns | 3.35 ns | 1.44 | 0.55 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Tests.Perf_UInt32.TryFormat(value: 0)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.3488838063961217 > 2.849379593600071.
IsChangePoint: Marked as a change because one of 10/3/2022 4:48:37 PM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsRegressionStdDev: Marked as regression because -11.815462584762773 (T) = (0 -3.3666043860348758) / Math.Sqrt((0.18011212934234638 / (35)) + (0.002188634659495856 / (5))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (5) - 2, .025) and -0.3554811521660737 = (2.4836969371761497 - 3.3666043860348758) / 2.4836969371761497 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.19041 |
| Baseline | 43f7699377fff108c74e19bea9f7ecf66286952f |
| Compare | 2b07c8c46523b816b1eac5c8af092e52b20a69d2 |
| Diff | Diff |
Regressions in System.Collections.IterateForEach<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Stack - Duration of single invocation | 3.51 μs | 3.77 μs | 1.07 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach<String>*'Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Collections.IterateForEach<String>.Stack(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.7684709147960422 > 3.6816225104103792.
IsChangePoint: Marked as a change because one of 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsRegressionStdDev: Marked as regression because -20.467046693893707 (T) = (0 -3768.3853375055633) / Math.Sqrt((5357.595247762171 / (35)) + (1.650033748434243 / (5))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (5) - 2, .025) and -0.07212108933360248 = (3514.8878004516037 - 3768.3853375055633) / 3514.8878004516037 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository