[Perf] Changes at 6/2/2022 10:27:31 PM #5826
Description
Run Information
| Architecture | x86 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | 1b4a9c47ba8e4ad53562e003240527b738b0e7d1 |
| Compare | 45589f2a1ea20b4bce66058994de52f317716558 |
| Diff | Diff |
Regressions in System.IO.Tests.Perf_Path
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| HasExtension - Duration of single invocation | 4.80 ns | 6.25 ns | 1.30 | 0.07 | False | |||||
| GetPathRoot - Duration of single invocation | 24.79 ns | 26.99 ns | 1.09 | 0.02 | False | |||||
| GetDirectoryName - Duration of single invocation | 53.84 ns | 68.96 ns | 1.28 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_Path*'Payloads
Histogram
System.IO.Tests.Perf_Path.HasExtension
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.253518619237168 > 5.042692086836503.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 5/30/2022 12:57:05 PM, 6/2/2022 5:58:32 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -17.043409622894682 (T) = (0 -6.551897351878714) / Math.Sqrt((0.22474777569114113 / (41)) + (0.029452144490879496 / (29))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (29) - 2, .025) and -0.2653079244893709 = (5.17810504863692 - 6.551897351878714) / 5.17810504863692 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.Perf_Path.GetPathRoot
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.989571609337574 > 26.077574433815595.
IsChangePoint: Marked as a change because one of 4/3/2022 1:52:36 PM, 4/11/2022 1:47:52 PM, 4/12/2022 5:02:33 PM, 5/23/2022 7:50:08 PM, 6/6/2022 7:24:37 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -22.01355095111655 (T) = (0 -26.89288468599952) / Math.Sqrt((0.1582992997372231 / (64)) + (0.015293683090122504 / (6))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (64) + (6) - 2, .025) and -0.06158305131524716 = (25.33281277680593 - 26.89288468599952) / 25.33281277680593 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.Perf_Path.GetDirectoryName
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 68.95790146611576 > 56.95861417191733.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/6/2022 7:24:37 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsRegressionStdDev: Marked as regression because -53.149700692717914 (T) = (0 -68.99018738421498) / Math.Sqrt((4.2684273825954575 / (65)) + (0.007215966910771626 / (6))) is less than -1.9949454151066512 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (65) + (6) - 2, .025) and -0.24878153595761907 = (55.2460021210278 - 68.99018738421498) / 55.2460021210278 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository