[Perf] Linux/arm64: 21 Regressions on 9/8/2023 12:12:11 AMΒ #21644
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 451f471161cbf6695b12562a366a0f2d106bdd20 |
| Compare | 9ae1a7cb4e024fce01bee35f533c268858e3320b |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_String
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 19.47 ns | 23.01 ns | 1.18 | 0.24 | False | ||||
| 4.03 ns | 5.66 ns | 1.40 | 0.41 | False | ||||
| 485.27 ns | 590.63 ns | 1.22 | 0.28 | False | ||||
| 6.65 ns | 8.32 ns | 1.25 | 0.31 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'Payloads
System.Tests.Perf_String.Trim_CharArr(s: " Te st ", c: [' ', 'β '])
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.013114125969327 > 20.408971896873528.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -22.014131159603 (T) = (0 -22.851806868747374) / Math.Sqrt((0.031149884604247967 / (12)) + (0.1983568132460126 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.1781384927042408 = (19.396537003297862 - 22.851806868747374) / 19.396537003297862 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_String.TrimEnd_CharArr(s: "Test", c: [' ', 'β '])
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.657229819314861 > 4.408611592607549.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -20.147943537954163 (T) = (0 -5.913516063415576) / Math.Sqrt((0.054252484923264725 / (12)) + (0.02139410504700083 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.3946560839631708 = (4.2401249536812236 - 5.913516063415576) / 4.2401249536812236 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_String.Split(s: "A B C D E F G H I J K L M N O P Q R S T U V W X Y Z", arr: [' '], options: RemoveEmptyEntries)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 590.6251645113299 > 515.5510726202084.
IsChangePoint: Marked as a change because one of 5/22/2023 2:06:57 AM, 6/8/2023 5:31:08 AM, 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -9.962282962961226 (T) = (0 -554.3290970970124) / Math.Sqrt((46.60577573341854 / (12)) + (288.5901234352892 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.12076824055351074 = (494.5974350801085 - 554.3290970970124) / 494.5974350801085 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_String.Trim_CharArr(s: "Test", c: [' ', 'β '])
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.320398723511316 > 6.808759345498687.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 4/25/2023 6:46:56 PM, 6/22/2023 12:06:43 AM, 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -20.222087611009375 (T) = (0 -8.29227308890645) / Math.Sqrt((0.03034738522024585 / (12)) + (0.049169925893840886 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.2787978886845569 = (6.4844281979823615 - 8.29227308890645) / 6.4844281979823615 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 451f471161cbf6695b12562a366a0f2d106bdd20 |
| Compare | 9ae1a7cb4e024fce01bee35f533c268858e3320b |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.IO.Tests.Perf_Path
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 33.47 ns | 38.34 ns | 1.15 | 0.16 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'Payloads
System.IO.Tests.Perf_Path.GetFileNameWithoutExtension
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.341065949281194 > 35.28722581182838.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -49.303588036630195 (T) = (0 -38.53693097141401) / Math.Sqrt((0.06980259650888111 / (12)) + (0.04039186671712157 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.14925900320943483 = (33.53198092318207 - 38.53693097141401) / 33.53198092318207 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 451f471161cbf6695b12562a366a0f2d106bdd20 |
| Compare | 9ae1a7cb4e024fce01bee35f533c268858e3320b |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Memory.ReadOnlySpan
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 21.87 ns | 23.57 ns | 1.08 | 0.02 | False | ||||
| 57.27 ns | 69.50 ns | 1.21 | 0.01 | False | ||||
| 57.28 ns | 69.48 ns | 1.21 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'Payloads
System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.570105321301707 > 22.96905355989376.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -44.3752667155622 (T) = (0 -23.673038142272222) / Math.Sqrt((1.6800665809655488E-05 / (12)) + (0.014775660269413208 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.08223277705689519 = (21.874257224633737 - 23.673038142272222) / 21.874257224633737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.4965150603305 > 60.15106037562524.
IsChangePoint: Marked as a change because one of 8/8/2023 8:10:39 AM, 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -233.46237568903837 (T) = (0 -69.50151943751797) / Math.Sqrt((0.00010301654909055471 / (12)) + (0.02455929239479569 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.2132217610227215 = (57.28673987757159 - 69.50151943751797) / 57.28673987757159 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.ReadOnlySpan.IndexOfString(input: "γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ xγ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ γ ", value: "x", comparisonType: Ordinal)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.48118664836963 > 60.159207279964136.
IsChangePoint: Marked as a change because one of 8/8/2023 8:10:39 AM, 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -1272.3060636506107 (T) = (0 -69.4603564527483) / Math.Sqrt((0.0001533790243428493 / (12)) + (0.0007086116683652054 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.21245635656240097 = (57.288954012072445 - 69.4603564527483) / 57.288954012072445 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 451f471161cbf6695b12562a366a0f2d106bdd20 |
| Compare | 9ae1a7cb4e024fce01bee35f533c268858e3320b |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Text.Json.Document.Tests.Perf_DocumentParse
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 6.07 ms | 7.94 ms | 1.31 | 0.17 | False | ||||
| 5.79 ms | 7.81 ms | 1.35 | 0.20 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'Payloads
System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400KB)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.9403799500000005 > 6.370259715914635.
IsChangePoint: Marked as a change because one of 5/28/2023 7:57:15 PM, 5/30/2023 4:48:39 PM, 7/11/2023 8:33:01 AM, 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -7.575371516539822 (T) = (0 -7904971.32777778) / Math.Sqrt((501630171834.97736 / (12)) + (43879494837.49861 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.2611018767116404 = (6268305.101876639 - 7904971.32777778) / 6268305.101876639 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: Json400KB)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.805967866666667 > 6.086010762708333.
IsChangePoint: Marked as a change because one of 7/11/2023 7:34:35 PM, 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -24.922716606324606 (T) = (0 -8082295.521076997) / Math.Sqrt((46452887.76657035 / (12)) + (75653105149.21043 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.39427747447409095 = (5796762.602168242 - 8082295.521076997) / 5796762.602168242 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 451f471161cbf6695b12562a366a0f2d106bdd20 |
| Compare | 9ae1a7cb4e024fce01bee35f533c268858e3320b |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Threading.Tasks.ValueTaskPerfTest
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 8.07 ns | 11.15 ns | 1.38 | 0.10 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tasks.ValueTaskPerfTest*'Payloads
System.Threading.Tasks.ValueTaskPerfTest.Copy_PassAsArgumentAndReturn_FromValueTaskSource
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.153718467094967 > 8.526569426874905.
IsChangePoint: Marked as a change because one of 5/22/2023 10:35:01 AM, 8/4/2023 10:42:53 AM, 8/27/2023 6:19:41 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -54.65413450013601 (T) = (0 -11.009311118234274) / Math.Sqrt((0.001029967153879777 / (12)) + (0.024553989523591457 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.357491843782155 = (8.110038501270735 - 11.009311118234274) / 8.110038501270735 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | ubuntu 20.04 |
| Queue | AmpereUbuntu |
| Baseline | 451f471161cbf6695b12562a366a0f2d106bdd20 |
| Compare | 9ae1a7cb4e024fce01bee35f533c268858e3320b |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.IO.Tests.BinaryReaderTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
|---|---|---|---|---|---|---|---|---|
| 84.87 ns | 101.04 ns | 1.19 | 0.02 | False | ||||
| 22.87 ns | 25.12 ns | 1.10 | 0.08 | False | ||||
| 51.07 ns | 55.81 ns | 1.09 | 0.05 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryReaderTests*'Payloads
System.IO.Tests.BinaryReaderTests.ReadNonAsciiChar
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 101.03932410091467 > 88.97320097708746.
IsChangePoint: Marked as a change because one of 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -84.13221421110111 (T) = (0 -100.90158631222977) / Math.Sqrt((0.2979793852939628 / (12)) + (0.12023103121690147 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.19466550308024166 = (84.46011544827586 - 100.90158631222977) / 84.46011544827586 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.IO.Tests.BinaryReaderTests.ReadAsciiChar
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.116522873001568 > 23.934180191819607.
IsChangePoint: Marked as a change because one of 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -11.295037505509478 (T) = (0 -24.928404049387726) / Math.Sqrt((0.3000740797346506 / (12)) + (0.0724954722852867 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.0897826832265189 = (22.874656051225017 - 24.928404049387726) / 22.874656051225017 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.IO.Tests.BinaryReaderTests.ReadSmallString
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.81142877436299 > 52.99450092259202.
IsChangePoint: Marked as a change because one of 8/31/2023 3:02:32 AM, 9/7/2023 2:36:34 PM, 9/13/2023 3:32:54 PM falls between 9/4/2023 9:28:57 AM and 9/13/2023 3:32:54 PM.
IsRegressionStdDev: Marked as regression because -24.716475724828825 (T) = (0 -56.07006577703155) / Math.Sqrt((0.4103455793607828 / (12)) + (0.12011903313288817 / (9))) is less than -2.093024054393918 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (9) - 2, .025) and -0.10633628151315641 = (50.68085238996546 - 56.07006577703155) / 50.68085238996546 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository