Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Perf] Linux/x64: 158 Improvements on 2/15/2023 8:55:14 PM #13314

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString_NonFlags_Small - Duration of single invocation 953.09 ns 720.20 ns 0.76 0.11 False
ToString_Format_Flags_Large - Duration of single invocation 817.08 ns 582.00 ns 0.71 0.14 False
StringFormat - Duration of single invocation 8.22 μs 6.93 μs 0.84 0.11 False
InterpolateIntoStringBuilder_NonFlags - Duration of single invocation 13.15 μs 11.27 μs 0.86 0.07 False
InterpolateIntoString - Duration of single invocation 13.77 μs 12.01 μs 0.87 0.05 False
ToString_Flags - Duration of single invocation 1.09 μs 827.19 ns 0.76 0.17 False
InterpolateIntoStringBuilder_Flags - Duration of single invocation 12.67 μs 11.35 μs 0.90 0.06 False
ToString_NonFlags_Large - Duration of single invocation 859.71 ns 619.81 ns 0.72 0.16 False
ToString_Flags - Duration of single invocation 1.12 μs 847.41 ns 0.75 0.11 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.ToString_NonFlags_Small(value: -1)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 720.203303281232 < 936.6130330016003.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/15/2023 7:15:22 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 35.249899840322435 (T) = (0 -734.5593417431369) / Math.Sqrt((7771.825305781427 / (194)) + (473.89661609312645 / (32))) is greater than 1.9706109611022053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (194) + (32) - 2, .975) and 0.26224692196468125 = (995.670995638965 - 734.5593417431369) / 995.670995638965 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.ToString_Format_Flags_Large(value: All, format: "d")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 582.0009367748637 < 754.8838858255058.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 23.798933708350674 (T) = (0 -596.3424029089945) / Math.Sqrt((12773.723670705198 / (194)) + (252.0049371920753 / (32))) is greater than 1.9706109611022053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (194) + (32) - 2, .975) and 0.25520472290620283 = (800.6796246559617 - 596.3424029089945) / 800.6796246559617 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.StringFormat(value: 32)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.929294960346256 < 7.901238860994894.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 26.096534015279584 (T) = (0 -6948.659460474508) / Math.Sqrt((1311801.5030933253 / (194)) + (55391.23441282889 / (32))) is greater than 1.9706109611022053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (194) + (32) - 2, .975) and 0.2571160714029868 = (9353.627387790611 - 6948.659460474508) / 9353.627387790611 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.InterpolateIntoStringBuilder_NonFlags(value: 42)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.268638191893219 < 12.179635376252927.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 7.777918305092729 (T) = (0 -11470.650397209016) / Math.Sqrt((1898370.6616823785 / (193)) + (37997.92541861701 / (32))) is greater than 1.9706589608356722 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (193) + (32) - 2, .975) and 0.06646124532748722 = (12287.278208640564 - 11470.650397209016) / 12287.278208640564 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.InterpolateIntoString(value: 32)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 12.010418258212374 < 12.842441585561138.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/15/2023 10:25:39 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 8.831736644254791 (T) = (0 -12181.7619402098) / Math.Sqrt((1738702.7511535508 / (195)) + (46435.448521687074 / (32))) is greater than 1.9705633902925863 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (195) + (32) - 2, .975) and 0.06874517664835579 = (13081.018894878718 - 12181.7619402098) / 13081.018894878718 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.ToString_Flags(value: 32)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 827.1881842248863 < 1.0485795461674177.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 34.53372202962532 (T) = (0 -815.52673279313) / Math.Sqrt((7377.452199015507 / (195)) + (590.6052572897656 / (32))) is greater than 1.9705633902925863 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (195) + (32) - 2, .975) and 0.24110262676917632 = (1074.6205765889272 - 815.52673279313) / 1074.6205765889272 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: 32)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.345271415678186 < 11.822569956623054.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 8.342950605926932 (T) = (0 -11237.086838265774) / Math.Sqrt((1800350.502344818 / (192)) + (39521.06018303514 / (32))) is greater than 1.9707073953201992 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (192) + (32) - 2, .975) and 0.07104846821319175 = (12096.526518075276 - 11237.086838265774) / 12096.526518075276 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.ToString_NonFlags_Large(value: 42)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 619.8104680394317 < 822.6725513279544.
IsChangePoint: Marked as a change because one of 1/9/2023 3:58:12 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 23.839697624176495 (T) = (0 -641.3830462586478) / Math.Sqrt((20089.012642210015 / (194)) + (457.10674993000526 / (32))) is greater than 1.9706109611022053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (194) + (32) - 2, .975) and 0.2874856831883279 = (900.1686438087036 - 641.3830462586478) / 900.1686438087036 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Enum.ToString_Flags(value: 36)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 847.4135177508504 < 1.0946999415456142.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 31.071613780238597 (T) = (0 -842.616156693363) / Math.Sqrt((7068.7545966472435 / (194)) + (1170.6915718188015 / (32))) is greater than 1.9706109611022053 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (194) + (32) - 2, .975) and 0.2396055306900975 = (1108.1303069682779 - 842.616156693363) / 1108.1303069682779 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat4 - Duration of single invocation 1.88 μs 992.86 ns 0.53 0.11 False
ToString3 - Duration of single invocation 1.70 μs 1.05 μs 0.62 0.06 False
ToStringL - Duration of single invocation 2.45 μs 1.76 μs 0.72 0.09 False
ToString4 - Duration of single invocation 2.13 μs 1.28 μs 0.60 0.08 False
TryFormat2 - Duration of single invocation 917.72 ns 518.13 ns 0.56 0.09 False
TryFormat3 - Duration of single invocation 1.40 μs 729.79 ns 0.52 0.09 False
TryFormatL - Duration of single invocation 2.18 μs 1.49 μs 0.68 0.09 False
ToString2 - Duration of single invocation 1.24 μs 806.70 ns 0.65 0.09 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.TryFormat4


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 992.860077158324 < 1.741770321024235.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 33.914861440937564 (T) = (0 -991.9501992425254) / Math.Sqrt((121261.67898038065 / (299)) + (1160.6088195635218 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4181527934196376 = (1704.8293573022788 - 991.9501992425254) / 1704.8293573022788 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString3

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0522938944031506 < 1.6112435431371757.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.02507418705528 (T) = (0 -1038.9232369445745) / Math.Sqrt((51295.002386310916 / (299)) + (1185.5122453880688 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.36877899324358693 = (1645.894584977735 - 1038.9232369445745) / 1645.894584977735 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToStringL

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.759734170347981 < 2.3714869509610077.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 58.630484397544656 (T) = (0 -1752.3469947296305) / Math.Sqrt((87500.96496173713 / (299)) + (2645.621073079174 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.39327520460843274 = (2888.207319100422 - 1752.3469947296305) / 2888.207319100422 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString4

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.2773342048199834 < 2.0173746366166463.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 38.84384159377969 (T) = (0 -1304.311543290916) / Math.Sqrt((96260.83418067398 / (299)) + (1897.6849864039011 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3676853351333497 = (2062.757066635461 - 1304.311543290916) / 2062.757066635461 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormat2

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 518.1311255992244 < 880.1244498680181.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 34.2167451024951 (T) = (0 -525.0333459397746) / Math.Sqrt((30015.323269150696 / (299)) + (204.56373606478658 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4024254445978987 = (878.6072653084859 - 525.0333459397746) / 878.6072653084859 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormat3

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 729.7945119526396 < 1.3210621384923098.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 34.62216904906278 (T) = (0 -751.6824621483197) / Math.Sqrt((67212.88168789464 / (299)) + (610.0615703044 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.41835511773954637 = (1292.3391661713706 - 751.6824621483197) / 1292.3391661713706 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.TryFormatL

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.486235989131342 < 2.037452798548641.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 63.55858470161165 (T) = (0 -1410.362470835293) / Math.Sqrt((67980.23808052542 / (299)) + (1296.642703539459 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.42448888026706155 = (2450.625926202366 - 1410.362470835293) / 2450.625926202366 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Version.ToString2

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 806.6976407488386 < 1.183086656598896.
IsChangePoint: Marked as a change because one of 1/15/2023 4:08:29 PM, 2/1/2023 12:50:52 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 42.03770357063851 (T) = (0 -806.8239625525238) / Math.Sqrt((19876.602651253037 / (299)) + (1244.9980230023202 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.3484777079102295 = (1238.367393331424 - 806.8239625525238) / 1238.367393331424 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 0202b242169f8d5955e0411dc5e4f42a5d016c1b
Compare 24107379cd4fbfe236f0a1cf92dbd6c4ad944fec
Diff Diff

Improvements in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Round - Duration of single invocation 374.85 ns 168.29 ns 0.45 0.06 False
Floor - Duration of single invocation 312.27 ns 108.96 ns 0.35 0.05 False
Divide - Duration of single invocation 1.71 μs 831.15 ns 0.49 0.06 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.Round


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 168.28632164937076 < 355.2228549720679.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 25.268794834230206 (T) = (0 -169.24566418931315) / Math.Sqrt((9620.159988871917 / (299)) + (3.38828547156693 / (31))) is greater than 1.9672228266464409 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (31) - 2, .975) and 0.45896771157944555 = (312.81989598697544 - 169.24566418931315) / 312.81989598697544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Floor

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 108.95687509549275 < 298.1062596391903.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 28.76282852439328 (T) = (0 -110.12381189581542) / Math.Sqrt((8924.69071801176 / (299)) + (4.331499159417129 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.5885096095285645 = (267.62183138626636 - 110.12381189581542) / 267.62183138626636 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Divide

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 831.1528819482463 < 1.6344916751246217.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 2/15/2023 6:58:43 PM, 2/21/2023 8:58:51 AM falls between 2/12/2023 2:35:10 PM and 2/21/2023 8:58:51 AM.
IsImprovementStdDev: Marked as improvement because 31.59828232508761 (T) = (0 -846.4472227867492) / Math.Sqrt((143824.18006140908 / (299)) + (219.2482947279804 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.4519185862504692 = (1544.382278895466 - 846.4472227867492) / 1544.382278895466 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions