Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Regressions from "Avoid xchg for resolution"  #81768

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9e05d3347be231ae1bfcc7037ddecfb5c4ca51a1
Compare 120b7215c9d029fccee94f46864494d79074a0eb
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 280.39 ms 303.10 ms 1.08 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: Compiled)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 303.0988273333333 > 291.22724740673084.
IsChangePoint: Marked as a change because one of 1/11/2023 2:37:52 AM, 1/27/2023 7:12:54 PM, 2/1/2023 8:39:12 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsRegressionStdDev: Marked as regression because -6.8991552954396305 (T) = (0 -303268185.5826008) / Math.Sqrt((192513142277717.06 / (27)) + (51382853519.585594 / (32))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (32) - 2, .025) and -0.06468243531225498 = (284843795.2239316 - 303268185.5826008) / 284843795.2239316 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 9e05d3347be231ae1bfcc7037ddecfb5c4ca51a1
Compare 120b7215c9d029fccee94f46864494d79074a0eb
Diff Diff

Regressions in System.Net.Http.Tests.SocketsHttpHandlerPerfTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Get_EnumerateHeaders_Unvalidated - Duration of single invocation 8.95 ms 17.44 ms 1.95 0.90 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Http.Tests.SocketsHttpHandlerPerfTest*'

Payloads

Baseline
Compare

Histogram

System.Net.Http.Tests.SocketsHttpHandlerPerfTest.Get_EnumerateHeaders_Unvalidated(ssl: True, chunkedResponse: True, responseLength: 100000)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.440992809375 > 11.242045705142887.
IsChangePoint: Marked as a change because one of 2/1/2023 8:39:12 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsRegressionStdDev: Marked as regression because -3.548097165978861 (T) = (0 -15794333.253045475) / Math.Sqrt((17374797894054.414 / (27)) + (37834194257680.664 / (32))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (32) - 2, .025) and -0.4358447430604137 = (11000028.609904464 - 15794333.253045475) / 11000028.609904464 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9e05d3347be231ae1bfcc7037ddecfb5c4ca51a1
Compare 120b7215c9d029fccee94f46864494d79074a0eb
Diff Diff

Regressions in SciMark2.kernel

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
benchSparseMult - Duration of single invocation 609.18 ms 696.21 ms 1.14 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SciMark2.kernel*'

Payloads

Baseline
Compare

Histogram

SciMark2.kernel.benchSparseMult


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 696.2089355333333 > 639.6030126750002.
IsChangePoint: Marked as a change because one of 2/1/2023 8:39:12 AM, 2/7/2023 8:22:47 AM falls between 1/29/2023 5:12:48 PM and 2/7/2023 8:22:47 AM.
IsRegressionStdDev: Marked as regression because -657.0518642017826 (T) = (0 -696370455.7566222) / Math.Sqrt((209470639230.72266 / (26)) + (303351012897.6243 / (32))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (32) - 2, .025) and -0.14278876574405194 = (609360606.8162792 - 696370455.7566222) / 609360606.8162792 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Labels

area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMItenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions