-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | f55549222e4dc1469190caa56e03361ee46208ca |
Compare | f21a2666c577306e437f80fe934d76cdb15072a5 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
1.56 ms | 1.47 ms | 0.94 | 0.00 | False | ||||
320.86 ΞΌs | 303.46 ΞΌs | 0.95 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
Payloads
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson", Options: Compiled)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/27/2023 6:19:41 AM, 10/6/2023 9:14:38 AM, 10/11/2023 7:15:38 PM falls between 10/2/2023 7:07:56 PM and 10/11/2023 7:15:38 PM.
IsImprovementStdDev: Marked as improvement because 19.09759874396002 (T) = (0 -1461489.06503219) / Math.Sqrt((479213264.23872846 / (299)) + (314736819.999678 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.08765482028752052 = (1601903.6407829437 - 1461489.06503219) / 1601903.6407829437 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/15/2023 5:11:34 PM, 10/6/2023 9:14:38 AM, 10/11/2023 7:15:38 PM falls between 10/2/2023 7:07:56 PM and 10/11/2023 7:15:38 PM.
IsImprovementStdDev: Marked as improvement because 34.34964125501859 (T) = (0 -302916.3169496102) / Math.Sqrt((18585399.119986266 / (299)) + (1849959.3200064418 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06456518894950103 = (323824.0798516289 - 302916.3169496102) / 323824.0798516289 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | f55549222e4dc1469190caa56e03361ee46208ca |
Compare | f21a2666c577306e437f80fe934d76cdb15072a5 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in ByteMark
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
834.98 ms | 765.76 ms | 0.92 | 0.00 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'
Payloads
ByteMark.BenchNeural
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/28/2023 11:34:37 PM, 10/6/2023 9:14:38 AM, 10/11/2023 7:15:38 PM falls between 10/2/2023 7:07:56 PM and 10/11/2023 7:15:38 PM.
IsImprovementStdDev: Marked as improvement because 196.09108890703018 (T) = (0 -766955150.3601953) / Math.Sqrt((40055164876125.29 / (299)) + (690788109546.3832 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11316527525904561 = (864823093.8230615 - 766955150.3601953) / 864823093.8230615 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | f55549222e4dc1469190caa56e03361ee46208ca |
Compare | f21a2666c577306e437f80fe934d76cdb15072a5 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
2.68 ms | 2.53 ms | 0.94 | 0.03 | False | ||||
42.89 ms | 39.55 ms | 0.92 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
Payloads
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: Compiled)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/9/2023 7:27:03 AM, 5/9/2023 11:02:55 PM, 10/6/2023 9:14:38 AM, 10/11/2023 7:15:38 PM falls between 10/2/2023 7:07:56 PM and 10/11/2023 7:15:38 PM.
IsImprovementStdDev: Marked as improvement because 29.52349312640571 (T) = (0 -2537453.543792517) / Math.Sqrt((8078663426.1844225 / (299)) + (10915076.490233384 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.058807354506667145 = (2695998.0572972842 - 2537453.543792517) / 2695998.0572972842 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Tom|Sawyer|Huckleberry|Finn", Options: Compiled)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/19/2023 8:08:41 AM, 10/6/2023 9:14:38 AM, 10/11/2023 7:15:38 PM falls between 10/2/2023 7:07:56 PM and 10/11/2023 7:15:38 PM.
IsImprovementStdDev: Marked as improvement because 90.86402005454875 (T) = (0 -39326837.91819292) / Math.Sqrt((244180679550.03146 / (299)) + (13051347210.77212 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11219941669014354 = (44296927.32525186 - 39326837.91819292) / 44296927.32525186 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | f55549222e4dc1469190caa56e03361ee46208ca |
Compare | f21a2666c577306e437f80fe934d76cdb15072a5 |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in Benchstone.MDBenchF.MDLLoops
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
961.26 ms | 909.32 ms | 0.95 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchF.MDLLoops*'
Payloads
Benchstone.MDBenchF.MDLLoops.Test
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 9/28/2023 11:34:37 PM, 10/6/2023 9:14:38 AM, 10/11/2023 7:15:38 PM falls between 10/2/2023 7:07:56 PM and 10/11/2023 7:15:38 PM.
IsImprovementStdDev: Marked as improvement because 102.02913900055998 (T) = (0 -909138560.8058606) / Math.Sqrt((206502588186699.5 / (299)) + (60789564582.01882 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.0858790665109412 = (994549547.548177 - 909138560.8058606) / 994549547.548177 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository