-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Count - Duration of single invocation | 41.93 μs | 23.27 μs | 0.56 | 0.04 | False | |||||
Count - Duration of single invocation | 56.12 μs | 46.10 μs | 0.82 | 0.05 | False | |||||
Count - Duration of single invocation | 41.98 μs | 31.62 μs | 0.75 | 0.06 | False | |||||
Count - Duration of single invocation | 47.06 μs | 29.19 μs | 0.62 | 0.22 | False | |||||
Count - Duration of single invocation | 46.34 μs | 41.88 μs | 0.90 | 0.01 | False | |||||
Count - Duration of single invocation | 46.88 μs | 36.40 μs | 0.78 | 0.07 | False | |||||
Count - Duration of single invocation | 54.24 μs | 36.95 μs | 0.68 | 0.34 | False | |||||
Count - Duration of single invocation | 53.31 μs | 43.07 μs | 0.81 | 0.50 | False | |||||
Count - Duration of single invocation | 41.94 μs | 23.26 μs | 0.55 | 0.04 | False | |||||
Count - Duration of single invocation | 74.40 μs | 56.87 μs | 0.76 | 0.13 | False | |||||
Count - Duration of single invocation | 57.88 μs | 40.05 μs | 0.69 | 0.11 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: Compiled)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.274324156746033 < 39.82167540773011.
IsChangePoint: Marked as a change because one of 4/20/2022 8:23:28 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 11.79783583314107 (T) = (0 -30150.901046571733) / Math.Sqrt((47040826.3401236 / (299)) + (10335220.20746258 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.1901317839289659 = (37229.39170627629 - 30150.901046571733) / 37229.39170627629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.101047197640106 < 53.46209060280475.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/16/2022 6:32:29 PM, 5/19/2022 10:26:24 PM, 5/23/2022 3:43:39 PM, 5/27/2022 11:47:05 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 9.520439099911544 (T) = (0 -44462.79271352502) / Math.Sqrt((21233891.275719523 / (299)) + (3379773.259869362 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.07350503674261799 = (47990.32318233248 - 44462.79271352502) / 47990.32318233248 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.620482323232324 < 39.88941730840375.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/11/2022 9:09:01 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 13.031007718484814 (T) = (0 -27714.15189955554) / Math.Sqrt((36289479.83406081 / (299)) + (17504894.107101444 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.24270654347565707 = (36596.31766363305 - 27714.15189955554) / 36596.31766363305 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 29.187718827736322 < 44.51849547807371.
IsChangePoint: Marked as a change because one of 4/17/2022 8:26:34 PM, 5/16/2022 6:32:29 PM, 5/30/2022 9:21:47 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 14.128741097452378 (T) = (0 -34744.804517154276) / Math.Sqrt((36371068.88826243 / (299)) + (10079383.921445403 / (53))) is greater than 1.9667650028636081 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (53) - 2, .975) and 0.18505244909876975 = (42634.405709583225 - 34744.804517154276) / 42634.405709583225 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.878473022273425 < 43.71402631948059.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 25.350395860259106 (T) = (0 -42246.03462613482) / Math.Sqrt((2218246.108822689 / (173)) + (44848.80871259648 / (30))) is greater than 1.971836506779775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (173) + (30) - 2, .975) and 0.06699058173141537 = (45279.32280097679 - 42246.03462613482) / 45279.32280097679 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.39936918604651 < 44.27499116886337.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/10/2022 1:42:57 AM, 5/16/2022 6:32:29 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 21.528573378320655 (T) = (0 -30285.777377861927) / Math.Sqrt((37369515.80968887 / (299)) + (8647314.459271194 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.2783878038635448 = (41969.602980677664 - 30285.777377861927) / 41969.602980677664 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.94614361702128 < 51.25579643370279.
IsChangePoint: Marked as a change because one of 4/20/2022 8:23:28 AM, 5/9/2022 10:19:44 PM, 5/16/2022 6:32:29 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 10.687636807488444 (T) = (0 -41941.55531591839) / Math.Sqrt((37751158.21279481 / (299)) + (14902147.561120145 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.14151299242188262 = (48855.20100559233 - 41941.55531591839) / 48855.20100559233 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 43.06536607332062 < 46.15841648951247.
IsChangePoint: Marked as a change because one of 4/18/2022 5:36:22 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 12.427212503310983 (T) = (0 -39281.82512594438) / Math.Sqrt((33227746.956785798 / (299)) + (13855469.620896176 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.16369512910911282 = (46970.699912459895 - 39281.82512594438) / 46970.699912459895 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aqj", Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.262415579212448 < 39.83531149463807.
IsChangePoint: Marked as a change because one of 4/17/2022 3:25:40 PM, 5/9/2022 11:56:12 AM, 5/25/2022 1:42:32 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 11.314731765718472 (T) = (0 -30121.97455276819) / Math.Sqrt((32642345.94679207 / (299)) + (10254043.146057954 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.1730207898057648 = (36424.10133344628 - 30121.97455276819) / 36424.10133344628 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 56.86950715421304 < 65.79574121721406.
IsChangePoint: Marked as a change because one of 4/19/2022 4:39:14 PM, 4/27/2022 1:24:17 PM, 5/9/2022 11:56:12 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 15.621241143157395 (T) = (0 -60267.75833899022) / Math.Sqrt((31686714.39581323 / (299)) + (13475659.833050666 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.1362236498154809 = (69772.41079373831 - 60267.75833899022) / 69772.41079373831 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.04731517094017 < 54.98122433020333.
IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/11/2022 9:09:01 PM, 5/26/2022 4:44:18 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 9.198524649660534 (T) = (0 -46857.32407385459) / Math.Sqrt((26001179.721443553 / (299)) + (10483525.949047199 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.09598312074690796 = (51832.35529027797 - 46857.32407385459) / 51832.35529027797 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Count - Duration of single invocation | 13.51 ms | 10.13 ms | 0.75 | 0.00 | True | |||||
Count - Duration of single invocation | 95.65 ms | 84.29 ms | 0.88 | 0.03 | False | |||||
Count - Duration of single invocation | 31.29 ms | 26.83 ms | 0.86 | 0.05 | False | |||||
Count - Duration of single invocation | 1.82 ms | 1.61 ms | 0.88 | 0.15 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: Compiled)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.129781666666666 < 12.833392546296295.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 696.0241789354882 (T) = (0 -10064162.3186903) / Math.Sqrt((404943209.82758385 / (299)) + (1160711273.1511881 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.25351198060769187 = (13482014.522996914 - 10064162.3186903) / 13482014.522996914 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "(?i)Tom|Sawyer|Huckleberry|Finn", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 84.28782166666667 < 88.4152946875.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 4/16/2022 5:36:25 PM, 5/9/2022 5:23:46 PM, 5/11/2022 9:09:01 PM, 5/23/2022 3:43:39 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 16.23730291469372 (T) = (0 -84814067.81843016) / Math.Sqrt((22078169919786.06 / (299)) + (285431508780.4188 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.05119568164084197 = (89390474.07067645 - 84814067.81843016) / 89390474.07067645 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.83114 < 29.714121849999998.
IsChangePoint: Marked as a change because one of 4/6/2022 12:51:22 AM, 4/16/2022 5:36:25 PM, 5/9/2022 5:23:46 PM, 5/31/2022 6:06:22 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 12.174840509930057 (T) = (0 -22803661.52187864) / Math.Sqrt((14405202875918.146 / (299)) + (1693891229253.6987 / (43))) is greater than 1.9669657340893643 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (43) - 2, .975) and 0.13643702911283956 = (26406483.70836449 - 22803661.52187864) / 26406483.70836449 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Twain", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.6051607083333332 < 1.7326859272613961.
IsChangePoint: Marked as a change because one of 4/20/2022 8:23:28 AM, 5/20/2022 9:09:19 PM, 5/25/2022 1:42:32 AM, 5/30/2022 10:25:35 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 20.634250572135343 (T) = (0 -1613570.0967430673) / Math.Sqrt((10619242221.691132 / (299)) + (62170405.82716843 / (52))) is greater than 1.9667845565748345 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (52) - 2, .975) and 0.07191036994772025 = (1738592.9596607755 - 1613570.0967430673) / 1738592.9596607755 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Collections.IterateFor<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ImmutableArray - Duration of single invocation | 291.38 ns | 151.87 ns | 0.52 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor<String>*'
Payloads
Histogram
System.Collections.IterateFor<String>.ImmutableArray(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 151.86861893965008 < 276.92223131956257.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/25/2022 1:42:32 AM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 8.44629070801227 (T) = (0 -152.2251129276328) / Math.Sqrt((3149.734785864599 / (299)) + (0.25187996030327386 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.1526347893384327 = (179.64522382124397 - 152.2251129276328) / 179.64522382124397 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Collections.IterateForEach<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Dictionary - Duration of single invocation | 3.02 μs | 2.59 μs | 0.86 | 0.06 | False | |||||
ConcurrentStack - Duration of single invocation | 4.92 μs | 3.73 μs | 0.76 | 0.47 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach<String>*'
Payloads
Histogram
System.Collections.IterateForEach<String>.Dictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5886734853416034 < 2.8673547387674856.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 51.305556521754085 (T) = (0 -2589.536613935208) / Math.Sqrt((18745.118805918188 / (299)) + (1.1840381007036103 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.13562329860504915 = (2995.84268034545 - 2589.536613935208) / 2995.84268034545 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.IterateForEach<String>.ConcurrentStack(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.7283095482986206 < 4.628318348123352.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 5/3/2022 7:02:53 AM, 6/2/2022 7:10:11 AM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 10.551949607464733 (T) = (0 -3961.359456650697) / Math.Sqrt((102467.62524395893 / (299)) + (46150.95969196729 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.10003257499035166 = (4401.6698233363595 - 3961.359456650697) / 4401.6698233363595 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IsMatch - Duration of single invocation | 185.42 ns | 171.43 ns | 0.92 | 0.04 | False | |||||
IsMatch - Duration of single invocation | 82.38 ns | 76.22 ns | 0.93 | 0.21 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'
Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: None)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 171.43007401899746 < 178.41746822188213.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/26/2022 5:29:00 PM, 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 18.387817243240526 (T) = (0 -170.3160044942821) / Math.Sqrt((153.80050748434493 / (299)) + (6.457453841021971 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.07956073252230547 = (185.03774286054073 - 170.3160044942821) / 185.03774286054073 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 76.22375647001155 < 77.68288808786357.
IsChangePoint: Marked as a change because one of 4/6/2022 12:51:22 AM, 5/21/2022 3:14:39 PM, 6/2/2022 5:58:32 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 12.855294950382216 (T) = (0 -72.94281016257138) / Math.Sqrt((19.72575620769316 / (299)) + (1.060143222312961 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.053115800532525265 = (77.0345626250752 - 72.94281016257138) / 77.0345626250752 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Collections.Tests.Add_Remove_SteadyState<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ConcurrentBag - Duration of single invocation | 42.32 ns | 38.59 ns | 0.91 | 0.07 | False | |||||
ImmutableStack - Duration of single invocation | 13.22 ns | 10.33 ns | 0.78 | 0.17 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState<String>*'
Payloads
Histogram
System.Collections.Tests.Add_Remove_SteadyState<String>.ConcurrentBag(Count: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 38.593297982401594 < 40.156769463135724.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 27.239343512990875 (T) = (0 -38.84721701182596) / Math.Sqrt((3.165723185474683 / (299)) + (0.41973094917741177 / (51))) is greater than 1.9668042230483112 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (51) - 2, .975) and 0.08774760218501629 = (42.58384752385673 - 38.84721701182596) / 42.58384752385673 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.Tests.Add_Remove_SteadyState<String>.ImmutableStack(Count: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.333372912107944 < 12.62078851407888.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 5/28/2022 5:31:50 AM, 5/31/2022 9:21:33 PM, 6/7/2022 9:00:42 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 9:00:42 AM.
IsImprovementStdDev: Marked as improvement because 7.725523924205293 (T) = (0 -10.342786726847805) / Math.Sqrt((0.8856854161103433 / (299)) + (0.3015016366438628 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.06995665064718326 = (11.120757687310999 - 10.342786726847805) / 11.120757687310999 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository