-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Collections.IndexerSet<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Dictionary - Duration of single invocation | 3.39 μs | 2.33 μs | 0.69 | 0.14 | False | |||||
List - Duration of single invocation | 543.55 ns | 470.69 ns | 0.87 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet<Int32>*'
Payloads
Histogram
System.Collections.IndexerSet<Int32>.Dictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.3337001238924766 < 3.1987836757644192.
IsChangePoint: Marked as a change because one of 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/23/2022 3:43:39 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 36.29445825671002 (T) = (0 -2330.1385645978503) / Math.Sqrt((226575.8518280943 / (299)) + (1774.924178826274 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.30492973724878436 = (3352.3784421084774 - 2330.1385645978503) / 3352.3784421084774 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.IndexerSet<Int32>.List(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 470.69285464238385 < 516.6320622218282.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 13.89346016604831 (T) = (0 -454.7490074337159) / Math.Sqrt((1887.78994570729 / (299)) + (48.13539398257358 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.07941921801149754 = (493.9805569821199 - 454.7490074337159) / 493.9805569821199 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Globalization.Tests.StringSearch
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IsPrefix_FirstHalf - Duration of single invocation | 181.21 ns | 157.42 ns | 0.87 | 0.01 | False | |||||
IsPrefix_DifferentFirstChar - Duration of single invocation | 9.28 ns | 8.16 ns | 0.88 | 0.05 | False | |||||
IsPrefix_FirstHalf - Duration of single invocation | 181.12 ns | 157.33 ns | 0.87 | 0.01 | False | |||||
LastIndexOf_Word_NotFound - Duration of single invocation | 635.75 ns | 590.40 ns | 0.93 | 0.03 | False | |||||
IsPrefix_FirstHalf - Duration of single invocation | 181.11 ns | 157.42 ns | 0.87 | 0.01 | False | |||||
IsPrefix_DifferentFirstChar - Duration of single invocation | 9.32 ns | 7.98 ns | 0.86 | 0.07 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'
Payloads
Histogram
System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, False))
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 157.41527605179968 < 172.3763318579394.
IsChangePoint: Marked as a change because one of 4/1/2022 9:47:46 AM, 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 53.83427635971666 (T) = (0 -157.24722818086354) / Math.Sqrt((84.0275234936444 / (299)) + (0.010786200603913073 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.15366051811975148 = (185.79687176063115 - 157.24722818086354) / 185.79687176063115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreNonSpace, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.156966404347946 < 8.822731526311772.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 9.233940009745627 (T) = (0 -8.334504190700594) / Math.Sqrt((0.557306354386664 / (299)) + (0.021371158188245504 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.05036262207103851 = (8.776512366096087 - 8.334504190700594) / 8.776512366096087 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreNonSpace, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 157.3265622251475 < 172.45851775767747.
IsChangePoint: Marked as a change because one of 4/1/2022 9:47:46 AM, 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 52.42164800512294 (T) = (0 -157.3277461855133) / Math.Sqrt((86.25958882485833 / (299)) + (0.48664317788564987 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.15394843463485722 = (185.95526871652683 - 157.3277461855133) / 185.95526871652683 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 590.3994919315126 < 604.0730628559566.
IsChangePoint: Marked as a change because one of 4/14/2022 8:50:18 PM, 4/25/2022 11:05:49 AM, 5/31/2022 6:06:22 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 37.80278816146691 (T) = (0 -543.4168696874438) / Math.Sqrt((477.1941423943729 / (299)) + (112.91932689544414 / (42))) is greater than 1.9669864608404333 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (42) - 2, .975) and 0.12586891567476144 = (621.6651935068977 - 543.4168696874438) / 621.6651935068977 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, None, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 157.42180658450872 < 172.20408679002293.
IsChangePoint: Marked as a change because one of 4/1/2022 9:47:46 AM, 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 53.78845391348985 (T) = (0 -157.23063628684014) / Math.Sqrt((84.15974315062292 / (299)) + (0.016817547789128418 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.1536935007413146 = (185.78450765126453 - 157.23063628684014) / 185.78450765126453 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.980219828415134 < 8.825864556348746.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 9.545782438321645 (T) = (0 -8.328116068986606) / Math.Sqrt((0.5539923424669119 / (299)) + (0.017097123726230182 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.05096148630512013 = (8.77531938779055 - 8.328116068986606) / 8.77531938779055 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Text.Json.Serialization.Tests.WriteJson<Location>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SerializeToUtf8Bytes - Duration of single invocation | 618.74 ns | 566.78 ns | 0.92 | 0.02 | False | |||||
SerializeToString - Duration of single invocation | 681.07 ns | 622.85 ns | 0.91 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<Location>*'
Payloads
Histogram
System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToUtf8Bytes
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 566.7841670242648 < 593.2439524582076.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 29.297679280303083 (T) = (0 -579.2660957170524) / Math.Sqrt((261.21585618597385 / (299)) + (62.19391257713573 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.06855292884782367 = (621.8991005044602 - 579.2660957170524) / 621.8991005044602 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToString
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 622.8526268898306 < 642.2102782447714.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 23.64701697289941 (T) = (0 -632.1684036556783) / Math.Sqrt((250.25958716657684 / (299)) + (99.13693141596602 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.05910039859795396 = (671.8765771753717 - 632.1684036556783) / 671.8765771753717 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Tests.Perf_String
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Replace_Char - Duration of single invocation | 22.20 ns | 18.45 ns | 0.83 | 0.04 | False | |||||
Format_MultipleArgs - Duration of single invocation | 258.29 ns | 215.35 ns | 0.83 | 0.03 | False | |||||
Substring_IntInt - Duration of single invocation | 8.57 ns | 7.54 ns | 0.88 | 0.12 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'
Payloads
Histogram
System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'i', newChar: 'I')
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.448943575638854 < 20.883009425497217.
IsChangePoint: Marked as a change because one of 5/20/2022 9:09:19 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 19.680621393122625 (T) = (0 -19.952510112365477) / Math.Sqrt((0.5469089002260612 / (299)) + (0.27021798876403463 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.07739796036360358 = (21.626345114334338 - 19.952510112365477) / 21.626345114334338 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.Format_MultipleArgs
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 215.35059277494892 < 243.6079300544258.
IsChangePoint: Marked as a change because one of 5/18/2022 10:04:15 AM, 6/6/2022 10:51:35 AM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 16.29791470454287 (T) = (0 -215.8549553328591) / Math.Sqrt((226.38523781688366 / (246)) + (2.7174294074100978 / (4))) is greater than 1.969575653625878 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (246) + (4) - 2, .975) and 0.08717029947944786 = (236.4679361438012 - 215.8549553328591) / 236.4679361438012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.543217130745458 < 8.09337481035794.
IsChangePoint: Marked as a change because one of 4/13/2022 5:30:39 PM, 5/9/2022 5:23:46 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 17.077569484048606 (T) = (0 -7.5074631770501545) / Math.Sqrt((0.17726710357687356 / (299)) + (0.04804530970329039 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.08228739240842055 = (8.180625519303414 - 7.5074631770501545) / 8.180625519303414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Buffers.Text.Tests.Utf8FormatterTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
FormatterUInt32 - Duration of single invocation | 13.61 ns | 10.29 ns | 0.76 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
Payloads
Histogram
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 4294967295)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.286786269589676 < 12.922510797655786.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 36.50507485400911 (T) = (0 -10.068326042213572) / Math.Sqrt((1.9500517943073412 / (299)) + (0.012554825263181435 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.22981641782801984 = (13.07263135085284 - 10.068326042213572) / 13.07263135085284 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SerializeToStream - Duration of single invocation | 8.19 μs | 7.46 μs | 0.91 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>*'
Payloads
Histogram
System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToStream
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.463658986930889 < 7.63468948978694.
IsChangePoint: Marked as a change because one of 4/28/2022 7:34:39 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 28.980079726825196 (T) = (0 -7524.742346092752) / Math.Sqrt((29068.064643839592 / (299)) + (9463.966710077693 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.061198623624334696 = (8015.265566762117 - 7524.742346092752) / 8015.265566762117 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Collections.IndexerSet<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Dictionary - Duration of single invocation | 12.09 μs | 10.93 μs | 0.90 | 0.00 | True |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet<String>*'
Payloads
Histogram
System.Collections.IndexerSet<String>.Dictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.928078107865616 < 11.62737099987248.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 81.06227786295933 (T) = (0 -10928.358193940081) / Math.Sqrt((13584.320651492424 / (299)) + (12212.128439851002 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.11209493346156044 = (12308.025492573272 - 10928.358193940081) / 12308.025492573272 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Numerics.Tests.Perf_BigInteger
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToStringD - Duration of single invocation | 849.60 ns | 754.14 ns | 0.89 | 0.02 | False | |||||
Ctor_ByteArray - Duration of single invocation | 129.22 ns | 102.11 ns | 0.79 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'
Payloads
Histogram
System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 754.1384407925408 < 806.5535588699387.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 5/25/2022 1:42:32 AM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 28.16202531370103 (T) = (0 -758.1635171454686) / Math.Sqrt((2490.0728699932724 / (299)) + (5.843785912771317 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.09742714499898765 = (840.0025692603156 - 758.1635171454686) / 840.0025692603156 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 102.10737302212023 < 122.65133675480537.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 14.948591395198713 (T) = (0 -101.24350541297852) / Math.Sqrt((226.40046975619228 / (299)) + (1.2110460184754195 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.11658871054984837 = (114.60517498705953 - 101.24350541297852) / 114.60517498705953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Collections.ContainsTrue<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
HashSet - Duration of single invocation | 2.93 μs | 2.13 μs | 0.72 | 0.07 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue<Int32>*'
Payloads
Histogram
System.Collections.ContainsTrue<Int32>.HashSet(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1256164173057615 < 2.782408549781251.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/21/2022 3:06:05 PM, 4/25/2022 11:05:49 AM, 5/23/2022 3:43:39 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 20.719787786977307 (T) = (0 -2171.9293148654224) / Math.Sqrt((77128.65034308269 / (299)) + (5931.031482000382 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.15620741003385083 = (2574.0085190278273 - 2171.9293148654224) / 2574.0085190278273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 9380113a892ec86d147e6e7506a10eda407a82be |
Compare | 842b6a87aaa6b6f25555541f541d5ee202d3972a |
Diff | Diff |
Improvements in System.Collections.AddGivenSize<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
HashSet - Duration of single invocation | 3.80 μs | 3.16 μs | 0.83 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize<Int32>*'
Payloads
Histogram
System.Collections.AddGivenSize<Int32>.HashSet(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.157572658666128 < 3.6181229693556887.
IsChangePoint: Marked as a change because one of 4/21/2022 3:06:05 PM, 4/22/2022 6:17:37 AM, 5/11/2022 9:09:01 PM, 5/15/2022 3:27:46 PM, 5/30/2022 12:57:05 PM, 6/7/2022 5:42:36 AM falls between 5/29/2022 5:15:01 PM and 6/7/2022 5:42:36 AM.
IsImprovementStdDev: Marked as improvement because 21.609535212149446 (T) = (0 -3264.598624444853) / Math.Sqrt((138139.82417630538 / (299)) + (3522.4300328890017 / (50))) is greater than 1.9668240032621265 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (50) - 2, .975) and 0.13250279782202481 = (3763.2382170785263 - 3264.598624444853) / 3763.2382170785263 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository