-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | 6123cb00bbec8ae3500c970890f5b0efcf7950e8 |
Compare | ffbcb8336ac67496b0565112f663f74fda6cac34 |
Diff | Diff |
Improvements in Benchstone.BenchF.NewtR
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 117.99 ms | 92.23 ms | 0.78 | 0.04 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.NewtR*'
Payloads
Histogram
Benchstone.BenchF.NewtR.Test
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 92.227065 < 112.36117281249999.
IsChangePoint: Marked as a change because one of 12/20/2022 1:34:12 PM, 1/3/2023 6:29:42 AM, 1/13/2023 11:01:49 AM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 29.870345705483576 (T) = (0 -92480237.5) / Math.Sqrt((125773842473797.31 / (299)) + (52104112350 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.17466914233560207 = (112052320.15886287 - 92480237.5) / 112052320.15886287 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | 6123cb00bbec8ae3500c970890f5b0efcf7950e8 |
Compare | ffbcb8336ac67496b0565112f663f74fda6cac34 |
Diff | Diff |
Improvements in System.Collections.ContainsTrue<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Stack - Duration of single invocation | 463.68 μs | 426.28 μs | 0.92 | 0.24 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue<String>*'
Payloads
Histogram
System.Collections.ContainsTrue<String>.Stack(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 426.27587837837837 < 438.95097652714924.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 37.75440822620825 (T) = (0 -427142.22798691544) / Math.Sqrt((214750575.32231653 / (299)) + (2532902.7230440797 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.0862478593186416 = (467459.61948544154 - 427142.22798691544) / 467459.61948544154 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | 6123cb00bbec8ae3500c970890f5b0efcf7950e8 |
Compare | ffbcb8336ac67496b0565112f663f74fda6cac34 |
Diff | Diff |
Improvements in System.Linq.Tests.Perf_Enumerable
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Select - Duration of single invocation | 1.24 μs | 1.12 μs | 0.90 | 0.03 | False | |||||
Select - Duration of single invocation | 1.25 μs | 1.12 μs | 0.90 | 0.02 | False | |||||
SelectToArray - Duration of single invocation | 532.60 ns | 471.63 ns | 0.89 | 0.07 | False | |||||
WhereSelect - Duration of single invocation | 1.01 μs | 914.03 ns | 0.90 | 0.02 | False | |||||
Repeat - Duration of single invocation | 496.28 ns | 438.06 ns | 0.88 | 0.03 | False | |||||
SelectToArray - Duration of single invocation | 306.15 ns | 276.54 ns | 0.90 | 0.03 | False | |||||
SequenceEqual - Duration of single invocation | 1.23 μs | 1.13 μs | 0.93 | 0.04 | False | |||||
Range - Duration of single invocation | 497.90 ns | 446.74 ns | 0.90 | 0.04 | False | |||||
SelectToList - Duration of single invocation | 1.02 μs | 939.37 ns | 0.92 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Payloads
Histogram
System.Linq.Tests.Perf_Enumerable.Select(input: IEnumerable)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1165913346430105 < 1.178844317863114.
IsChangePoint: Marked as a change because one of 1/18/2023 11:30:09 AM, 2/9/2023 12:40:40 PM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 28.51815574425668 (T) = (0 -1116.8047513498661) / Math.Sqrt((2523.6734698736263 / (299)) + (0.15619295191032043 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06916206446356905 = (1199.7843112252024 - 1116.8047513498661) / 1199.7843112252024 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.Select(input: IList)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1216159191881143 < 1.1856337233893206.
IsChangePoint: Marked as a change because one of 1/18/2023 3:20:52 PM, 2/9/2023 12:40:40 PM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 25.21735729076924 (T) = (0 -1123.7567274004205) / Math.Sqrt((2775.4192933171735 / (299)) + (8.45306188631201 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06835797775624025 = (1206.2108627238315 - 1123.7567274004205) / 1206.2108627238315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: IList)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 471.6278805954929 < 504.8984717100981.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/9/2023 12:40:40 PM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 23.020114940457532 (T) = (0 -471.9736341370936) / Math.Sqrt((829.2689833500283 / (299)) + (0.6541902238644065 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.0764756623145085 = (511.057061387185 - 471.9736341370936) / 511.057061387185 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 914.0322966723271 < 0.9583441760881528.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 16.835851293663755 (T) = (0 -913.2748662197699) / Math.Sqrt((829.941885630325 / (299)) + (41.934743170990096 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.05446813625813326 = (965.8848117561662 - 913.2748662197699) / 965.8848117561662 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.Repeat
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 438.0629942866792 < 472.48699924848705.
IsChangePoint: Marked as a change because one of 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM, 1/11/2023 2:37:52 AM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 28.88120907818418 (T) = (0 -436.52284022859084) / Math.Sqrt((459.260416116762 / (299)) + (6.270610346950133 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.09608139015857317 = (482.92272719683183 - 436.52284022859084) / 482.92272719683183 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Array)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 276.54011888846446 < 291.34213669960036.
IsChangePoint: Marked as a change because one of 12/21/2022 9:23:17 AM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/9/2023 12:40:40 PM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 18.77111551083683 (T) = (0 -277.45081588371687) / Math.Sqrt((219.92117278727576 / (299)) + (1.6587280156306252 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.06372266140357012 = (296.3340075064109 - 277.45081588371687) / 296.3340075064109 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: IEnumerable, input2: IEnumerable)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1339291233766233 < 1.164421735273495.
IsChangePoint: Marked as a change because one of 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/9/2023 12:40:40 PM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 9.320976455332252 (T) = (0 -1134.3120510693172) / Math.Sqrt((13629.48130415707 / (299)) + (0.35547787719769036 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.05259571105635953 = (1197.284057404975 - 1134.3120510693172) / 1197.284057404975 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.Range
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 446.7421435779155 < 472.8229320917001.
IsChangePoint: Marked as a change because one of 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM, 1/11/2023 2:37:52 AM, 1/13/2023 4:37:08 PM, 2/9/2023 12:40:40 PM, 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 17.231754339666768 (T) = (0 -442.34202297801517) / Math.Sqrt((752.775954266607 / (299)) + (18.61514272785443 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.08454411955032579 = (483.19316356429505 - 442.34202297801517) / 483.19316356429505 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Linq.Tests.Perf_Enumerable.SelectToList(input: IEnumerable)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 939.3710378721047 < 0.9606095401279322.
IsChangePoint: Marked as a change because one of 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/9/2023 12:40:40 PM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 23.187824602153743 (T) = (0 -940.8383899029624) / Math.Sqrt((448.01747655540265 / (299)) + (23.87224995837337 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.05453384056207288 = (995.1053038876443 - 940.8383899029624) / 995.1053038876443 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | 6123cb00bbec8ae3500c970890f5b0efcf7950e8 |
Compare | ffbcb8336ac67496b0565112f663f74fda6cac34 |
Diff | Diff |
Improvements in System.Buffers.Text.Tests.Utf8ParserTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TryParseUInt32Hex - Duration of single invocation | 7.67 ns | 6.21 ns | 0.81 | 0.05 | False | |||||
TryParseUInt64 - Duration of single invocation | 7.01 ns | 5.94 ns | 0.85 | 0.05 | False | |||||
TryParseUInt64Hex - Duration of single invocation | 15.28 ns | 13.56 ns | 0.89 | 0.09 | False | |||||
TryParseInt64 - Duration of single invocation | 7.39 ns | 6.11 ns | 0.83 | 0.06 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
Payloads
Histogram
System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: 3039)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.213626447230096 < 7.200839894532608.
IsChangePoint: Marked as a change because one of 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM, 1/13/2023 11:20:20 PM, 1/18/2023 3:20:52 PM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 38.641315017515666 (T) = (0 -6.160589626897849) / Math.Sqrt((0.156256876215024 / (299)) + (0.0007333462713241393 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1373924878215085 = (7.141822369874162 - 6.160589626897849) / 7.141822369874162 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 12345)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.943460361408201 < 6.526752838185618.
IsChangePoint: Marked as a change because one of 12/28/2022 5:13:50 PM, 1/18/2023 3:20:52 PM, 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 14.674484978117498 (T) = (0 -5.933155212970824) / Math.Sqrt((0.2635984301857318 / (299)) + (0.0002794024306909022 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.07007126053112188 = (6.380225668000649 - 5.933155212970824) / 6.380225668000649 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: FFFFFFFFFFFFFFFF)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.562577643204522 < 14.397801096016892.
IsChangePoint: Marked as a change because one of 1/18/2023 1:01:18 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 23.89203910521543 (T) = (0 -13.652342855258373) / Math.Sqrt((0.28959285987853367 / (299)) + (0.010379046804232416 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.08333160963193367 = (14.893436927367596 - 13.652342855258373) / 14.893436927367596 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 12345)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.108665435993148 < 7.014407140535605.
IsChangePoint: Marked as a change because one of 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 2/20/2023 10:31:48 AM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 37.16583667036925 (T) = (0 -6.108420849730861) / Math.Sqrt((0.18401322186727273 / (299)) + (2.484324654151857E-07 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.13114868894405945 = (7.030455927271512 - 6.108420849730861) / 7.030455927271512 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | 6123cb00bbec8ae3500c970890f5b0efcf7950e8 |
Compare | ffbcb8336ac67496b0565112f663f74fda6cac34 |
Diff | Diff |
Improvements in PerfLabTests.CastingPerf
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
FooObjIsFoo2 - Duration of single invocation | 438.34 μs | 379.80 μs | 0.87 | 0.04 | False | |||||
CheckArrayIsInterfaceYes - Duration of single invocation | 62.39 μs | 31.36 μs | 0.50 | 0.03 | False | |||||
CheckArrayIsInterfaceNo - Duration of single invocation | 62.41 μs | 31.22 μs | 0.50 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'
Payloads
Histogram
PerfLabTests.CastingPerf.FooObjIsFoo2
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 379.7994742063492 < 414.6857077991453.
IsChangePoint: Marked as a change because one of 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 37.11367807413152 (T) = (0 -379314.59864924964) / Math.Sqrt((451510688.8595705 / (299)) + (554352.8453540459 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.11020881150185384 = (426296.19572821824 - 379314.59864924964) / 426296.19572821824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.CastingPerf.CheckArrayIsInterfaceYes
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.360828323313292 < 59.27121518094289.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 16.49198575223303 (T) = (0 -31272.035051939816) / Math.Sqrt((243694375.21376815 / (299)) + (4479.1454969654615 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.32264206666677925 = (46167.66632975389 - 31272.035051939816) / 46167.66632975389 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### PerfLabTests.CastingPerf.CheckArrayIsInterfaceNo
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.220349301397203 < 59.27361530070195.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/18/2023 3:20:52 PM, 1/27/2023 7:12:54 PM, 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 16.55458578132166 (T) = (0 -31220.11798366027) / Math.Sqrt((243633048.0223547 / (299)) + (0.2058332450390743 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.3237065434197072 = (46163.566540368665 - 31220.11798366027) / 46163.566540368665 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | 6123cb00bbec8ae3500c970890f5b0efcf7950e8 |
Compare | ffbcb8336ac67496b0565112f663f74fda6cac34 |
Diff | Diff |
Improvements in System.MathBenchmarks.Double
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ScaleB - Duration of single invocation | 13.47 μs | 11.66 μs | 0.86 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Double*'
Payloads
Histogram
System.MathBenchmarks.Double.ScaleB
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.655171226983306 < 12.802394845545978.
IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 23.940436963855383 (T) = (0 -11679.610561953885) / Math.Sqrt((376217.81339245784 / (299)) + (798.9392271741665 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.07102865692740223 = (12572.62740024171 - 11679.610561953885) / 12572.62740024171 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.18362 |
Baseline | 6123cb00bbec8ae3500c970890f5b0efcf7950e8 |
Compare | ffbcb8336ac67496b0565112f663f74fda6cac34 |
Diff | Diff |
Improvements in System.Drawing.Tests.Perf_Color
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
GetBrightness - Duration of single invocation | 862.93 ns | 735.15 ns | 0.85 | 0.02 | False | |||||
GetHue - Duration of single invocation | 1.07 μs | 1.00 μs | 0.94 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Drawing.Tests.Perf_Color*'
Payloads
Histogram
System.Drawing.Tests.Perf_Color.GetBrightness
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 735.1481218911256 < 819.5774762462397.
IsChangePoint: Marked as a change because one of 1/18/2023 11:30:09 AM, 2/1/2023 8:39:12 AM, 2/9/2023 4:27:10 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 75.78635758922306 (T) = (0 -736.7511336499682) / Math.Sqrt((478.2267810197477 / (299)) + (4.722619996141215 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.1371207420762049 = (853.828767912085 - 736.7511336499682) / 853.828767912085 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Drawing.Tests.Perf_Color.GetHue
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0038216125050543 < 1.0177699758986702.
IsChangePoint: Marked as a change because one of 1/5/2023 11:35:29 PM, 1/11/2023 2:37:52 AM, 1/13/2023 4:37:08 PM, 1/18/2023 3:20:52 PM, 2/9/2023 4:27:10 AM, 3/6/2023 11:14:52 AM, 3/7/2023 4:31:45 PM falls between 2/26/2023 10:10:21 PM and 3/7/2023 4:31:45 PM.
IsImprovementStdDev: Marked as improvement because 27.253056263848904 (T) = (0 -995.0382043551161) / Math.Sqrt((1424.2759380116017 / (299)) + (58.97315243713895 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.094715837352527 = (1099.1446060926996 - 995.0382043551161) / 1099.1446060926996 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository