-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 486b4d1a36aef5bbe6a77bb2c3412772e712961e |
Compare | afacf6a99e1b98331af958caec638bd93dc46ffe |
Diff | Diff |
Improvements in System.IO.Tests.Perf_StreamWriter
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
WriteString - Duration of single invocation | 196.18 ms | 151.74 ms | 0.77 | 0.04 | False | |||||
WriteCharArray - Duration of single invocation | 199.46 ms | 150.98 ms | 0.76 | 0.06 | False | |||||
WritePartialCharArray - Duration of single invocation | 191.66 ms | 146.64 ms | 0.77 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_StreamWriter*'
Payloads
Histogram
System.IO.Tests.Perf_StreamWriter.WriteString(writeLength: 2)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 151.73525357142856 < 186.84593270833332.
IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 1/28/2022 4:34:00 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 172.16671191980632 (T) = (0 -151328244.02116403) / Math.Sqrt((12932873332958.814 / (299)) + (236960313406.99203 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.23083402938806896 = (196743290.5290528 - 151328244.02116403) / 196743290.5290528 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.Perf_StreamWriter.WriteCharArray(writeLength: 2)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 150.98134615384615 < 188.73668999999998.
IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 1/28/2022 4:34:00 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 148.79842210152302 (T) = (0 -151715793.88481888) / Math.Sqrt((13202171268583.852 / (299)) + (433309619115.89484 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.22956441018656046 = (196922099.51198745 - 151715793.88481888) / 196922099.51198745 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.IO.Tests.Perf_StreamWriter.WritePartialCharArray(writeLength: 2)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 146.64172666666667 < 186.76815021428573.
IsChangePoint: Marked as a change because one of 1/27/2022 1:01:32 PM, 1/28/2022 4:34:00 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 170.6564613489559 (T) = (0 -145754476.98412699) / Math.Sqrt((9011195415844.836 / (299)) + (507455811699.0318 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.25617391286580904 = (195952359.70506093 - 145754476.98412699) / 195952359.70506093 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 486b4d1a36aef5bbe6a77bb2c3412772e712961e |
Compare | afacf6a99e1b98331af958caec638bd93dc46ffe |
Diff | Diff |
Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Dictionary - Duration of single invocation | 5.73 μs | 4.40 μs | 0.77 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyTrue<Int32, Int32>*'
Payloads
Histogram
System.Collections.ContainsKeyTrue<Int32, Int32>.Dictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.401652330489036 < 5.557115129525874.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 39.938390651184704 (T) = (0 -4404.123418151574) / Math.Sqrt((192569.7652053424 / (299)) + (1614.6964416637309 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.2064918326679666 = (5550.192927388893 - 4404.123418151574) / 5550.192927388893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 486b4d1a36aef5bbe6a77bb2c3412772e712961e |
Compare | afacf6a99e1b98331af958caec638bd93dc46ffe |
Diff | Diff |
Improvements in System.Collections.AddGivenSize<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
HashSet - Duration of single invocation | 6.07 μs | 5.68 μs | 0.94 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize<Int32>*'
Payloads
Histogram
System.Collections.AddGivenSize<Int32>.HashSet(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.67574318620357 < 5.762637292526002.
IsChangePoint: Marked as a change because one of 3/18/2022 2:15:20 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 43.89298937532879 (T) = (0 -5638.487005796759) / Math.Sqrt((26480.04303474873 / (299)) + (515.0012235333845 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.08591602233246895 = (6168.456228917273 - 5638.487005796759) / 6168.456228917273 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 486b4d1a36aef5bbe6a77bb2c3412772e712961e |
Compare | afacf6a99e1b98331af958caec638bd93dc46ffe |
Diff | Diff |
Improvements in System.Collections.TryGetValueTrue<Int32, Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Dictionary - Duration of single invocation | 5.89 μs | 4.45 μs | 0.76 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue<Int32, Int32>*'
Payloads
Histogram
System.Collections.TryGetValueTrue<Int32, Int32>.Dictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.450786627685871 < 5.617562460877068.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/9/2022 6:32:22 AM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 83.93218111747682 (T) = (0 -4483.957037412014) / Math.Sqrt((179935.9849890743 / (299)) + (488.74967744960986 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.3240772395797023 = (6633.830520256235 - 4483.957037412014) / 6633.830520256235 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 486b4d1a36aef5bbe6a77bb2c3412772e712961e |
Compare | afacf6a99e1b98331af958caec638bd93dc46ffe |
Diff | Diff |
Improvements in System.Collections.IndexerSet<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Dictionary - Duration of single invocation | 6.07 μs | 4.40 μs | 0.72 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet<Int32>*'
Payloads
Histogram
System.Collections.IndexerSet<Int32>.Dictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.403316685293845 < 5.784804943605239.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 38.6320642201483 (T) = (0 -4413.7053870179625) / Math.Sqrt((333883.9203247102 / (299)) + (1026.0119683097798 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.2349209230532113 = (5768.953197141132 - 4413.7053870179625) / 5768.953197141132 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 486b4d1a36aef5bbe6a77bb2c3412772e712961e |
Compare | afacf6a99e1b98331af958caec638bd93dc46ffe |
Diff | Diff |
Improvements in System.Collections.ContainsTrue<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
HashSet - Duration of single invocation | 5.98 μs | 4.45 μs | 0.74 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue<Int32>*'
Payloads
Histogram
System.Collections.ContainsTrue<Int32>.HashSet(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.44929253720664 < 5.680304040594204.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/30/2022 4:33:23 PM, 6/2/2022 5:13:37 AM falls between 5/24/2022 3:42:52 PM and 6/2/2022 5:13:37 AM.
IsImprovementStdDev: Marked as improvement because 37.59044200901237 (T) = (0 -4492.347908268364) / Math.Sqrt((292638.7844724054 / (299)) + (874.0647997572504 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.2153537628513833 = (5725.316321650169 - 4492.347908268364) / 5725.316321650169 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository