-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | 7d5efbb9e10b6d8beb91c90cbdefd7360869cece |
Compare | 0e24ea7c2a0436a8f2bf83e8f5981ec035518b99 |
Diff | Diff |
Improvements in IfStatements.IfStatements
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Single - Duration of single invocation | 65.43 μs | 32.24 μs | 0.49 | 0.00 | False | |||||
SingleArray - Duration of single invocation | 78.21 μs | 35.58 μs | 0.45 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'IfStatements.IfStatements*'
Related Issues
Regressions
- [Perf] Linux/arm64: 4 Regressions on 11/1/2022 1:15:18 PM
- [Perf] Windows/x64: 2 Regressions on 11/1/2022 1:15:18 PM
- [Perf] Windows/arm64: 1 Regression on 11/1/2022 1:15:18 PM
Improvements
Payloads
Histogram
Edge Detector Info
IfStatements.IfStatements.Single
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 32.24360716735254 < 61.848335634941215.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsImprovementStdDev: Marked as improvement because 322.04281412524614 (T) = (0 -32265.290136511227) / Math.Sqrt((2169224.6288032285 / (248)) + (8303.988088221295 / (29))) is greater than 1.9686278708474185 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (248) + (29) - 2, .975) and 0.48682034075998704 = (62873.28337271611 - 32265.290136511227) / 62873.28337271611 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### IfStatements.IfStatements.SingleArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.579103274487466 < 74.15371607016998.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsImprovementStdDev: Marked as improvement because 460.6068564118743 (T) = (0 -35520.21771321987) / Math.Sqrt((1916274.9811927783 / (248)) + (523.5674591827595 / (12))) is greater than 1.9692013862338849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (248) + (12) - 2, .975) and 0.5333841066167349 = (76123.03442061406 - 35520.21771321987) / 76123.03442061406 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | 7d5efbb9e10b6d8beb91c90cbdefd7360869cece |
Compare | 0e24ea7c2a0436a8f2bf83e8f5981ec035518b99 |
Diff | Diff |
Improvements in System.Tests.Perf_Random
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Next_long - Duration of single invocation | 48.09 ns | 35.97 ns | 0.75 | 0.02 | False | |||||
NextBytes_span - Duration of single invocation | 8.20 μs | 4.19 μs | 0.51 | 0.00 | False | |||||
NextDouble - Duration of single invocation | 9.95 ns | 5.48 ns | 0.55 | 0.09 | False | |||||
Next_int_int - Duration of single invocation | 11.31 ns | 6.85 ns | 0.61 | 0.04 | False | |||||
Next_long_long - Duration of single invocation | 52.76 ns | 40.62 ns | 0.77 | 0.02 | False | |||||
NextSingle - Duration of single invocation | 10.31 ns | 5.47 ns | 0.53 | 0.06 | False | |||||
NextBytes - Duration of single invocation | 8.09 μs | 4.34 μs | 0.54 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Random*'
Related Issues
Regressions
- [Perf] Linux/arm64: 4 Regressions on 11/1/2022 1:15:18 PM
- [Perf] Windows/x64: 2 Regressions on 11/1/2022 1:15:18 PM
- [Perf] Windows/arm64: 1 Regression on 11/1/2022 1:15:18 PM
Improvements
Payloads
Histogram
Edge Detector Info
System.Tests.Perf_Random.Next_long
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.96697726093585 < 45.69433633762902.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsImprovementStdDev: Marked as improvement because 159.90437584277848 (T) = (0 -36.185021005774345) / Math.Sqrt((0.3689284038129063 / (299)) + (0.12865912497105744 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.24721327843238497 = (48.06809149133513 - 36.185021005774345) / 48.06809149133513 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.NextBytes_span
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.191785717880148 < 7.794704944073333.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsImprovementStdDev: Marked as improvement because 1240.8423326247803 (T) = (0 -4193.968212114851) / Math.Sqrt((1325.8635453480001 / (299)) + (39.771960758686646 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.4848056764408173 = (8140.5559423503055 - 4193.968212114851) / 8140.5559423503055 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.NextDouble
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.480201574563121 < 9.343759610936175.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsImprovementStdDev: Marked as improvement because 106.55533984739628 (T) = (0 -5.454400606891136) / Math.Sqrt((0.0679117371993256 / (299)) + (0.04367706971842197 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.444890434743205 = (9.825809080353205 - 5.454400606891136) / 9.825809080353205 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.Next_int_int
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 6.848887828290941 < 10.497395256481296.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsImprovementStdDev: Marked as improvement because 58.96800885332623 (T) = (0 -6.8810543906213955) / Math.Sqrt((0.07655222310064919 / (299)) + (0.13402021989734336 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.37065838120083244 = (10.933734850955796 - 6.8810543906213955) / 10.933734850955796 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.Next_long_long
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 40.617226702758195 < 50.298054806309906.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsImprovementStdDev: Marked as improvement because 186.85518993588823 (T) = (0 -40.60267573906873) / Math.Sqrt((0.578957880947749 / (299)) + (0.01498110740037631 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.22709956546209423 = (52.532867009376005 - 40.60267573906873) / 52.532867009376005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.NextSingle
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.465740334277692 < 9.439761886047595.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsImprovementStdDev: Marked as improvement because 61.38157335255541 (T) = (0 -5.442078090930788) / Math.Sqrt((0.03189901407676259 / (299)) + (0.1503090866063253 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.4465436080178418 = (9.83289409928113 - 5.442078090930788) / 9.83289409928113 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Random.NextBytes
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.337037524211399 < 7.681368820761547.
IsChangePoint: Marked as a change because one of 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsImprovementStdDev: Marked as improvement because 615.9690733638695 (T) = (0 -4325.635204662771) / Math.Sqrt((1216.612759890411 / (299)) + (224.42638689908563 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.46118878108209094 = (8028.109016270884 - 4325.635204662771) / 8028.109016270884 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | ubuntu 20.04 |
Baseline | 7d5efbb9e10b6d8beb91c90cbdefd7360869cece |
Compare | 0e24ea7c2a0436a8f2bf83e8f5981ec035518b99 |
Diff | Diff |
Improvements in System.Perf_Convert
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ChangeType - Duration of single invocation | 36.60 ns | 33.58 ns | 0.92 | 0.48 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Perf_Convert*'
Related Issues
Regressions
- [Perf] Linux/arm64: 4 Regressions on 11/1/2022 1:15:18 PM
- [Perf] Windows/x64: 2 Regressions on 11/1/2022 1:15:18 PM
- [Perf] Windows/arm64: 1 Regression on 11/1/2022 1:15:18 PM
Improvements
Payloads
Histogram
Edge Detector Info
System.Perf_Convert.ChangeType
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.57579374431891 < 37.33007405662946.
IsChangePoint: Marked as a change because one of 10/6/2022 10:14:50 PM, 11/1/2022 8:41:52 AM, 11/9/2022 1:54:51 PM falls between 10/31/2022 10:23:25 PM and 11/9/2022 1:54:51 PM.
IsImprovementStdDev: Marked as improvement because 19.996806904403982 (T) = (0 -33.62141858753712) / Math.Sqrt((10.861177080278246 / (299)) + (3.611075022545403 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.19259306860472147 = (41.64123105735048 - 33.62141858753712) / 41.64123105735048 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository