-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 43f7699377fff108c74e19bea9f7ecf66286952f |
Compare | 2b07c8c46523b816b1eac5c8af092e52b20a69d2 |
Diff | Diff |
Improvements in System.Tests.Perf_Version
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TryFormat3 - Duration of single invocation | 18.85 ns | 17.07 ns | 0.91 | 0.14 | False | |||||
TryFormat2 - Duration of single invocation | 11.92 ns | 10.25 ns | 0.86 | 0.23 | False | |||||
TryFormatL - Duration of single invocation | 62.36 ns | 50.87 ns | 0.82 | 0.01 | False | |||||
TryFormat4 - Duration of single invocation | 25.01 ns | 23.20 ns | 0.93 | 0.04 | False | |||||
ToStringL - Duration of single invocation | 86.93 ns | 72.09 ns | 0.83 | 0.01 | False | |||||
ToString2 - Duration of single invocation | 32.60 ns | 28.56 ns | 0.88 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Version*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Tests.Perf_Version.TryFormat3
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 17.06638112810962 < 17.892721410717233.
IsChangePoint: Marked as a change because one of 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 17.697243500014526 (T) = (0 -16.91536973694947) / Math.Sqrt((0.16065483781687928 / (299)) + (0.06530955405024776 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10873914881202774 = (18.97914590818476 - 16.91536973694947) / 18.97914590818476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Version.TryFormat2
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.246444202435388 < 11.442564222643483.
IsChangePoint: Marked as a change because one of 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 13.451928632901865 (T) = (0 -10.288632279341902) / Math.Sqrt((0.1770925878245822 / (299)) + (0.08021546349340716 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14429954946088108 = (12.023637796216809 - 10.288632279341902) / 12.023637796216809 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Version.TryFormatL
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 50.8710180369288 < 67.78322694734683.
IsChangePoint: Marked as a change because one of 10/3/2022 10:42:38 PM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 68.67379839638392 (T) = (0 -50.24466252209767) / Math.Sqrt((6.159951316917306 / (299)) + (0.3455347947921297 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2904643738058036 = (70.81344567798482 - 50.24466252209767) / 70.81344567798482 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Version.TryFormat4
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.199487934940972 < 23.816478177820894.
IsChangePoint: Marked as a change because one of 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 15.4936951394833 (T) = (0 -22.779656832746213) / Math.Sqrt((0.18110206065314202 / (299)) + (0.10249856957524854 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08992526886345803 = (25.030534365345975 - 22.779656832746213) / 25.030534365345975 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Version.ToStringL
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 72.09211880786636 < 93.38333493187469.
IsChangePoint: Marked as a change because one of 10/3/2022 10:42:38 PM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 81.18924870442027 (T) = (0 -72.82785638557093) / Math.Sqrt((10.45784004923226 / (299)) + (0.3024866067959365 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2562084167633833 = (97.91433249171732 - 72.82785638557093) / 97.91433249171732 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Version.ToString2
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 28.556364100745164 < 31.947910649691963.
IsChangePoint: Marked as a change because one of 10/3/2022 10:42:38 PM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 15.367670866230098 (T) = (0 -29.57715123170422) / Math.Sqrt((0.5669816591685818 / (299)) + (0.37628397856504553 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12611901317979038 = (33.845742930426475 - 29.57715123170422) / 33.845742930426475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 43f7699377fff108c74e19bea9f7ecf66286952f |
Compare | 2b07c8c46523b816b1eac5c8af092e52b20a69d2 |
Diff | Diff |
Improvements in System.Tests.Perf_UInt32
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToString - Duration of single invocation | 18.71 ns | 16.24 ns | 0.87 | 0.03 | False | |||||
ToString - Duration of single invocation | 13.52 ns | 11.46 ns | 0.85 | 0.05 | False | |||||
TryFormat - Duration of single invocation | 9.99 ns | 8.08 ns | 0.81 | 0.07 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Tests.Perf_UInt32.ToString(value: 4294967295)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 16.243592618497736 < 19.677636428493813.
IsChangePoint: Marked as a change because one of 10/4/2022 3:52:14 AM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 59.22540759693244 (T) = (0 -16.322686948599774) / Math.Sqrt((0.5169030596895788 / (299)) + (0.02342586491716565 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.22515993602980727 = (21.065878892431265 - 16.322686948599774) / 21.065878892431265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_UInt32.ToString(value: 12345)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.462557127580794 < 13.660086711638533.
IsChangePoint: Marked as a change because one of 9/7/2022 11:31:53 PM, 10/3/2022 10:42:38 PM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 8.264662990405762 (T) = (0 -11.760149255386931) / Math.Sqrt((0.19993486718381986 / (299)) + (0.5403442183866631 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1881407161461795 = (14.48545270008196 - 11.760149255386931) / 14.48545270008196 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.075498956292847 < 9.920022748314564.
IsChangePoint: Marked as a change because one of 10/3/2022 10:42:38 PM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 116.77642893460272 (T) = (0 -8.052247852403983) / Math.Sqrt((0.12271996384753832 / (299)) + (0.0004267326022807325 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.24799955532425536 = (10.707770067710578 - 8.052247852403983) / 10.707770067710578 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 43f7699377fff108c74e19bea9f7ecf66286952f |
Compare | 2b07c8c46523b816b1eac5c8af092e52b20a69d2 |
Diff | Diff |
Improvements in System.Text.Tests.Perf_StringBuilder
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Append_Primitives - Duration of single invocation | 1.17 μs | 1.06 μs | 0.91 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Text.Tests.Perf_StringBuilder.Append_Primitives
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.060918831902989 < 1.1419019120143032.
IsChangePoint: Marked as a change because one of 8/17/2022 3:53:31 PM, 10/3/2022 10:42:38 PM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 86.5936323004948 (T) = (0 -1063.5519130203238) / Math.Sqrt((909.4425817979622 / (299)) + (4.441185697311401 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13897345482449802 = (1235.2138490730752 - 1063.5519130203238) / 1235.2138490730752 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 43f7699377fff108c74e19bea9f7ecf66286952f |
Compare | 2b07c8c46523b816b1eac5c8af092e52b20a69d2 |
Diff | Diff |
Improvements in System.Tests.Perf_Int64
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToString - Duration of single invocation | 61.66 ns | 34.15 ns | 0.55 | 0.00 | False | |||||
TryFormat - Duration of single invocation | 47.54 ns | 25.15 ns | 0.53 | 0.00 | False | |||||
ToString - Duration of single invocation | 18.68 ns | 13.69 ns | 0.73 | 0.16 | False | |||||
ToString - Duration of single invocation | 70.92 ns | 46.31 ns | 0.65 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int64*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Tests.Perf_Int64.ToString(value: 9223372036854775807)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.14501018739127 < 57.57793724902923.
IsChangePoint: Marked as a change because one of 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 103.28609691294528 (T) = (0 -34.46011771215059) / Math.Sqrt((0.23309822317915718 / (299)) + (0.2644315652765322 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.4366740009334165 = (61.17260302071999 - 34.46011771215059) / 61.17260302071999 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.150980735735562 < 44.642905596770774.
IsChangePoint: Marked as a change because one of 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 551.8116595795784 (T) = (0 -25.159265548133828) / Math.Sqrt((0.19505892504426756 / (258)) + (0.00397759789509718 / (5))) is greater than 1.9690947240420478 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (258) + (5) - 2, .975) and 0.4634985446315777 = (46.895055542499215 - 25.159265548133828) / 46.895055542499215 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int64.ToString(value: 12345)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 13.687311019000822 < 17.00087489238104.
IsChangePoint: Marked as a change because one of 10/4/2022 3:52:14 AM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 14.31982908126187 (T) = (0 -14.432018889927019) / Math.Sqrt((0.19146483960725977 / (299)) + (0.31262297263857414 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.19959828961336418 = (18.030969577708174 - 14.432018889927019) / 18.030969577708174 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int64.ToString(value: -9223372036854775808)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 46.308578164282714 < 67.71673340739014.
IsChangePoint: Marked as a change because one of 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 85.53085320343915 (T) = (0 -45.73061384009955) / Math.Sqrt((0.26466783092705337 / (299)) + (0.4479489865795304 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.3600304946519263 = (71.45748892398721 - 45.73061384009955) / 71.45748892398721 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 43f7699377fff108c74e19bea9f7ecf66286952f |
Compare | 2b07c8c46523b816b1eac5c8af092e52b20a69d2 |
Diff | Diff |
Improvements in PerfLabTests.LowLevelPerf
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
IntegerFormatting - Duration of single invocation | 2.12 ms | 1.71 ms | 0.81 | 0.42 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
PerfLabTests.LowLevelPerf.IntegerFormatting
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7126947544642857 < 2.057822501717033.
IsChangePoint: Marked as a change because one of 9/29/2022 6:03:53 PM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 12.45034320829006 (T) = (0 -1790249.3305479244) / Math.Sqrt((4912412972.3347 / (299)) + (5138661647.083049 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.18349028013974808 = (2192563.4037208166 - 1790249.3305479244) / 2192563.4037208166 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 43f7699377fff108c74e19bea9f7ecf66286952f |
Compare | 2b07c8c46523b816b1eac5c8af092e52b20a69d2 |
Diff | Diff |
Improvements in System.Buffers.Text.Tests.Utf8FormatterTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
FormatterDecimal - Duration of single invocation | 52.76 ns | 49.04 ns | 0.93 | 0.00 | False | |||||
FormatterInt32 - Duration of single invocation | 3.38 ns | 1.52 ns | 0.45 | 0.55 | False | |||||
FormatterInt32 - Duration of single invocation | 20.70 ns | 19.33 ns | 0.93 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDecimal(value: 123456.789)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 49.03550187368838 < 49.83132927921636.
IsChangePoint: Marked as a change because one of 9/29/2022 6:03:53 PM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 21.093636782326524 (T) = (0 -48.97100777740722) / Math.Sqrt((0.37480864883849074 / (299)) + (0.21132513448201604 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.08244804857380743 = (53.37137336070112 - 48.97100777740722) / 53.37137336070112 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 4)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5206143862966137 < 3.159393297824153.
IsChangePoint: Marked as a change because one of 8/17/2022 3:53:31 PM, 10/10/2022 3:36:43 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 19.261980405943195 (T) = (0 -1.4527159079022767) / Math.Sqrt((0.5799137536305943 / (299)) + (0.019239383643726352 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.47584959600595594 = (2.771563079666698 - 1.4527159079022767) / 2.771563079666698 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: -2147483648)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 19.333271855157953 < 19.62747460688072.
IsChangePoint: Marked as a change because one of 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 15.527949580536676 (T) = (0 -19.2073145254658) / Math.Sqrt((0.07826944778238651 / (299)) + (0.040306448166700405 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.06868848814092822 = (20.62394191512184 - 19.2073145254658) / 20.62394191512184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | arm64 |
---|---|
OS | Windows 10.0.19041 |
Baseline | 43f7699377fff108c74e19bea9f7ecf66286952f |
Compare | 2b07c8c46523b816b1eac5c8af092e52b20a69d2 |
Diff | Diff |
Improvements in System.Tests.Perf_Int32
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToString - Duration of single invocation | 18.88 ns | 15.78 ns | 0.84 | 0.25 | False | |||||
TryFormat - Duration of single invocation | 10.56 ns | 8.89 ns | 0.84 | 0.15 | False | |||||
ToString - Duration of single invocation | 13.58 ns | 11.61 ns | 0.85 | 0.09 | False | |||||
TryFormat - Duration of single invocation | 21.03 ns | 18.10 ns | 0.86 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Tests.Perf_Int32.ToString(value: 2147483647)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 15.776106451521697 < 18.69691318637421.
IsChangePoint: Marked as a change because one of 10/1/2022 3:26:37 AM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 23.46734490855513 (T) = (0 -16.354777954410707) / Math.Sqrt((0.45058095478011795 / (299)) + (0.12769796313394172 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.19092544221987406 = (20.214179023604004 - 16.354777954410707) / 20.214179023604004 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryFormat(value: 2147483647)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.888232942342928 < 10.785954378091953.
IsChangePoint: Marked as a change because one of 10/3/2022 10:42:38 PM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 32.78640898998213 (T) = (0 -8.96175599105338) / Math.Sqrt((0.18464278479554477 / (299)) + (0.028235477763578095 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.22454566559944228 = (11.556781093990535 - 8.96175599105338) / 11.556781093990535 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.ToString(value: 12345)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.60838838838839 < 12.913567599955948.
IsChangePoint: Marked as a change because one of 8/21/2022 5:29:20 AM, 8/23/2022 11:36:32 PM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 35.02648275485593 (T) = (0 -11.525082762849058) / Math.Sqrt((0.16881302439583382 / (299)) + (0.01961254649714083 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.16914548195449863 = (13.871360764770966 - 11.525082762849058) / 13.871360764770966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Int32.TryFormat(value: -2147483648)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 18.096376417920492 < 20.97964697406131.
IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 10/3/2022 10:42:38 PM, 10/10/2022 11:56:05 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM.
IsImprovementStdDev: Marked as improvement because 53.96298086862716 (T) = (0 -17.946126887359846) / Math.Sqrt((0.1773769427237612 / (299)) + (0.02461932288133293 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1825705921924263 = (21.954344578198047 - 17.946126887359846) / 21.954344578198047 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository