-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | d6fdb3ab48ba885e5386a529ce53e143d56ab963 |
Compare | d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3 |
Diff | Diff |
Regressions in System.Tests.Perf_Int32
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TryFormat - Duration of single invocation | 3.67 ns | 4.87 ns | 1.33 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int32*'
Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Tests.Perf_Int32.TryFormat(value: 4)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.868059612986492 > 3.834267735765675.
IsChangePoint: Marked as a change because one of 10/3/2022 8:45:37 PM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -68.14795025109647 (T) = (0 -4.756428741619189) / Math.Sqrt((0.0005458045787438215 / (22)) + (0.007761579446670202 / (32))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (32) - 2, .025) and -0.30594541877839454 = (3.6421344056388207 - 4.756428741619189) / 3.6421344056388207 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | d6fdb3ab48ba885e5386a529ce53e143d56ab963 |
Compare | 2b07c8c46523b816b1eac5c8af092e52b20a69d2 |
Diff | Diff |
Regressions in System.Tests.Perf_DateTimeOffset
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
ToString - Duration of single invocation | 200.79 ns | 218.69 ns | 1.09 | 0.13 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_DateTimeOffset*'
Related Issues
Regressions
- [Perf] Windows/x64: 1 Regression on 10/11/2022 6:23:40 AM
- [Perf] Windows/x86: 1 Regression on 10/11/2022 6:23:40 AM
Improvements
Payloads
Histogram
Edge Detector Info
System.Tests.Perf_DateTimeOffset.ToString(format: "G")
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 218.69260068456393 > 212.2591562122658.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/26/2022 8:26:31 AM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -6.613850194065127 (T) = (0 -223.08247030936658) / Math.Sqrt((59.906310503079716 / (22)) + (63.483844739517714 / (31))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (31) - 2, .025) and -0.0692410733984532 = (208.63627095835926 - 223.08247030936658) / 208.63627095835926 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | Windows 10.0.19042 |
Baseline | d6fdb3ab48ba885e5386a529ce53e143d56ab963 |
Compare | d6c8ba1f7806c03b90047bcf8fa4eabdf0e143a3 |
Diff | Diff |
Regressions in System.IO.Tests.Perf_StreamWriter
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
WriteFormat - Duration of single invocation | 46.40 ns | 50.06 ns | 1.08 | 0.32 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_StreamWriter*'
Related Issues
Regressions
- [Perf] Windows/x64: 1 Regression on 10/11/2022 6:23:40 AM
- [Perf] Windows/x86: 1 Regression on 10/11/2022 6:23:40 AM
Improvements
Payloads
Histogram
Edge Detector Info
System.IO.Tests.Perf_StreamWriter.WriteFormat
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 50.0577586358479 > 48.13785145861373.
IsChangePoint: Marked as a change because one of 8/17/2022 6:09:21 PM, 9/29/2022 7:42:19 AM, 10/11/2022 2:16:14 AM, 10/18/2022 1:35:00 AM falls between 10/7/2022 9:25:17 PM and 10/18/2022 1:35:00 AM.
IsRegressionStdDev: Marked as regression because -10.824795971668838 (T) = (0 -50.326600126878624) / Math.Sqrt((2.0608984795036966 / (22)) + (0.1943162839597799 / (32))) is less than -2.0066468050606243 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (32) - 2, .025) and -0.07288368503413797 = (46.907787702333536 - 50.326600126878624) / 46.907787702333536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository