-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf3328cceecccf781afa3e1ba619cf42830ead60 |
Compare | 0241bc154b2669e12201706ba95dd23e043fae3d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.IO.Tests.BinaryReaderTests
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
52.36 ns | 47.92 ns | 0.92 | 0.01 | True | ||||
94.76 ns | 80.08 ns | 0.85 | 0.01 | True | ||||
2.75 ns | 1.32 ns | 0.48 | 0.09 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryReaderTests*'
Payloads
System.IO.Tests.BinaryReaderTests.ReadSmallString
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 47.917440235219075 < 49.44419698320688.
IsChangePoint: Marked as a change because one of 8/30/2023 5:24:30 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 17.53261530247232 (T) = (0 -46.540199465879695) / Math.Sqrt((0.41679839257158424 / (58)) + (0.5854731269450498 / (8))) is greater than 1.9977296543170047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (58) + (8) - 2, .975) and 0.09649340650121499 = (51.51063622641098 - 46.540199465879695) / 51.51063622641098 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.IO.Tests.BinaryReaderTests.ReadNonAsciiChar
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 80.07758313650508 < 90.39422047934944.
IsChangePoint: Marked as a change because one of 8/30/2023 5:24:30 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 108.18737352851154 (T) = (0 -79.77457240080219) / Math.Sqrt((0.9721922457147384 / (59)) + (0.039336612983788474 / (8))) is greater than 1.9971379083914016 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (59) + (8) - 2, .975) and 0.1655304295558794 = (95.59913893366374 - 79.77457240080219) / 95.59913893366374 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.IO.Tests.BinaryReaderTests.ReadBool
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.317978801681393 < 2.564796952095436.
IsChangePoint: Marked as a change because one of 8/30/2023 5:24:30 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 26.23775471891111 (T) = (0 -1.353705798000354) / Math.Sqrt((0.0685859783423745 / (58)) + (0.0007450433818333428 / (8))) is greater than 1.9977296543170047 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (58) + (8) - 2, .975) and 0.409073390883556 = (2.2908188210113276 - 1.353705798000354) / 2.2908188210113276 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf3328cceecccf781afa3e1ba619cf42830ead60 |
Compare | 0241bc154b2669e12201706ba95dd23e043fae3d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_String
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
5.51 ns | 3.88 ns | 0.70 | 0.43 | False | ||||
7.76 ns | 6.35 ns | 0.82 | 0.12 | False | ||||
18.20 ns | 16.36 ns | 0.90 | 0.07 | False | ||||
18.53 ns | 16.67 ns | 0.90 | 0.06 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
Payloads
System.Tests.Perf_String.TrimEnd_CharArr(s: "Test", c: [' ', 'β '])
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.876685214166383 < 5.192230605909357.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/22/2023 12:06:43 AM, 8/30/2023 5:24:30 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 29.421095099575346 (T) = (0 -3.892628931424457) / Math.Sqrt((0.35717610709238573 / (299)) + (0.013015461567799565 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.2864648551512167 = (5.4554130368020015 - 3.892628931424457) / 5.4554130368020015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_String.Trim_CharArr(s: "Test", c: [' ', 'β '])
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.345169231921776 < 7.526853522769098.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 12:51:17 PM, 6/22/2023 12:06:43 AM, 8/29/2023 4:36:45 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 13.258702170627613 (T) = (0 -6.374285081702449) / Math.Sqrt((0.32981399505924913 / (299)) + (0.08345447387707801 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.17483242563009802 = (7.724837087266612 - 6.374285081702449) / 7.724837087266612 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', 'β '])
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.355646154412554 < 17.271431109666487.
IsChangePoint: Marked as a change because one of 5/2/2023 12:51:17 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/22/2023 12:06:43 AM, 8/28/2023 2:57:38 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 7.596737157659804 (T) = (0 -16.392763059076877) / Math.Sqrt((0.9868712296858162 / (299)) + (0.20732982394079288 / (11))) is greater than 1.9676960046166507 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (11) - 2, .975) and 0.06451855251982867 = (17.52334383886789 - 16.392763059076877) / 17.52334383886789 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_String.Trim_CharArr(s: " Test", c: [' ', 'β '])
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.665162904178864 < 17.444206404528067.
IsChangePoint: Marked as a change because one of 5/2/2023 12:51:17 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/22/2023 12:06:43 AM, 8/30/2023 5:24:30 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 10.631096571251883 (T) = (0 -16.15242782952352) / Math.Sqrt((1.2378542224395621 / (299)) + (0.09380157367339113 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.07655503148580001 = (17.4914893472346 - 16.15242782952352) / 17.4914893472346 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf3328cceecccf781afa3e1ba619cf42830ead60 |
Compare | 0241bc154b2669e12201706ba95dd23e043fae3d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Memory.Span<Int32>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
24.04 ns | 18.99 ns | 0.79 | 0.07 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Int32>*'
Payloads
System.Memory.Span<Int32>.BinarySearch(Size: 512)
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.989617648365464 < 22.59586061677242.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/29/2023 11:09:12 AM, 8/30/2023 5:24:30 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 34.16010066727141 (T) = (0 -18.98701012581365) / Math.Sqrt((1.1379233805861557 / (299)) + (0.0030577973708013893 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.10428786086638417 = (21.197669760486864 - 18.98701012581365) / 21.197669760486864 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf3328cceecccf781afa3e1ba619cf42830ead60 |
Compare | 0241bc154b2669e12201706ba95dd23e043fae3d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Double
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
131.43 ns | 118.49 ns | 0.90 | 0.02 | False | ||||
119.17 ns | 111.79 ns | 0.94 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'
Payloads
System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 118.48698066859966 < 124.58952036480166.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 8/30/2023 5:24:30 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 27.718178642636772 (T) = (0 -118.1442230043765) / Math.Sqrt((72.0845169696637 / (299)) + (0.7899097181050145 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.12031175139132523 = (134.30237722424368 - 118.1442230043765) / 134.30237722424368 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 111.79447267812297 < 113.47710247876476.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 8/30/2023 5:24:30 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 23.610834147947866 (T) = (0 -111.84777797200366) / Math.Sqrt((22.09274326788706 / (299)) + (0.7202434645574094 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.07873761086321734 = (121.40708151214591 - 111.84777797200366) / 121.40708151214591 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf3328cceecccf781afa3e1ba619cf42830ead60 |
Compare | 0241bc154b2669e12201706ba95dd23e043fae3d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Tests.Perf_StringBuilder
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
29.82 ΞΌs | 20.35 ΞΌs | 0.68 | 0.07 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
Payloads
System.Text.Tests.Perf_StringBuilder.Insert_Primitives
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.34625309770972 < 28.28657254704417.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 8/30/2023 5:24:30 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 85.45700277304354 (T) = (0 -20414.861689320518) / Math.Sqrt((3395401.698000846 / (299)) + (7075.153330866126 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.3165304908572268 = (29869.454915297414 - 20414.861689320518) / 29869.454915297414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf3328cceecccf781afa3e1ba619cf42830ead60 |
Compare | 0241bc154b2669e12201706ba95dd23e043fae3d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Enum
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
21.18 ns | 16.42 ns | 0.78 | 0.04 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
Payloads
System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red")
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.42054639184891 < 20.038784270296155.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/30/2023 5:24:30 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 5.5051872336831424 (T) = (0 -17.038583603462033) / Math.Sqrt((3.9224549287005863 / (299)) + (0.13854657340954973 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.05336091235090976 = (17.999028167932643 - 17.038583603462033) / 17.999028167932643 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf3328cceecccf781afa3e1ba619cf42830ead60 |
Compare | 0241bc154b2669e12201706ba95dd23e043fae3d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Numerics.Tests.Perf_Matrix4x4
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
16.11 ns | 14.97 ns | 0.93 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'
Payloads
System.Numerics.Tests.Perf_Matrix4x4.CreateShadowBenchmark
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.967080413853978 < 15.31080670590344.
IsChangePoint: Marked as a change because one of 6/28/2023 11:43:34 AM, 8/30/2023 5:24:30 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 14.421913340356534 (T) = (0 -14.842733977257405) / Math.Sqrt((0.1682468844134552 / (299)) + (0.022498809960510544 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.05343193735660336 = (15.680577618272286 - 14.842733977257405) / 15.680577618272286 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | arm64 |
OS | Windows 10.0.19041 |
Queue | SurfaceWindows |
Baseline | cf3328cceecccf781afa3e1ba619cf42830ead60 |
Compare | 0241bc154b2669e12201706ba95dd23e043fae3d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Improvements in IfStatements.IfStatements
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio |
---|---|---|---|---|---|---|---|---|
87.98 ΞΌs | 75.91 ΞΌs | 0.86 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'IfStatements.IfStatements*'
Payloads
IfStatements.IfStatements.AndOr
ETL Files
Histogram
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 75.91105935127673 < 83.5356585110816.
IsChangePoint: Marked as a change because one of 5/22/2023 10:35:01 AM, 6/22/2023 12:06:43 AM, 7/14/2023 7:13:27 PM, 8/30/2023 5:24:30 PM, 9/5/2023 8:21:55 PM falls between 8/27/2023 2:42:29 PM and 9/5/2023 8:21:55 PM.
IsImprovementStdDev: Marked as improvement because 14.60813312011734 (T) = (0 -75848.95044762327) / Math.Sqrt((47670756.2934858 / (299)) + (1414.3990465020984 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.07144683725729083 = (81685.09191609969 - 75848.95044762327) / 81685.09191609969 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository