-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f |
Compare | c62f69be1405a8e41b56ffc05f22d791bf4c7d2d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Memory.Constructors<String>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
MemoryFromArray - Duration of single invocation | 5.31 ns | 8.56 ns | 1.61 | 0.05 | False | |||||
MemoryFromArrayStartLength - Duration of single invocation | 5.51 ns | 9.40 ns | 1.70 | 0.09 | False | |||||
SpanFromArrayStartLength - Duration of single invocation | 4.36 ns | 7.17 ns | 1.64 | 0.03 | False | |||||
ArrayAsSpan - Duration of single invocation | 5.00 ns | 7.17 ns | 1.43 | 0.04 | False | |||||
ArrayAsMemory - Duration of single invocation | 6.14 ns | 9.12 ns | 1.48 | 0.06 | False | |||||
ArrayAsSpanStartLength - Duration of single invocation | 6.70 ns | 8.84 ns | 1.32 | 0.02 | False | |||||
SpanFromArray - Duration of single invocation | 3.68 ns | 6.61 ns | 1.80 | 0.04 | False | |||||
ArrayAsMemoryStartLength - Duration of single invocation | 7.58 ns | 11.07 ns | 1.46 | 0.08 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors<String>*'
Payloads
Histogram
System.Memory.Constructors<String>.MemoryFromArray
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.5628071127807 > 5.576861737856464.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -114.29696864484852 (T) = (0 -8.56335967797707) / Math.Sqrt((0.012549942720872486 / (16)) + (6.253382033457137E-06 / (30))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (30) - 2, .025) and -0.5970871430828291 = (5.361861257894399 - 8.56335967797707) / 5.361861257894399 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Constructors<String>.MemoryFromArrayStartLength
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.398950750317228 > 5.791729372601059.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -56.14036431163663 (T) = (0 -9.45874529620186) / Math.Sqrt((0.01982487842745171 / (16)) + (0.10873033855118588 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.6917898214345194 = (5.590969502453624 - 9.45874529620186) / 5.590969502453624 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Constructors<String>.SpanFromArrayStartLength
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.169314647696989 > 4.580104133495271.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -450.55367231294326 (T) = (0 -7.1721130414871235) / Math.Sqrt((0.0006130631234442999 / (16)) + (7.0028805852530025E-06 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.6393535919007765 = (4.374964057126501 - 7.1721130414871235) / 4.374964057126501 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Constructors<String>.ArrayAsSpan
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.169400897681058 > 5.25197093830835.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -14.58038539395956 (T) = (0 -7.178042033747532) / Math.Sqrt((0.3138218647088309 / (16)) + (0.0011417692380134845 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.3980975048374429 = (5.134149806369996 - 7.178042033747532) / 5.134149806369996 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Constructors<String>.ArrayAsMemory
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.119449323928043 > 6.45028325642643.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -186.3893540830592 (T) = (0 -9.121704730905492) / Math.Sqrt((0.0041578465481122324 / (16)) + (1.4438612304434867E-05 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.4918511186238148 = (6.1143532468038595 - 9.121704730905492) / 6.1143532468038595 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Constructors<String>.ArrayAsSpanStartLength
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.839990538722486 > 7.038701374398039.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -1400.091479845232 (T) = (0 -8.842444950641594) / Math.Sqrt((3.2044950324718434E-05 / (15)) + (5.839239863127528E-06 / (31))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (31) - 2, .025) and -0.3182463118426248 = (6.707733502612086 - 8.842444950641594) / 6.707733502612086 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Constructors<String>.SpanFromArray
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.611695106307649 > 3.854121377477046.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -3345.7347868060615 (T) = (0 -6.614649111595471) / Math.Sqrt((8.693099725988118E-06 / (16)) + (7.143163256485408E-06 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.801546236905135 = (3.671651038476112 - 6.614649111595471) / 3.671651038476112 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Memory.Constructors<String>.ArrayAsMemoryStartLength
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.07024710086991 > 7.964445498962209.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -47.237088328240226 (T) = (0 -11.14503850073057) / Math.Sqrt((0.0015381990195552661 / (16)) + (0.1746076273820724 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.4723091869938087 = (7.569767681397641 - 11.14503850073057) / 7.569767681397641 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f |
Compare | c62f69be1405a8e41b56ffc05f22d791bf4c7d2d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Numerics.Tests.Perf_BigInteger
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Add - Duration of single invocation | 12.39 ns | 15.58 ns | 1.26 | 0.08 | True | |||||
Remainder - Duration of single invocation | 6.93 ns | 10.23 ns | 1.48 | 0.07 | True | |||||
Ctor_ByteArray - Duration of single invocation | 10.81 ns | 12.80 ns | 1.18 | 0.05 | False | |||||
Divide - Duration of single invocation | 7.49 ns | 11.91 ns | 1.59 | 0.05 | True | |||||
Subtract - Duration of single invocation | 54.41 ns | 60.57 ns | 1.11 | 0.02 | False | |||||
Ctor_ByteArray - Duration of single invocation | 7.92 ns | 9.83 ns | 1.24 | 0.03 | True | |||||
Subtract - Duration of single invocation | 13.37 ns | 16.34 ns | 1.22 | 0.09 | False | |||||
Multiply - Duration of single invocation | 10.07 ns | 13.86 ns | 1.38 | 0.01 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'
Payloads
Histogram
System.Numerics.Tests.Perf_BigInteger.Add(arguments: 16,16 bits)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.581370779445141 > 12.968371075357716.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -20.906313446665564 (T) = (0 -15.470450822042853) / Math.Sqrt((0.0019033542101310893 / (16)) + (0.6887269187340401 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.2530783519578782 = (12.34595649814792 - 15.470450822042853) / 12.34595649814792 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 16,8 bits)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.23349919324876 > 7.278155594520967.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -23.40402924517638 (T) = (0 -11.065734584093102) / Math.Sqrt((8.108752297878292E-05 / (16)) + (0.9360561467251515 / (30))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (30) - 2, .025) and -0.5964878245485067 = (6.931299076598055 - 11.065734584093102) / 6.931299076598055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: -2147483648)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.79576397472724 > 11.369937956395981.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 3/5/2023 11:54:18 PM, 3/31/2023 8:05:26 PM, 4/4/2023 4:23:54 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -17.511841009317905 (T) = (0 -11.762767580954609) / Math.Sqrt((0.0010535368992931466 / (16)) + (0.08816747842965245 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.0873220373615365 = (10.818108321889431 - 11.762767580954609) / 10.818108321889431 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 16,8 bits)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.907938729981035 > 7.861394233866898.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -4866.475256836527 (T) = (0 -11.90852583148143) / Math.Sqrt((6.372669969972835E-06 / (16)) + (1.3245857942141041E-05 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.5906065021392861 = (7.486783070146551 - 11.90852583148143) / 7.486783070146551 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 1024,1024 bits)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.56822193041907 > 57.079355939834144.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 3/25/2023 12:18:49 AM, 4/7/2023 3:57:10 PM, 4/9/2023 11:04:55 PM, 4/25/2023 3:46:39 AM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -90.45165337700212 (T) = (0 -60.968442012715535) / Math.Sqrt((0.03994901728245559 / (16)) + (0.08816064862423996 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.1216047585788907 = (54.35822338161663 - 60.968442012715535) / 54.35822338161663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 123)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.833492807799011 > 8.372652657595594.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -78.1752056227852 (T) = (0 -9.923811339662212) / Math.Sqrt((0.00987403611657527 / (16)) + (0.0012024263169792366 / (30))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (30) - 2, .025) and -0.2530529622204308 = (7.919706220619001 - 9.923811339662212) / 7.919706220619001 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 16,16 bits)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.339767242826124 > 14.040911395721242.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -9.864867333782232 (T) = (0 -14.98089511806153) / Math.Sqrt((0.27767234083977627 / (16)) + (0.5850881451991788 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.14329685789724605 = (13.103241747392207 - 14.98089511806153) / 13.103241747392207 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 16,16 bits)
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.863552389845244 > 10.58804450760119.
IsChangePoint: Marked as a change because one of 4/9/2023 11:04:55 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -651.6642675225054 (T) = (0 -13.857738455304965) / Math.Sqrt((0.0005191068133591713 / (16)) + (2.658043329776799E-05 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.37244142024577526 = (10.09714385683823 - 13.857738455304965) / 10.09714385683823 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f |
Compare | c62f69be1405a8e41b56ffc05f22d791bf4c7d2d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
FirstSpanArray - Duration of single invocation | 6.57 ns | 11.47 ns | 1.75 | 0.01 | True | |||||
FirstSpanTenSegments - Duration of single invocation | 7.22 ns | 10.07 ns | 1.40 | 0.08 | False | |||||
FirstSpanMemory - Duration of single invocation | 7.28 ns | 12.21 ns | 1.68 | 0.02 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests<Char>*'
Payloads
Histogram
System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanArray
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.47276283046604 > 6.927530188944186.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -127.05603998230478 (T) = (0 -11.35228783246569) / Math.Sqrt((0.0058877183726034465 / (16)) + (0.032136680094136456 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.7225495331956076 = (6.59039848416165 - 11.35228783246569) / 6.59039848416165 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanTenSegments
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.073611038211213 > 7.683240410027857.
IsChangePoint: Marked as a change because one of 4/17/2023 1:54:35 PM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -25.44183933876455 (T) = (0 -9.815846024266609) / Math.Sqrt((0.10434611339523595 / (16)) + (0.05441361379801162 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.3085682732211152 = (7.501210464246046 - 9.815846024266609) / 7.501210464246046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanMemory
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.209979172708131 > 7.640756237301494.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -155.26474471812668 (T) = (0 -12.341566913945524) / Math.Sqrt((0.003177384365732622 / (16)) + (0.027036648437336514 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.6997116384048677 = (7.260976882836281 - 12.341566913945524) / 7.260976882836281 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f |
Compare | c62f69be1405a8e41b56ffc05f22d791bf4c7d2d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Tests.Perf_Boolean
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
TryParse - Duration of single invocation | 8.31 ns | 15.67 ns | 1.89 | 0.02 | True | |||||
TryParse - Duration of single invocation | 9.98 ns | 24.83 ns | 2.49 | 0.03 | True | |||||
TryParse - Duration of single invocation | 2.45 ns | 10.74 ns | 4.38 | 0.08 | True | |||||
Parse - Duration of single invocation | 10.53 ns | 22.56 ns | 2.14 | 0.02 | True |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Boolean*'
Payloads
Histogram
System.Tests.Perf_Boolean.TryParse(value: "Bogus")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.669525574502511 > 8.71929353058046.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -248.5314573566136 (T) = (0 -15.730624207104661) / Math.Sqrt((0.014032100358382444 / (16)) + (0.0009087461660479719 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.9070786922821502 = (8.248544892649523 - 15.730624207104661) / 8.248544892649523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Boolean.TryParse(value: " True ")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.82722600439258 > 10.475032178008643.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -175.3090670126055 (T) = (0 -25.11984107718479) / Math.Sqrt((6.0728805274680866E-05 / (16)) + (0.23134257830076416 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -1.5191342606331608 = (9.971616626289364 - 25.11984107718479) / 9.971616626289364 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Boolean.TryParse(value: "0")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.739960949854812 > 2.5762244072223277.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -106.701793528524 (T) = (0 -10.51461447405426) / Math.Sqrt((0.01847957294244507 / (15)) + (0.13736244637440556 / (31))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (31) - 2, .025) and -3.231264784754782 = (2.4849814438317224 - 10.51461447405426) / 2.4849814438317224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Boolean.Parse(value: " True ")
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.558552721057534 > 11.056567713801948.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -116.57583910448999 (T) = (0 -23.493254163881762) / Math.Sqrt((0.010919178652641477 / (16)) + (0.35930568893834797 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -1.2208303888434087 = (10.578590009350902 - 23.493254163881762) / 10.578590009350902 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Name | Value |
---|---|
Architecture | x64 |
OS | Windows 10.0.19042 |
Queue | OwlWindows |
Baseline | da0aa0cb6944dd49d6c1d1859c4530fe7e38b76f |
Compare | c62f69be1405a8e41b56ffc05f22d791bf4c7d2d |
Diff | Diff |
Configs | CompilationMode:tiered, RunKind:micro |
Regressions in MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
SystemTextJson_SourceGen_ - Duration of single invocation | 126.97 μs | 142.66 μs | 1.12 | 0.30 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>*'
Payloads
Histogram
MicroBenchmarks.Serializers.Json_ToStream<CollectionsOfPrimitives>.SystemTextJson_SourceGen_
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 142.66362012987014 > 133.5468004416823.
IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 5/2/2023 4:25:00 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -8.580090999982545 (T) = (0 -140300.62568017803) / Math.Sqrt((21055529.5147007 / (16)) + (2030181.3722807285 / (31))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (31) - 2, .025) and -0.07744561851113702 = (130215.96939068884 - 140300.62568017803) / 130215.96939068884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository