Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests #85988
Description
Run Information
| Name | Value |
|---|---|
| Architecture | x64 |
| OS | Windows 10.0.18362 |
| Queue | TigerWindows |
| Baseline | 442141d696ee7d4ae54420974559e99738d753e6 |
| Compare | c8f43d52f699c1da289a5cb36d371aec15901ab8 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| MeasureReverseUsingNtoH - Duration of single invocation | 605.61 ns | 946.10 ns | 1.56 | 0.01 | True | 6022.827101520892 | 8089.7074446351235 | 1.3431744442061604) | Trace | Trace |
| MeasureReverseEndianness - Duration of single invocation | 614.26 ns | 941.83 ns | 1.53 | 0.00 | True | 6056.527359489527 | 8075.261900385958 | 1.333315515818388) | Trace | Trace |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'Payloads
Histogram
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.MeasureReverseUsingNtoH
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 946.1020211179717 > 643.259435520526.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -145.33095665139552 (T) = (0 -943.5256872067196) / Math.Sqrt((100.76093042903165 / (21)) + (1.9921542971535473 / (13))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (13) - 2, .025) and -0.5214730941818898 = (620.1395810512588 - 943.5256872067196) / 620.1395810512588 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
System.Buffers.Binary.Tests.BinaryReadAndWriteTests.MeasureReverseEndianness
Description of detection logic
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 941.8282251056002 > 645.1990668248882.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/9/2023 7:24:34 AM falls between 4/30/2023 6:17:41 PM and 5/9/2023 7:24:34 AM.
IsRegressionStdDev: Marked as regression because -172.69616839909932 (T) = (0 -942.899495537063) / Math.Sqrt((71.28264791595957 / (21)) + (0.9109916109751146 / (13))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (13) - 2, .025) and -0.5172382205961809 = (621.4577794952738 - 942.899495537063) / 621.4577794952738 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository