[Perf] Windows/arm64: 26 Improvements on 3/15/2023 2:24:48 AM #14420
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Memory.Constructors<String>
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors<String>*'Payloads
Histogram
System.Memory.Constructors<String>.ArrayAsSpan
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.047227260719542 < 10.302439236912008.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 47.89027098148056 (T) = (0 -6.913799454040009) / Math.Sqrt((0.08864767162478909 / (299)) + (0.1727998260755824 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.35776943319575294 = (10.765291799241544 - 6.913799454040009) / 10.765291799241544 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.MemoryFromArrayStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.863980202767626 < 9.707855285334391.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 50.73729694394819 (T) = (0 -6.0221999643889275) / Math.Sqrt((1.467311786236657 / (299)) + (0.15648455014548104 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.46327485102545 = (11.220267907875662 - 6.0221999643889275) / 11.220267907875662 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ReadOnlySpanImplicitCastFromArraySegment
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.4058406179733227 < 5.29713939590428.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 46.76760709129246 (T) = (0 -3.4568160401082544) / Math.Sqrt((0.07456588395639553 / (299)) + (0.04573162885534651 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.3735820730644239 = (5.5183861946910255 - 3.4568160401082544) / 5.5183861946910255 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsMemory
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.578105416676162 < 10.959713093948551.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 46.06162367629759 (T) = (0 -7.401102416979027) / Math.Sqrt((1.8578911217607588 / (299)) + (0.14978210664522906 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.40092332402440956 = (12.354182217036584 - 7.401102416979027) / 12.354182217036584 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsSpanStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 8.503544568487886 < 11.338750794752938.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 46.52571409139684 (T) = (0 -8.456773797717167) / Math.Sqrt((0.5417648369803071 / (299)) + (0.12017648232326764 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.30061457358838284 = (12.091721500556414 - 8.456773797717167) / 12.091721500556414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanFromMemory
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.1056443933585127 < 4.146885117550052.
IsChangePoint: Marked as a change because one of 12/20/2022 7:58:25 PM, 12/21/2022 9:59:33 PM, 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 5.326115970825466 (T) = (0 -3.9449281993783494) / Math.Sqrt((0.08489059544887868 / (299)) + (0.10642859756959748 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07943613197520666 = (4.2853389497491134 - 3.9449281993783494) / 4.2853389497491134 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.MemoryFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.5642771161801905 < 9.523688697860146.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 53.19540823803803 (T) = (0 -5.502321384096) / Math.Sqrt((1.74851601062982 / (299)) + (0.11358072636387982 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.4903538459567608 = (10.7963561393405 - 5.502321384096) / 10.7963561393405 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanImplicitCastFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.99211023762334 < 8.776347890539851.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 111.83594163961521 (T) = (0 -3.981428627975323) / Math.Sqrt((0.24172459237998703 / (299)) + (0.04646970134288859 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.5825460555583667 = (9.537408092527892 - 3.981428627975323) / 9.537408092527892 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.ArrayAsMemoryStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.824372408531051 < 11.70499036611286.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 37.87265224961908 (T) = (0 -9.11112874329891) / Math.Sqrt((3.6409602643218033 / (299)) + (0.07039345633375786 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.3350300935264446 = (13.70156552138836 - 9.11112874329891) / 13.70156552138836 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanFromArrayStartLength
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.771147662944652 < 9.334898435964284.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 67.75583547202338 (T) = (0 -5.661831020259747) / Math.Sqrt((0.16298372810835055 / (299)) + (0.0973406370293405 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.43146388794766544 = (9.958612830803908 - 5.661831020259747) / 9.958612830803908 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanImplicitCastFromArraySegment
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.494943446806453 < 11.215052735397167.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 113.41474020835057 (T) = (0 -5.64497979852714) / Math.Sqrt((0.36175659795554677 / (299)) + (0.04727674923061858 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.522272800488834 = (11.816324890655924 - 5.64497979852714) / 11.816324890655924 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Constructors<String>.SpanFromArray
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.069304086902723 < 8.9334618352584.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 158.19813320156115 (T) = (0 -5.035782835700757) / Math.Sqrt((0.12269833787698735 / (299)) + (0.010062708372263425 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.4656923641517481 = (9.424875292500898 - 5.035782835700757) / 9.424875292500898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsKeyTrue<String, String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IDictionary - Duration of single invocation | 16.31 μs | 14.82 μs | 0.91 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue<String, String>*'Payloads
Histogram
System.Collections.ContainsKeyTrue<String, String>.IDictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.822033445603362 < 15.435239031470743.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 40.414237372940484 (T) = (0 -14848.120637323433) / Math.Sqrt((42519.99080298896 / (299)) + (35924.09653286814 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09318016204888639 = (16373.837465743543 - 14848.120637323433) / 16373.837465743543 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.IterateFor<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IList - Duration of single invocation | 2.53 μs | 2.30 μs | 0.91 | 0.13 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor<String>*'Payloads
Histogram
System.Collections.IterateFor<String>.IList(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.299515020420192 < 2.3931557010201394.
IsChangePoint: Marked as a change because one of 1/1/2023 3:57:25 AM, 1/2/2023 8:42:52 PM, 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 7.26735256990309 (T) = (0 -2358.547854438239) / Math.Sqrt((97512.37298102825 / (299)) + (44970.70740811456 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.11928740642791157 = (2677.999465037951 - 2358.547854438239) / 2677.999465037951 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.IterateForEach<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Queue - Duration of single invocation | 3.73 μs | 3.35 μs | 0.90 | 0.07 | False | |||||
| ImmutableQueue - Duration of single invocation | 4.39 μs | 4.06 μs | 0.92 | 0.09 | False | |||||
| ImmutableSortedDictionary - Duration of single invocation | 28.32 μs | 26.71 μs | 0.94 | 0.26 | False | |||||
| HashSet - Duration of single invocation | 2.38 μs | 2.12 μs | 0.89 | 0.17 | False | |||||
| Stack - Duration of single invocation | 3.46 μs | 3.09 μs | 0.89 | 0.15 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach<String>*'Payloads
Histogram
System.Collections.IterateForEach<String>.Queue(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.3464309587471357 < 3.487004715826534.
IsChangePoint: Marked as a change because one of 12/15/2022 5:36:17 AM, 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 52.536794260323305 (T) = (0 -3344.5000286717977) / Math.Sqrt((4242.7892645397205 / (299)) + (798.9386441212308 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.09311463298756033 = (3687.897225301598 - 3344.5000286717977) / 3687.897225301598 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.IterateForEach<String>.ImmutableQueue(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 4.05745532239232 < 4.165093952131425.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 43.33246735206256 (T) = (0 -4098.8759115228995) / Math.Sqrt((14895.95417610393 / (299)) + (325.6865957709928 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.07653063157734966 = (4438.5618534636005 - 4098.8759115228995) / 4438.5618534636005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.IterateForEach<String>.ImmutableSortedDictionary(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 26.714099002849 < 26.722841387040575.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 12.460829210533703 (T) = (0 -26602.243309487887) / Math.Sqrt((558873.8501168856 / (299)) + (501182.75852996303 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.06178969375312089 = (28354.243320886966 - 26602.243309487887) / 28354.243320886966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.IterateForEach<String>.HashSet(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.1241638771234683 < 2.263402976422254.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 25.180072127913828 (T) = (0 -2142.394986562177) / Math.Sqrt((6076.856402821578 / (299)) + (2434.658499123669 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1085223567198575 = (2403.1954168579637 - 2142.394986562177) / 2403.1954168579637 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.IterateForEach<String>.Stack(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 3.085146370691623 < 3.317581325711207.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 23.246966468249944 (T) = (0 -3137.8183249030926) / Math.Sqrt((9566.526865754395 / (299)) + (6738.542581381379 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.1089996135136402 = (3521.6800940760636 - 3137.8183249030926) / 3521.6800940760636 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | d6b89c66d492c1e0b0a88d797264039c50c4f4b7 |
| Compare | ccefbf9eb36cb3d8adb36f4b141914c2c6cc489b |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Collections.ContainsFalse<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| List - Duration of single invocation | 1.07 ms | 989.76 μs | 0.92 | 0.17 | False | |||||
| ICollection - Duration of single invocation | 1.07 ms | 989.01 μs | 0.92 | 0.14 | False | |||||
| Queue - Duration of single invocation | 1.08 ms | 997.68 μs | 0.92 | 0.15 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<String>*'Payloads
Histogram
System.Collections.ContainsFalse<String>.List(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 989.7601283482143 < 1.0178709762896825.
IsChangePoint: Marked as a change because one of 2/19/2023 7:20:18 PM, 2/20/2023 2:00:07 PM, 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 4.378754976663198 (T) = (0 -1020121.0459378252) / Math.Sqrt((3253486211.3368974 / (299)) + (6956384997.313642 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.06465355534884419 = (1090634.4400745402 - 1020121.0459378252) / 1090634.4400745402 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsFalse<String>.ICollection(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 989.011875 < 1.0173969164663463.
IsChangePoint: Marked as a change because one of 12/18/2022 12:43:31 PM, 12/19/2022 8:13:55 PM, 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 7.189846335184794 (T) = (0 -1004523.8013857885) / Math.Sqrt((2509011497.5058722 / (299)) + (3521370139.2737556 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.0765549305465081 = (1087800.27596041 - 1004523.8013857885) / 1087800.27596041 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsFalse<String>.Queue(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 997.6800260416667 < 1.0261720377938035.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 3/23/2023 3:52:33 AM falls between 3/14/2023 3:18:57 PM and 3/23/2023 3:52:33 AM.
IsImprovementStdDev: Marked as improvement because 5.19524723666028 (T) = (0 -1019985.6534517893) / Math.Sqrt((1945729485.7873638 / (299)) + (5088664325.540135 / (28))) is greater than 1.967290077109849 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (28) - 2, .975) and 0.06531876225607007 = (1091265.7837379526 - 1019985.6534517893) / 1091265.7837379526 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository