Add a gatekeeper job for PR #8771
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 65 metrics, 6 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.009 s) : 0, 1008513
Total [baseline] (8.649 s) : 0, 8648950
Agent [candidate] (1.01 s) : 0, 1010093
Total [candidate] (8.635 s) : 0, 8634936
section iast
Agent [baseline] (1.137 s) : 0, 1136950
Total [baseline] (9.206 s) : 0, 9206160
Agent [candidate] (1.146 s) : 0, 1146297
Total [candidate] (9.227 s) : 0, 9226580
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.138 s) : 0, 1138292
Total [baseline] (9.213 s) : 0, 9212631
Agent [candidate] (1.145 s) : 0, 1144658
Total [candidate] (9.211 s) : 0, 9211237
section iast_TELEMETRY_OFF
Agent [baseline] (1.142 s) : 0, 1141655
Total [baseline] (9.209 s) : 0, 9209167
Agent [candidate] (1.133 s) : 0, 1133421
Total [candidate] (9.182 s) : 0, 9182088
gantt
title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (670.588 ms) : 0, 670588
BytebuddyAgent [candidate] (674.001 ms) : 0, 674001
GlobalTracer [baseline] (240.362 ms) : 0, 240362
GlobalTracer [candidate] (240.831 ms) : 0, 240831
AppSec [baseline] (54.588 ms) : 0, 54588
AppSec [candidate] (54.923 ms) : 0, 54923
Debugger [baseline] (7.645 ms) : 0, 7645
Debugger [candidate] (6.161 ms) : 0, 6161
Remote Config [baseline] (694.114 µs) : 0, 694
Remote Config [candidate] (705.119 µs) : 0, 705
Telemetry [baseline] (11.278 ms) : 0, 11278
Telemetry [candidate] (9.945 ms) : 0, 9945
section iast
BytebuddyAgent [baseline] (789.515 ms) : 0, 789515
BytebuddyAgent [candidate] (797.117 ms) : 0, 797117
GlobalTracer [baseline] (230.185 ms) : 0, 230185
GlobalTracer [candidate] (231.455 ms) : 0, 231455
IAST [baseline] (22.757 ms) : 0, 22757
IAST [candidate] (23.07 ms) : 0, 23070
AppSec [baseline] (56.578 ms) : 0, 56578
AppSec [candidate] (56.648 ms) : 0, 56648
Debugger [baseline] (5.926 ms) : 0, 5926
Debugger [candidate] (5.92 ms) : 0, 5920
Remote Config [baseline] (595.079 µs) : 0, 595
Remote Config [candidate] (585.843 µs) : 0, 586
Telemetry [baseline] (7.945 ms) : 0, 7945
Telemetry [candidate] (7.909 ms) : 0, 7909
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (790.733 ms) : 0, 790733
BytebuddyAgent [candidate] (795.769 ms) : 0, 795769
GlobalTracer [baseline] (230.481 ms) : 0, 230481
GlobalTracer [candidate] (231.354 ms) : 0, 231354
IAST [baseline] (22.776 ms) : 0, 22776
IAST [candidate] (23.076 ms) : 0, 23076
AppSec [baseline] (56.461 ms) : 0, 56461
AppSec [candidate] (56.47 ms) : 0, 56470
Debugger [baseline] (5.921 ms) : 0, 5921
Debugger [candidate] (5.92 ms) : 0, 5920
Remote Config [baseline] (610.643 µs) : 0, 611
Remote Config [candidate] (597.667 µs) : 0, 598
Telemetry [baseline] (7.888 ms) : 0, 7888
Telemetry [candidate] (7.895 ms) : 0, 7895
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (792.349 ms) : 0, 792349
BytebuddyAgent [candidate] (786.658 ms) : 0, 786658
GlobalTracer [baseline] (232.124 ms) : 0, 232124
GlobalTracer [candidate] (230.307 ms) : 0, 230307
IAST [baseline] (22.664 ms) : 0, 22664
IAST [candidate] (22.403 ms) : 0, 22403
AppSec [baseline] (56.505 ms) : 0, 56505
AppSec [candidate] (56.287 ms) : 0, 56287
Debugger [baseline] (5.981 ms) : 0, 5981
Debugger [candidate] (5.976 ms) : 0, 5976
Remote Config [baseline] (612.278 µs) : 0, 612
Remote Config [candidate] (600.254 µs) : 0, 600
Telemetry [baseline] (7.835 ms) : 0, 7835
Telemetry [candidate] (7.754 ms) : 0, 7754
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.009 s) : 0, 1008848
Total [baseline] (10.472 s) : 0, 10471920
Agent [candidate] (1.011 s) : 0, 1011101
Total [candidate] (10.533 s) : 0, 10532736
section appsec
Agent [baseline] (1.15 s) : 0, 1149539
Total [baseline] (10.709 s) : 0, 10708875
Agent [candidate] (1.153 s) : 0, 1153053
Total [candidate] (10.674 s) : 0, 10673959
section iast
Agent [baseline] (1.147 s) : 0, 1147231
Total [baseline] (10.941 s) : 0, 10941335
Agent [candidate] (1.139 s) : 0, 1139150
Total [candidate] (10.864 s) : 0, 10864312
section profiling
Agent [baseline] (1.261 s) : 0, 1261464
Total [baseline] (10.828 s) : 0, 10827875
Agent [candidate] (1.258 s) : 0, 1258026
Total [candidate] (10.801 s) : 0, 10800587
gantt
title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (671.977 ms) : 0, 671977
BytebuddyAgent [candidate] (674.329 ms) : 0, 674329
GlobalTracer [baseline] (240.403 ms) : 0, 240403
GlobalTracer [candidate] (241.209 ms) : 0, 241209
AppSec [baseline] (54.757 ms) : 0, 54757
AppSec [candidate] (55.062 ms) : 0, 55062
Debugger [baseline] (6.836 ms) : 0, 6836
Debugger [candidate] (6.195 ms) : 0, 6195
Remote Config [baseline] (693.513 µs) : 0, 694
Remote Config [candidate] (703.326 µs) : 0, 703
Telemetry [baseline] (10.677 ms) : 0, 10677
Telemetry [candidate] (10.053 ms) : 0, 10053
section appsec
BytebuddyAgent [baseline] (688.918 ms) : 0, 688918
BytebuddyAgent [candidate] (691.286 ms) : 0, 691286
GlobalTracer [baseline] (236.55 ms) : 0, 236550
GlobalTracer [candidate] (236.798 ms) : 0, 236798
IAST [baseline] (21.613 ms) : 0, 21613
IAST [candidate] (21.891 ms) : 0, 21891
AppSec [baseline] (175.9 ms) : 0, 175900
AppSec [candidate] (175.411 ms) : 0, 175411
Debugger [baseline] (5.874 ms) : 0, 5874
Debugger [candidate] (5.934 ms) : 0, 5934
Remote Config [baseline] (632.146 µs) : 0, 632
Remote Config [candidate] (648.135 µs) : 0, 648
Telemetry [baseline] (7.436 ms) : 0, 7436
Telemetry [candidate] (8.537 ms) : 0, 8537
section iast
BytebuddyAgent [baseline] (797.244 ms) : 0, 797244
BytebuddyAgent [candidate] (791.227 ms) : 0, 791227
GlobalTracer [baseline] (232.381 ms) : 0, 232381
GlobalTracer [candidate] (230.511 ms) : 0, 230511
IAST [baseline] (22.988 ms) : 0, 22988
IAST [candidate] (22.955 ms) : 0, 22955
AppSec [baseline] (56.626 ms) : 0, 56626
AppSec [candidate] (56.441 ms) : 0, 56441
Debugger [baseline] (5.928 ms) : 0, 5928
Debugger [candidate] (5.947 ms) : 0, 5947
Remote Config [baseline] (590.792 µs) : 0, 591
Remote Config [candidate] (605.926 µs) : 0, 606
Telemetry [baseline] (7.916 ms) : 0, 7916
Telemetry [candidate] (7.952 ms) : 0, 7952
section profiling
BytebuddyAgent [baseline] (664.284 ms) : 0, 664284
BytebuddyAgent [candidate] (662.639 ms) : 0, 662639
GlobalTracer [baseline] (381.102 ms) : 0, 381102
GlobalTracer [candidate] (379.098 ms) : 0, 379098
AppSec [baseline] (54.553 ms) : 0, 54553
AppSec [candidate] (54.087 ms) : 0, 54087
Debugger [baseline] (6.152 ms) : 0, 6152
Debugger [candidate] (6.152 ms) : 0, 6152
Remote Config [baseline] (668.417 µs) : 0, 668
Remote Config [candidate] (661.297 µs) : 0, 661
Telemetry [baseline] (8.238 ms) : 0, 8238
Telemetry [candidate] (8.179 ms) : 0, 8179
ProfilingAgent [baseline] (96.128 ms) : 0, 96128
ProfilingAgent [candidate] (97.053 ms) : 0, 97053
Profiling [baseline] (96.152 ms) : 0, 96152
Profiling [candidate] (97.076 ms) : 0, 97076
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics. Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section baseline
no_agent (1.359 ms) : 1339, 1378
. : milestone, 1359,
appsec (1.738 ms) : 1714, 1762
. : milestone, 1738,
appsec_no_iast (1.726 ms) : 1703, 1749
. : milestone, 1726,
code_origins (1.672 ms) : 1646, 1699
. : milestone, 1672,
iast (1.511 ms) : 1488, 1535
. : milestone, 1511,
profiling (1.502 ms) : 1479, 1525
. : milestone, 1502,
tracing (1.507 ms) : 1481, 1532
. : milestone, 1507,
section candidate
no_agent (1.348 ms) : 1328, 1367
. : milestone, 1348,
appsec (1.735 ms) : 1711, 1758
. : milestone, 1735,
appsec_no_iast (1.717 ms) : 1693, 1740
. : milestone, 1717,
code_origins (1.664 ms) : 1637, 1691
. : milestone, 1664,
iast (1.531 ms) : 1508, 1555
. : milestone, 1531,
profiling (1.482 ms) : 1459, 1505
. : milestone, 1482,
tracing (1.479 ms) : 1455, 1504
. : milestone, 1479,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section baseline
no_agent (381.844 µs) : 362, 402
. : milestone, 382,
iast (520.101 µs) : 497, 543
. : milestone, 520,
iast_FULL (729.863 µs) : 706, 753
. : milestone, 730,
iast_GLOBAL (557.736 µs) : 534, 581
. : milestone, 558,
iast_HARDCODED_SECRET_DISABLED (513.525 µs) : 491, 537
. : milestone, 514,
iast_INACTIVE (459.251 µs) : 437, 481
. : milestone, 459,
iast_TELEMETRY_OFF (502.823 µs) : 480, 526
. : milestone, 503,
tracing (459.319 µs) : 437, 481
. : milestone, 459,
section candidate
no_agent (376.032 µs) : 356, 396
. : milestone, 376,
iast (520.729 µs) : 498, 543
. : milestone, 521,
iast_FULL (728.278 µs) : 705, 752
. : milestone, 728,
iast_GLOBAL (575.529 µs) : 551, 600
. : milestone, 576,
iast_HARDCODED_SECRET_DISABLED (525.277 µs) : 503, 548
. : milestone, 525,
iast_INACTIVE (464.945 µs) : 442, 488
. : milestone, 465,
iast_TELEMETRY_OFF (500.544 µs) : 478, 523
. : milestone, 501,
tracing (460.736 µs) : 439, 483
. : milestone, 461,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section baseline
no_agent (14.976 s) : 14976000, 14976000
. : milestone, 14976000,
appsec (15.256 s) : 15256000, 15256000
. : milestone, 15256000,
iast (18.731 s) : 18731000, 18731000
. : milestone, 18731000,
iast_GLOBAL (17.968 s) : 17968000, 17968000
. : milestone, 17968000,
profiling (15.62 s) : 15620000, 15620000
. : milestone, 15620000,
tracing (14.817 s) : 14817000, 14817000
. : milestone, 14817000,
section candidate
no_agent (14.94 s) : 14940000, 14940000
. : milestone, 14940000,
appsec (14.938 s) : 14938000, 14938000
. : milestone, 14938000,
iast (18.822 s) : 18822000, 18822000
. : milestone, 18822000,
iast_GLOBAL (18.158 s) : 18158000, 18158000
. : milestone, 18158000,
profiling (15.021 s) : 15021000, 15021000
. : milestone, 15021000,
tracing (14.911 s) : 14911000, 14911000
. : milestone, 14911000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~d5c12752b6, baseline=1.49.0-SNAPSHOT~5bfab6549b
dateFormat X
axisFormat %s
section baseline
no_agent (1.476 ms) : 1465, 1488
. : milestone, 1476,
appsec (2.388 ms) : 2341, 2436
. : milestone, 2388,
iast (2.173 ms) : 2113, 2232
. : milestone, 2173,
iast_GLOBAL (2.211 ms) : 2151, 2271
. : milestone, 2211,
profiling (2.047 ms) : 1998, 2097
. : milestone, 2047,
tracing (2.013 ms) : 1966, 2059
. : milestone, 2013,
section candidate
no_agent (1.477 ms) : 1466, 1489
. : milestone, 1477,
appsec (2.391 ms) : 2344, 2439
. : milestone, 2391,
iast (2.164 ms) : 2104, 2224
. : milestone, 2164,
iast_GLOBAL (2.213 ms) : 2153, 2274
. : milestone, 2213,
profiling (2.017 ms) : 1969, 2065
. : milestone, 2017,
tracing (1.993 ms) : 1946, 2039
. : milestone, 1993,
|
There was a problem hiding this comment.
Interesting 🤔 That will solve the issue of having to craft GraphQL query to set branch protection rules.
This jobs will check ALL other job
What about the GitHub workflows? Are they checked too?
Or can we excluded them from the gateway and still requiring them from the GitHub protection rules (if they working fine already)?
And in general how does this work if the checks are not complete when the gatekeeper runs? Is that the retry part? Will it run for 40 mins?
This will be useful for #8475 cc @sarahchen6 @bric3
smola
reviewed
May 6, 2025
Yes they are. By default, if you see something in your PR page, it should be checked.
Yes you can, either by setting their names in
Yes, there is a retry. With the current parameters : So it runs for 1000 + 60*60 = 4600 seconds (1 hour and 10 minutes). Note that it perform an initial check at the very begining, to avoid waiting 1000s for nothing on retries. |
|
Thanks for the detailed answer! That looks handy 🙌
That’s a lot! What about using the status event then? https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#status |
Niiiiiiiice, I was not aware of that option! I need to perform proper verification to ensure it's a good fit. If you don't mind, I'll use system-tests repo to do the guinea pig, and if it's good, I'll be ack to use it here. BTW, yes, it's a lot, but it will mostly stop before that, at your first failing job, or worst case scenario, your overall CI time. |
PerfectSlayer
approved these changes
May 7, 2025
There was a problem hiding this comment.
Sure, we can merge it as is, and iterate later if the new trigger work.
One thing I wonder is about the concurrency parameter. You might need to set it up in order to avoid having dozen of parallel runs for the same PR (which can trigger race condition if the last call -- with all the checks completed -- ends up before any of the previous one -- with not all the checks completed.
I will approve but I found can update the README file in the same folder about this automation / workflow, it could be nice (trigger, action, recovery) 👍
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Add a job in the CI that runs in PR. The job will be a success if all other jobs are skipped/success, and fails otherwise
Motivation
While it's possible to enforce a green CI policy using GitHub's native "required status checks" feature, doing so requires explicitly listing all job names under branch protection rules. This approach has two key drawbacks:
This jobs will check ALL other job, and we'll be able to set this as a requirement. The action used offers a
ignored-name-patternsparameters`, I added few job that failed more than 10% over the last 60 days on merges.The plan is to merge this PR, wait few days to be sure that everything is fine. Then add it as a requirement.
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any usefull labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]