Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Skip tests based on git changes on GitLab #9039

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Jun 30, 2025
Merged

Skip tests based on git changes on GitLab #9039

merged 7 commits into from
Jun 30, 2025

Conversation

smola
Copy link
Member

@smola smola commented Jun 26, 2025 •
edited
Loading

What Does This Do

Bringing back #7464 to GitLab. Tests are skipped based on Git changes. See the original PR for guidance.

Motivation

Bring down pipeline flakes, pipeline mean time, costs.

Additional Notes

Test PR: #9059

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@smola smola added tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Jun 26, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jun 26, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/git-pr-skip
git_commit_date 1751046924 1751267928
git_commit_sha 46d0949 bf4cb57
release_version 1.51.0-SNAPSHOT~46d09492c9 1.51.0-SNAPSHOT~bf4cb576d0
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1751269751 1751269751
ci_job_id 1004190726 1004190726
ci_pipeline_id 69107919 69107919
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-tcrivaq7-project-304-concurrent-0-8lmwu9lr 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-tcrivaq7-project-304-concurrent-0-8lmwu9lr 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 9 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.51.0-SNAPSHOT~bf4cb576d0, baseline=1.51.0-SNAPSHOT~46d09492c9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (995.434 ms) : 0, 995434
Total [baseline] (10.653 s) : 0, 10652814
Agent [candidate] (996.384 ms) : 0, 996384
Total [candidate] (10.664 s) : 0, 10663620
section appsec
Agent [baseline] (1.181 s) : 0, 1181263
Total [baseline] (10.744 s) : 0, 10744258
Agent [candidate] (1.185 s) : 0, 1184613
Total [candidate] (10.732 s) : 0, 10731770
section iast
Agent [baseline] (1.139 s) : 0, 1138933
Total [baseline] (10.806 s) : 0, 10805610
Agent [candidate] (1.136 s) : 0, 1136320
Total [candidate] (10.804 s) : 0, 10803625
section profiling
Agent [baseline] (1.254 s) : 0, 1253998
Total [baseline] (10.915 s) : 0, 10915395
Agent [candidate] (1.261 s) : 0, 1261044
Total [candidate] (11.088 s) : 0, 11087520
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 995.434 ms -
Agent appsec 1.181 s 185.829 ms (18.7%)
Agent iast 1.139 s 143.498 ms (14.4%)
Agent profiling 1.254 s 258.564 ms (26.0%)
Total tracing 10.653 s -
Total appsec 10.744 s 91.444 ms (0.9%)
Total iast 10.806 s 152.797 ms (1.4%)
Total profiling 10.915 s 262.582 ms (2.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 996.384 ms -
Agent appsec 1.185 s 188.229 ms (18.9%)
Agent iast 1.136 s 139.936 ms (14.0%)
Agent profiling 1.261 s 264.66 ms (26.6%)
Total tracing 10.664 s -
Total appsec 10.732 s 68.149 ms (0.6%)
Total iast 10.804 s 140.004 ms (1.3%)
Total profiling 11.088 s 423.899 ms (4.0%) 
gantt
    title petclinic - break down per module: candidate=1.51.0-SNAPSHOT~bf4cb576d0, baseline=1.51.0-SNAPSHOT~46d09492c9

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (686.82 ms) : 0, 686820
BytebuddyAgent [candidate] (687.69 ms) : 0, 687690
GlobalTracer [baseline] (242.488 ms) : 0, 242488
GlobalTracer [candidate] (242.515 ms) : 0, 242515
AppSec [baseline] (30.347 ms) : 0, 30347
AppSec [candidate] (30.511 ms) : 0, 30511
Debugger [baseline] (6.027 ms) : 0, 6027
Debugger [candidate] (6.025 ms) : 0, 6025
Remote Config [baseline] (675.322 µs) : 0, 675
Remote Config [candidate] (667.122 µs) : 0, 667
Telemetry [baseline] (8.239 ms) : 0, 8239
Telemetry [candidate] (8.184 ms) : 0, 8184
section appsec
BytebuddyAgent [baseline] (715.123 ms) : 0, 715123
BytebuddyAgent [candidate] (716.61 ms) : 0, 716610
GlobalTracer [baseline] (236.899 ms) : 0, 236899
GlobalTracer [candidate] (237.723 ms) : 0, 237723
IAST [baseline] (22.15 ms) : 0, 22150
IAST [candidate] (22.444 ms) : 0, 22444
AppSec [baseline] (171.615 ms) : 0, 171615
AppSec [candidate] (172.206 ms) : 0, 172206
Debugger [baseline] (5.831 ms) : 0, 5831
Debugger [candidate] (5.852 ms) : 0, 5852
Remote Config [baseline] (622.383 µs) : 0, 622
Remote Config [candidate] (628.251 µs) : 0, 628
Telemetry [baseline] (8.166 ms) : 0, 8166
Telemetry [candidate] (8.19 ms) : 0, 8190
section iast
BytebuddyAgent [baseline] (813.785 ms) : 0, 813785
BytebuddyAgent [candidate] (812.479 ms) : 0, 812479
GlobalTracer [baseline] (234.316 ms) : 0, 234316
GlobalTracer [candidate] (234.355 ms) : 0, 234355
IAST [baseline] (28.828 ms) : 0, 28828
IAST [candidate] (28.474 ms) : 0, 28474
AppSec [baseline] (26.808 ms) : 0, 26808
AppSec [candidate] (25.109 ms) : 0, 25109
Debugger [baseline] (5.841 ms) : 0, 5841
Debugger [candidate] (5.772 ms) : 0, 5772
Remote Config [baseline] (591.696 µs) : 0, 592
Remote Config [candidate] (585.682 µs) : 0, 586
Telemetry [baseline] (7.9 ms) : 0, 7900
Telemetry [candidate] (7.904 ms) : 0, 7904
section profiling
BytebuddyAgent [baseline] (684.288 ms) : 0, 684288
BytebuddyAgent [candidate] (686.242 ms) : 0, 686242
GlobalTracer [baseline] (362.812 ms) : 0, 362812
GlobalTracer [candidate] (366.181 ms) : 0, 366181
AppSec [baseline] (31.917 ms) : 0, 31917
AppSec [candidate] (32.19 ms) : 0, 32190
Debugger [baseline] (11.404 ms) : 0, 11404
Debugger [candidate] (9.928 ms) : 0, 9928
Remote Config [baseline] (1.4 ms) : 0, 1400
Remote Config [candidate] (682.299 µs) : 0, 682
Telemetry [baseline] (8.808 ms) : 0, 8808
Telemetry [candidate] (10.391 ms) : 0, 10391
ProfilingAgent [baseline] (104.348 ms) : 0, 104348
ProfilingAgent [candidate] (105.703 ms) : 0, 105703
Profiling [baseline] (104.372 ms) : 0, 104372
Profiling [candidate] (105.727 ms) : 0, 105727
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.51.0-SNAPSHOT~bf4cb576d0, baseline=1.51.0-SNAPSHOT~46d09492c9

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (995.151 ms) : 0, 995151
Total [baseline] (8.575 s) : 0, 8575466
Agent [candidate] (994.828 ms) : 0, 994828
Total [candidate] (8.552 s) : 0, 8551869
section iast
Agent [baseline] (1.129 s) : 0, 1129415
Total [baseline] (9.265 s) : 0, 9264645
Agent [candidate] (1.143 s) : 0, 1143325
Total [candidate] (9.354 s) : 0, 9354172
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 995.151 ms -
Agent iast 1.129 s 134.264 ms (13.5%)
Total tracing 8.575 s -
Total iast 9.265 s 689.179 ms (8.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 994.828 ms -
Agent iast 1.143 s 148.497 ms (14.9%)
Total tracing 8.552 s -
Total iast 9.354 s 802.303 ms (9.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.51.0-SNAPSHOT~bf4cb576d0, baseline=1.51.0-SNAPSHOT~46d09492c9

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (686.748 ms) : 0, 686748
BytebuddyAgent [candidate] (686.318 ms) : 0, 686318
GlobalTracer [baseline] (242.379 ms) : 0, 242379
GlobalTracer [candidate] (242.34 ms) : 0, 242340
AppSec [baseline] (30.281 ms) : 0, 30281
AppSec [candidate] (30.408 ms) : 0, 30408
Debugger [baseline] (6.016 ms) : 0, 6016
Debugger [candidate] (6.058 ms) : 0, 6058
Remote Config [baseline] (676.142 µs) : 0, 676
Remote Config [candidate] (663.201 µs) : 0, 663
Telemetry [baseline] (8.196 ms) : 0, 8196
Telemetry [candidate] (8.205 ms) : 0, 8205
section iast
BytebuddyAgent [baseline] (806.448 ms) : 0, 806448
BytebuddyAgent [candidate] (816.325 ms) : 0, 816325
GlobalTracer [baseline] (232.193 ms) : 0, 232193
GlobalTracer [candidate] (235.257 ms) : 0, 235257
AppSec [baseline] (30.346 ms) : 0, 30346
AppSec [candidate] (28.847 ms) : 0, 28847
Debugger [baseline] (6.634 ms) : 0, 6634
Debugger [candidate] (5.928 ms) : 0, 5928
Remote Config [baseline] (582.549 µs) : 0, 583
Remote Config [candidate] (595.381 µs) : 0, 595
Telemetry [baseline] (7.949 ms) : 0, 7949
Telemetry [candidate] (8.121 ms) : 0, 8121
IAST [baseline] (24.431 ms) : 0, 24431
IAST [candidate] (27.309 ms) : 0, 27309
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master smola/git-pr-skip
git_commit_date 1751046924 1751267928
git_commit_sha 46d0949 bf4cb57
release_version 1.51.0-SNAPSHOT~46d09492c9 1.51.0-SNAPSHOT~bf4cb576d0
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1751269426 1751269426
ci_job_id 1004190727 1004190727
ci_pipeline_id 69107919 69107919
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-iscs1df1-project-304-concurrent-0-vzkiwxca 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-iscs1df1-project-304-concurrent-0-vzkiwxca 6.8.0-1029-aws #31~22.04.1-Ubuntu SMP Thu Apr 24 21:16:18 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 3 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:iast_GLOBAL:high_load better
[-687.663µs; -301.090µs] or [-6.726%; -2.945%]		 unstable
[-31.739op/s; +77.677op/s] or [-6.985%; +17.094%]		 9.729ms 477.375op/s 10.223ms 454.406op/s
scenario:load:insecure-bank:iast:high_load worse
[+366.136µs; +705.468µs] or [+3.963%; +7.636%]		 unstable
[-84.405op/s; +29.842op/s] or [-16.803%; +5.941%]		 9.775ms 475.031op/s 9.239ms 502.312op/s
scenario:load:petclinic:no_agent:high_load worse
[+1.160ms; +1.821ms] or [+3.162%; +4.961%]		 unstable
[-13.596op/s; +3.746op/s] or [-10.674%; +2.941%]		 38.184ms 122.450op/s 36.694ms 127.375op/s
scenario:load:petclinic:code_origins:high_load worse
[+1.597ms; +2.413ms] or [+3.730%; +5.635%]		 unstable
[-12.126op/s; +2.376op/s] or [-11.103%; +2.176%]		 44.829ms 104.338op/s 42.824ms 109.213op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~bf4cb576d0, baseline=1.51.0-SNAPSHOT~46d09492c9
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.266 ms) : 4217, 4314
.   : milestone, 4266,
iast (9.239 ms) : 9089, 9389
.   : milestone, 9239,
iast_FULL (13.666 ms) : 13398, 13934
.   : milestone, 13666,
iast_GLOBAL (10.223 ms) : 10036, 10411
.   : milestone, 10223,
profiling (8.687 ms) : 8554, 8819
.   : milestone, 8687,
tracing (7.423 ms) : 7311, 7535
.   : milestone, 7423,
section candidate
no_agent (4.337 ms) : 4288, 4385
.   : milestone, 4337,
iast (9.775 ms) : 9610, 9939
.   : milestone, 9775,
iast_FULL (14.218 ms) : 13933, 14503
.   : milestone, 14218,
iast_GLOBAL (9.729 ms) : 9558, 9900
.   : milestone, 9729,
profiling (8.611 ms) : 8479, 8744
.   : milestone, 8611,
tracing (7.648 ms) : 7532, 7764
.   : milestone, 7648,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.266 ms [4.217 ms, 4.314 ms] -
iast 9.239 ms [9.089 ms, 9.389 ms] 4.973 ms (116.6%)
iast_FULL 13.666 ms [13.398 ms, 13.934 ms] 9.4 ms (220.4%)
iast_GLOBAL 10.223 ms [10.036 ms, 10.411 ms] 5.958 ms (139.7%)
profiling 8.687 ms [8.554 ms, 8.819 ms] 4.421 ms (103.7%)
tracing 7.423 ms [7.311 ms, 7.535 ms] 3.158 ms (74.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.337 ms [4.288 ms, 4.385 ms] -
iast 9.775 ms [9.61 ms, 9.939 ms] 5.438 ms (125.4%)
iast_FULL 14.218 ms [13.933 ms, 14.503 ms] 9.881 ms (227.9%)
iast_GLOBAL 9.729 ms [9.558 ms, 9.9 ms] 5.393 ms (124.3%)
profiling 8.611 ms [8.479 ms, 8.744 ms] 4.275 ms (98.6%)
tracing 7.648 ms [7.532 ms, 7.764 ms] 3.311 ms (76.4%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.0-SNAPSHOT~bf4cb576d0, baseline=1.51.0-SNAPSHOT~46d09492c9
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.694 ms) : 36390, 36997
.   : milestone, 36694,
appsec (46.377 ms) : 45977, 46777
.   : milestone, 46377,
code_origins (42.824 ms) : 42466, 43181
.   : milestone, 42824,
iast (46.031 ms) : 45615, 46447
.   : milestone, 46031,
profiling (48.354 ms) : 47931, 48777
.   : milestone, 48354,
tracing (41.964 ms) : 41611, 42316
.   : milestone, 41964,
section candidate
no_agent (38.184 ms) : 37874, 38494
.   : milestone, 38184,
appsec (46.61 ms) : 46204, 47016
.   : milestone, 46610,
code_origins (44.829 ms) : 44430, 45228
.   : milestone, 44829,
iast (46.298 ms) : 45882, 46713
.   : milestone, 46298,
profiling (47.077 ms) : 46643, 47510
.   : milestone, 47077,
tracing (41.938 ms) : 41586, 42289
.   : milestone, 41938,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.694 ms [36.39 ms, 36.997 ms] -
appsec 46.377 ms [45.977 ms, 46.777 ms] 9.683 ms (26.4%)
code_origins 42.824 ms [42.466 ms, 43.181 ms] 6.13 ms (16.7%)
iast 46.031 ms [45.615 ms, 46.447 ms] 9.338 ms (25.4%)
profiling 48.354 ms [47.931 ms, 48.777 ms] 11.66 ms (31.8%)
tracing 41.964 ms [41.611 ms, 42.316 ms] 5.27 ms (14.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 38.184 ms [37.874 ms, 38.494 ms] -
appsec 46.61 ms [46.204 ms, 47.016 ms] 8.426 ms (22.1%)
code_origins 44.829 ms [44.43 ms, 45.228 ms] 6.645 ms (17.4%)
iast 46.298 ms [45.882 ms, 46.713 ms] 8.114 ms (21.2%)
profiling 47.077 ms [46.643 ms, 47.51 ms] 8.893 ms (23.3%)
tracing 41.938 ms [41.586 ms, 42.289 ms] 3.754 ms (9.8%)

Dacapo

@smola smola force-pushed the smola/git-pr-skip branch from 31211b9 to 04a06f8 Compare June 26, 2025 08:26
@smola smola force-pushed the smola/git-pr-skip branch from 04a06f8 to 283207e Compare June 27, 2025 15:51
@smola smola marked this pull request as ready for review June 30, 2025 06:54
@smola smola requested a review from a team as a code owner June 30, 2025 06:54
@smola smola requested review from TonyCTHsu and removed request for a team June 30, 2025 06:54
bric3
bric3 reviewed Jun 30, 2025
View reviewed changes
@smola smola requested a review from bric3 June 30, 2025 07:19
bric3
bric3 reviewed Jun 30, 2025
View reviewed changes
.gitlab/find-gh-base-ref.sh
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Did you thought about making this part of our gradle itself. Not that I'm asking for it specifically. As I would like to avoid adding yet another groovy scripting at this time.

Also, I wonder does our gitlab CI have this resolved ?

cc @randomanderson

Copy link
Member Author

@smola smola Jun 30, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you thought about making this part of our gradle itself.

Can be done at gradle level. A bash script for CI seemed pretty self-contained and easily debuggable though.

Also, I wonder does our gitlab CI have this resolved ?

Not supported by GitLab, because GitHub PRs are not native to GitLab CI, and we do not use GitLab MR. We have an internal solution at DD that requires migrating to a new CI system for GitLab. That's apparently a lot of work for the repo since it was not used since the beginning.

In our current setup, this is the recommended way. I took it from dd-trace-py.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah thanks for the info !

@smola smola merged commit 7494696 into master Jun 30, 2025
509 of 511 checks passed
@smola smola deleted the smola/git-pr-skip branch June 30, 2025 11:58
@github-actions github-actions bot added this to the 1.51.0 milestone Jun 30, 2025
@PerfectSlayer PerfectSlayer added the type: enhancement Enhancements and improvements label Jul 1, 2025
@smola smola mentioned this pull request Jul 1, 2025
Merged
smola added a commit that referenced this pull request Jul 2, 2025
@smola
Skip tests based on git changes on GitLab (#9039)
6a78f3a
@smola smola mentioned this pull request Jul 2, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bric3 bric3 bric3 approved these changes

@TonyCTHsu TonyCTHsu Awaiting requested review from TonyCTHsu TonyCTHsu is a code owner automatically assigned from DataDog/apm-release-platform

Assignees
No one assigned
Labels
comp: tooling Build & Tooling tag: no release notes Changes to exclude from release notes type: enhancement Enhancements and improvements
Projects
None yet
Milestone
1.51.0
Development

Successfully merging this pull request may close these issues.
3 participants
@smola @bric3 @PerfectSlayer