Update smoke test tasks to use lazy API #9617

sarahchen6 · 2025-09-25T16:05:06Z

What Does This Do

Update smoke test gradle file tasks to use lazy API formatting

Motivation

Build improvements

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any useful labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

datadog-official · 2025-09-25T16:34:18Z

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 57.81% (-0.03%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 6b5709a | Docs | Was this helpful? Give us feedback!}

pr-commenter · 2025-09-25T16:44:57Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/convert-smoke-test-tasks-to-lazy-apis
git_commit_date	1758813010	1758816560
git_commit_sha	`050cad8`	`6b5709a`
release_version	1.54.0-SNAPSHOT~050cad8b0e	1.54.0-SNAPSHOT~6b5709a70e

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1758818510	1758818510
ci_job_id	1148078492	1148078492
ci_pipeline_id	77552694	77552694
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-4qfgaz0t 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-4qfgaz0t 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 6 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.54.0-SNAPSHOT~6b5709a70e, baseline=1.54.0-SNAPSHOT~050cad8b0e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.012 s) : 0, 1011986
Total [baseline] (8.682 s) : 0, 8681954
Agent [candidate] (1.013 s) : 0, 1013091
Total [candidate] (8.658 s) : 0, 8658086
section iast
Agent [baseline] (1.145 s) : 0, 1144569
Total [baseline] (9.239 s) : 0, 9238823
Agent [candidate] (1.142 s) : 0, 1141926
Total [candidate] (9.205 s) : 0, 9204604

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.012 s	-
Agent	iast	1.145 s	132.583 ms (13.1%)
Total	tracing	8.682 s	-
Total	iast	9.239 s	556.869 ms (6.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.013 s	-
Agent	iast	1.142 s	128.835 ms (12.7%)
Total	tracing	8.658 s	-
Total	iast	9.205 s	546.518 ms (6.3%)

gantt
    title insecure-bank - break down per module: candidate=1.54.0-SNAPSHOT~6b5709a70e, baseline=1.54.0-SNAPSHOT~050cad8b0e

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.474 ms) : 0, 1474
crashtracking [candidate] (1.457 ms) : 0, 1457
BytebuddyAgent [baseline] (692.387 ms) : 0, 692387
BytebuddyAgent [candidate] (693.701 ms) : 0, 693701
GlobalTracer [baseline] (248.854 ms) : 0, 248854
GlobalTracer [candidate] (249.128 ms) : 0, 249128
AppSec [baseline] (31.648 ms) : 0, 31648
AppSec [candidate] (31.354 ms) : 0, 31354
Debugger [baseline] (6.532 ms) : 0, 6532
Debugger [candidate] (6.453 ms) : 0, 6453
Remote Config [baseline] (686.644 µs) : 0, 687
Remote Config [candidate] (688.036 µs) : 0, 688
Telemetry [baseline] (9.15 ms) : 0, 9150
Telemetry [candidate] (9.127 ms) : 0, 9127
section iast
crashtracking [baseline] (1.456 ms) : 0, 1456
crashtracking [candidate] (1.461 ms) : 0, 1461
BytebuddyAgent [baseline] (809.572 ms) : 0, 809572
BytebuddyAgent [candidate] (807.956 ms) : 0, 807956
GlobalTracer [baseline] (237.022 ms) : 0, 237022
GlobalTracer [candidate] (236.844 ms) : 0, 236844
IAST [baseline] (26.241 ms) : 0, 26241
IAST [candidate] (26.07 ms) : 0, 26070
AppSec [baseline] (34.228 ms) : 0, 34228
AppSec [candidate] (33.49 ms) : 0, 33490
Debugger [baseline] (6.004 ms) : 0, 6004
Debugger [candidate] (6.029 ms) : 0, 6029
Remote Config [baseline] (585.83 µs) : 0, 586
Remote Config [candidate] (591.383 µs) : 0, 591
Telemetry [baseline] (8.275 ms) : 0, 8275
Telemetry [candidate] (8.281 ms) : 0, 8281

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.54.0-SNAPSHOT~6b5709a70e, baseline=1.54.0-SNAPSHOT~050cad8b0e

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.015 s) : 0, 1015033
Total [baseline] (10.817 s) : 0, 10817264
Agent [candidate] (1.007 s) : 0, 1006565
Total [candidate] (10.681 s) : 0, 10681471
section appsec
Agent [baseline] (1.189 s) : 0, 1188534
Total [baseline] (11.027 s) : 0, 11026586
Agent [candidate] (1.195 s) : 0, 1194527
Total [candidate] (10.982 s) : 0, 10981741
section iast
Agent [baseline] (1.152 s) : 0, 1152440
Total [baseline] (11.017 s) : 0, 11016532
Agent [candidate] (1.143 s) : 0, 1143003
Total [candidate] (10.93 s) : 0, 10929804
section profiling
Agent [baseline] (1.15 s) : 0, 1149505
Total [baseline] (11.041 s) : 0, 11041198
Agent [candidate] (1.161 s) : 0, 1160646
Total [candidate] (11.004 s) : 0, 11003950

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.015 s	-
Agent	appsec	1.189 s	173.501 ms (17.1%)
Agent	iast	1.152 s	137.407 ms (13.5%)
Agent	profiling	1.15 s	134.472 ms (13.2%)
Total	tracing	10.817 s	-
Total	appsec	11.027 s	209.322 ms (1.9%)
Total	iast	11.017 s	199.268 ms (1.8%)
Total	profiling	11.041 s	223.934 ms (2.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.007 s	-
Agent	appsec	1.195 s	187.962 ms (18.7%)
Agent	iast	1.143 s	136.438 ms (13.6%)
Agent	profiling	1.161 s	154.081 ms (15.3%)
Total	tracing	10.681 s	-
Total	appsec	10.982 s	300.27 ms (2.8%)
Total	iast	10.93 s	248.332 ms (2.3%)
Total	profiling	11.004 s	322.478 ms (3.0%)

gantt
    title petclinic - break down per module: candidate=1.54.0-SNAPSHOT~6b5709a70e, baseline=1.54.0-SNAPSHOT~050cad8b0e

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.46 ms) : 0, 1460
crashtracking [candidate] (1.461 ms) : 0, 1461
BytebuddyAgent [baseline] (694.281 ms) : 0, 694281
BytebuddyAgent [candidate] (689.192 ms) : 0, 689192
GlobalTracer [baseline] (249.836 ms) : 0, 249836
GlobalTracer [candidate] (247.708 ms) : 0, 247708
AppSec [baseline] (31.879 ms) : 0, 31879
AppSec [candidate] (31.143 ms) : 0, 31143
Debugger [baseline] (6.54 ms) : 0, 6540
Debugger [candidate] (6.345 ms) : 0, 6345
Remote Config [baseline] (699.05 µs) : 0, 699
Remote Config [candidate] (665.095 µs) : 0, 665
Telemetry [baseline] (9.139 ms) : 0, 9139
Telemetry [candidate] (8.98 ms) : 0, 8980
section appsec
crashtracking [baseline] (1.453 ms) : 0, 1453
crashtracking [candidate] (1.489 ms) : 0, 1489
BytebuddyAgent [baseline] (712.648 ms) : 0, 712648
BytebuddyAgent [candidate] (717.162 ms) : 0, 717162
GlobalTracer [baseline] (239.928 ms) : 0, 239928
GlobalTracer [candidate] (240.82 ms) : 0, 240820
IAST [baseline] (24.698 ms) : 0, 24698
IAST [candidate] (24.849 ms) : 0, 24849
AppSec [baseline] (170.587 ms) : 0, 170587
AppSec [candidate] (172.337 ms) : 0, 172337
Debugger [baseline] (6.068 ms) : 0, 6068
Debugger [candidate] (6.049 ms) : 0, 6049
Remote Config [baseline] (644.375 µs) : 0, 644
Remote Config [candidate] (641.706 µs) : 0, 642
Telemetry [baseline] (11.471 ms) : 0, 11471
Telemetry [candidate] (10.01 ms) : 0, 10010
section iast
crashtracking [baseline] (1.477 ms) : 0, 1477
crashtracking [candidate] (1.457 ms) : 0, 1457
BytebuddyAgent [baseline] (815.181 ms) : 0, 815181
BytebuddyAgent [candidate] (808.925 ms) : 0, 808925
GlobalTracer [baseline] (238.868 ms) : 0, 238868
GlobalTracer [candidate] (237.126 ms) : 0, 237126
IAST [baseline] (26.281 ms) : 0, 26281
IAST [candidate] (26.948 ms) : 0, 26948
AppSec [baseline] (34.122 ms) : 0, 34122
AppSec [candidate] (32.439 ms) : 0, 32439
Debugger [baseline] (6.081 ms) : 0, 6081
Debugger [candidate] (6.018 ms) : 0, 6018
Remote Config [baseline] (599.125 µs) : 0, 599
Remote Config [candidate] (569.907 µs) : 0, 570
Telemetry [baseline] (8.412 ms) : 0, 8412
Telemetry [candidate] (8.255 ms) : 0, 8255
section profiling
crashtracking [baseline] (1.44 ms) : 0, 1440
crashtracking [candidate] (1.451 ms) : 0, 1451
BytebuddyAgent [baseline] (717.46 ms) : 0, 717460
BytebuddyAgent [candidate] (723.778 ms) : 0, 723778
GlobalTracer [baseline] (223.087 ms) : 0, 223087
GlobalTracer [candidate] (225.386 ms) : 0, 225386
AppSec [baseline] (31.176 ms) : 0, 31176
AppSec [candidate] (31.621 ms) : 0, 31621
Debugger [baseline] (6.506 ms) : 0, 6506
Debugger [candidate] (8.098 ms) : 0, 8098
Remote Config [baseline] (701.35 µs) : 0, 701
Remote Config [candidate] (697.477 µs) : 0, 697
Telemetry [baseline] (16.324 ms) : 0, 16324
Telemetry [candidate] (13.986 ms) : 0, 13986
ProfilingAgent [baseline] (100.56 ms) : 0, 100560
ProfilingAgent [candidate] (103.124 ms) : 0, 103124
Profiling [baseline] (101.146 ms) : 0, 101146
Profiling [candidate] (103.725 ms) : 0, 103725

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/convert-smoke-test-tasks-to-lazy-apis
git_commit_date	1758813010	1758816560
git_commit_sha	`050cad8`	`6b5709a`
release_version	1.54.0-SNAPSHOT~050cad8b0e	1.54.0-SNAPSHOT~6b5709a70e

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1758818173	1758818173
ci_job_id	1148078493	1148078493
ci_pipeline_id	77552694	77552694
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-gexjw6sl 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-gexjw6sl 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 5 performance regressions! Performance is the same for 6 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:profiling:high_load	worse [+505.156µs; +823.360µs] or [+5.753%; +9.377%]	unstable [-99.103op/s; +25.790op/s] or [-18.759%; +4.882%]	9.445ms	491.625op/s	8.781ms	528.281op/s
scenario:load:insecure-bank:iast_GLOBAL:high_load	worse [+512.196µs; +920.467µs] or [+4.956%; +8.906%]	unstable [-77.217op/s; +19.280op/s] or [-17.174%; +4.288%]	11.052ms	420.656op/s	10.335ms	449.625op/s
scenario:load:insecure-bank:no_agent:high_load	worse [+189.447µs; +298.547µs] or [+4.407%; +6.945%]	unstable [-162.817op/s; +50.130op/s] or [-15.252%; +4.696%]	4.543ms	1011.188op/s	4.299ms	1067.531op/s
scenario:load:insecure-bank:iast:high_load	better [-849.711µs; -485.444µs] or [-8.308%; -4.746%]	unstable [-23.159op/s; +85.784op/s] or [-5.098%; +18.882%]	9.560ms	485.625op/s	10.228ms	454.312op/s
scenario:load:petclinic:iast:high_load	worse [+2.950ms; +3.763ms] or [+6.833%; +8.718%]	unstable [-14.841op/s; -0.759op/s] or [-13.693%; -0.700%]	46.525ms	100.588op/s	43.168ms	108.388op/s
scenario:load:petclinic:profiling:high_load	worse [+1.367ms; +2.364ms] or [+2.842%; +4.912%]	unstable [-10.274op/s; +3.074op/s] or [-10.561%; +3.160%]	49.988ms	93.688op/s	48.122ms	97.287op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.54.0-SNAPSHOT~6b5709a70e, baseline=1.54.0-SNAPSHOT~050cad8b0e
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.299 ms) : 4249, 4349
.   : milestone, 4299,
iast (10.228 ms) : 10054, 10401
.   : milestone, 10228,
iast_FULL (14.324 ms) : 14042, 14606
.   : milestone, 14324,
iast_GLOBAL (10.335 ms) : 10155, 10516
.   : milestone, 10335,
profiling (8.781 ms) : 8643, 8919
.   : milestone, 8781,
tracing (7.872 ms) : 7759, 7986
.   : milestone, 7872,
section candidate
no_agent (4.543 ms) : 4491, 4595
.   : milestone, 4543,
iast (9.56 ms) : 9395, 9725
.   : milestone, 9560,
iast_FULL (14.71 ms) : 14419, 15000
.   : milestone, 14710,
iast_GLOBAL (11.052 ms) : 10853, 11250
.   : milestone, 11052,
profiling (9.445 ms) : 9288, 9602
.   : milestone, 9445,
tracing (7.719 ms) : 7609, 7829
.   : milestone, 7719,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.299 ms [4.249 ms, 4.349 ms]	-
iast	10.228 ms [10.054 ms, 10.401 ms]	5.929 ms (137.9%)
iast_FULL	14.324 ms [14.042 ms, 14.606 ms]	10.025 ms (233.2%)
iast_GLOBAL	10.335 ms [10.155 ms, 10.516 ms]	6.036 ms (140.4%)
profiling	8.781 ms [8.643 ms, 8.919 ms]	4.482 ms (104.3%)
tracing	7.872 ms [7.759 ms, 7.986 ms]	3.573 ms (83.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.543 ms [4.491 ms, 4.595 ms]	-
iast	9.56 ms [9.395 ms, 9.725 ms]	5.017 ms (110.4%)
iast_FULL	14.71 ms [14.419 ms, 15.0 ms]	10.167 ms (223.8%)
iast_GLOBAL	11.052 ms [10.853 ms, 11.25 ms]	6.509 ms (143.3%)
profiling	9.445 ms [9.288 ms, 9.602 ms]	4.902 ms (107.9%)
tracing	7.719 ms [7.609 ms, 7.829 ms]	3.176 ms (69.9%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.54.0-SNAPSHOT~6b5709a70e, baseline=1.54.0-SNAPSHOT~050cad8b0e
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.378 ms) : 36097, 36659
.   : milestone, 36378,
appsec (49.258 ms) : 48822, 49694
.   : milestone, 49258,
code_origins (44.879 ms) : 44490, 45268
.   : milestone, 44879,
iast (43.168 ms) : 42808, 43528
.   : milestone, 43168,
profiling (48.122 ms) : 47673, 48571
.   : milestone, 48122,
tracing (45.469 ms) : 45075, 45862
.   : milestone, 45469,
section candidate
no_agent (36.414 ms) : 36132, 36696
.   : milestone, 36414,
appsec (49.861 ms) : 49422, 50300
.   : milestone, 49861,
code_origins (45.074 ms) : 44680, 45468
.   : milestone, 45074,
iast (46.525 ms) : 46129, 46920
.   : milestone, 46525,
profiling (49.988 ms) : 49511, 50464
.   : milestone, 49988,
tracing (44.562 ms) : 44176, 44948
.   : milestone, 44562,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.378 ms [36.097 ms, 36.659 ms]	-
appsec	49.258 ms [48.822 ms, 49.694 ms]	12.88 ms (35.4%)
code_origins	44.879 ms [44.49 ms, 45.268 ms]	8.502 ms (23.4%)
iast	43.168 ms [42.808 ms, 43.528 ms]	6.79 ms (18.7%)
profiling	48.122 ms [47.673 ms, 48.571 ms]	11.744 ms (32.3%)
tracing	45.469 ms [45.075 ms, 45.862 ms]	9.091 ms (25.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.414 ms [36.132 ms, 36.696 ms]	-
appsec	49.861 ms [49.422 ms, 50.3 ms]	13.447 ms (36.9%)
code_origins	45.074 ms [44.68 ms, 45.468 ms]	8.66 ms (23.8%)
iast	46.525 ms [46.129 ms, 46.92 ms]	10.111 ms (27.8%)
profiling	49.988 ms [49.511 ms, 50.464 ms]	13.574 ms (37.3%)
tracing	44.562 ms [44.176 ms, 44.948 ms]	8.148 ms (22.4%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	sarahchen6/convert-smoke-test-tasks-to-lazy-apis
git_commit_date	1758813010	1758816560
git_commit_sha	`050cad8`	`6b5709a`
release_version	1.54.0-SNAPSHOT~050cad8b0e	1.54.0-SNAPSHOT~6b5709a70e

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1758818644	1758818644
ci_job_id	1148078494	1148078494
ci_pipeline_id	77552694	77552694
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-r6whkfi5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-r6whkfi5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.54.0-SNAPSHOT~6b5709a70e, baseline=1.54.0-SNAPSHOT~050cad8b0e
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.477 ms) : 1465, 1488
.   : milestone, 1477,
appsec (3.697 ms) : 3482, 3912
.   : milestone, 3697,
iast (2.214 ms) : 2151, 2277
.   : milestone, 2214,
iast_GLOBAL (2.25 ms) : 2187, 2314
.   : milestone, 2250,
profiling (2.049 ms) : 1999, 2100
.   : milestone, 2049,
tracing (2.019 ms) : 1971, 2068
.   : milestone, 2019,
section candidate
no_agent (1.479 ms) : 1468, 1491
.   : milestone, 1479,
appsec (3.631 ms) : 3419, 3844
.   : milestone, 3631,
iast (2.206 ms) : 2143, 2270
.   : milestone, 2206,
iast_GLOBAL (2.249 ms) : 2185, 2312
.   : milestone, 2249,
profiling (2.074 ms) : 2022, 2126
.   : milestone, 2074,
tracing (2.028 ms) : 1979, 2077
.   : milestone, 2028,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.477 ms [1.465 ms, 1.488 ms]	-
appsec	3.697 ms [3.482 ms, 3.912 ms]	2.22 ms (150.3%)
iast	2.214 ms [2.151 ms, 2.277 ms]	736.883 µs (49.9%)
iast_GLOBAL	2.25 ms [2.187 ms, 2.314 ms]	773.485 µs (52.4%)
profiling	2.049 ms [1.999 ms, 2.1 ms]	572.673 µs (38.8%)
tracing	2.019 ms [1.971 ms, 2.068 ms]	542.384 µs (36.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.479 ms [1.468 ms, 1.491 ms]	-
appsec	3.631 ms [3.419 ms, 3.844 ms]	2.152 ms (145.5%)
iast	2.206 ms [2.143 ms, 2.27 ms]	727.066 µs (49.2%)
iast_GLOBAL	2.249 ms [2.185 ms, 2.312 ms]	769.631 µs (52.0%)
profiling	2.074 ms [2.022 ms, 2.126 ms]	594.384 µs (40.2%)
tracing	2.028 ms [1.979 ms, 2.077 ms]	548.83 µs (37.1%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.54.0-SNAPSHOT~6b5709a70e, baseline=1.54.0-SNAPSHOT~050cad8b0e
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.361 s) : 15361000, 15361000
.   : milestone, 15361000,
appsec (14.819 s) : 14819000, 14819000
.   : milestone, 14819000,
iast (18.542 s) : 18542000, 18542000
.   : milestone, 18542000,
iast_GLOBAL (17.697 s) : 17697000, 17697000
.   : milestone, 17697000,
profiling (15.863 s) : 15863000, 15863000
.   : milestone, 15863000,
tracing (15.104 s) : 15104000, 15104000
.   : milestone, 15104000,
section candidate
no_agent (15.301 s) : 15301000, 15301000
.   : milestone, 15301000,
appsec (15.032 s) : 15032000, 15032000
.   : milestone, 15032000,
iast (18.432 s) : 18432000, 18432000
.   : milestone, 18432000,
iast_GLOBAL (18.133 s) : 18133000, 18133000
.   : milestone, 18133000,
profiling (15.87 s) : 15870000, 15870000
.   : milestone, 15870000,
tracing (15.335 s) : 15335000, 15335000
.   : milestone, 15335000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.361 s [15.361 s, 15.361 s]	-
appsec	14.819 s [14.819 s, 14.819 s]	-542.0 ms (-3.5%)
iast	18.542 s [18.542 s, 18.542 s]	3.181 s (20.7%)
iast_GLOBAL	17.697 s [17.697 s, 17.697 s]	2.336 s (15.2%)
profiling	15.863 s [15.863 s, 15.863 s]	502.0 ms (3.3%)
tracing	15.104 s [15.104 s, 15.104 s]	-257.0 ms (-1.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.301 s [15.301 s, 15.301 s]	-
appsec	15.032 s [15.032 s, 15.032 s]	-269.0 ms (-1.8%)
iast	18.432 s [18.432 s, 18.432 s]	3.131 s (20.5%)
iast_GLOBAL	18.133 s [18.133 s, 18.133 s]	2.832 s (18.5%)
profiling	15.87 s [15.87 s, 15.87 s]	569.0 ms (3.7%)
tracing	15.335 s [15.335 s, 15.335 s]	34.0 ms (0.2%)

sarahchen6 · 2025-09-25T17:00:58Z

dd-smoke-tests/appsec/springboot-security/build.gradle


-forbiddenApisMain {
+tasks.named("forbiddenApisMain") {
  failOnMissingClasses = false


Wasn't able to add the ForbiddenApisTask class because the Groovy compiler fails to recognize / load this class until after the de.thetaphi.forbiddenapis.gradle.ForbiddenApisTask plugin is applied... but it hasn't been loaded? Let me know if there's a better way to do this!

Not sure, maybe use import ?

I tried import and still got the error of unable to resolve class... from the import line 😬

We can keep as is. This ForbiddenApisTask kind of problematic to me. It is not working well with Java 21+ for example. We may think how to replace it all together, maybe with fork or internal tool. Like @bric3 recently did for version plugin.

Unsure yet. But the issue with forbidden task is tricky with script plug-ins, there are different classloaders for script plugins. And probably require a proper declaration in the plugins {} block. This should be solved with convention plugins.

Okay sounds good.

sarahchen6 added 4 commits September 25, 2025 11:06

Update smoke test tasks A-C

9049985

Update smoke test tasks D-M

faca8da

Update smoke test tasks N-R

72a7685

Update smoke test tasks S-Z

b7a656a

sarahchen6 added type: enhancement Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: tooling Build & Tooling labels Sep 25, 2025

Add missing types

6b5709a

sarahchen6 commented Sep 25, 2025

View reviewed changes

sarahchen6 marked this pull request as ready for review September 25, 2025 17:02

sarahchen6 requested review from a team as code owners September 25, 2025 17:02

sarahchen6 requested review from jpbempel, nikita-tkachenko-datadog, robertpi, jandro996, amarziali, bric3 and AlexeyKuznetsov-DD and removed request for a team September 25, 2025 17:02

AlexeyKuznetsov-DD approved these changes Sep 25, 2025

View reviewed changes

bric3 approved these changes Sep 25, 2025

View reviewed changes

sarahchen6 merged commit 61c2d97 into master Sep 25, 2025
543 checks passed

sarahchen6 deleted the sarahchen6/convert-smoke-test-tasks-to-lazy-apis branch September 25, 2025 17:50

github-actions bot added this to the 1.54.0 milestone Sep 25, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update smoke test tasks to use lazy API #9617

Update smoke test tasks to use lazy API #9617

Uh oh!

sarahchen6 commented Sep 25, 2025

Uh oh!

datadog-official bot commented Sep 25, 2025 •

edited by datadog-datadog-prod-us1 bot

Loading

Uh oh!

pr-commenter bot commented Sep 25, 2025

Uh oh!

sarahchen6 Sep 25, 2025

Uh oh!

AlexeyKuznetsov-DD Sep 25, 2025

Uh oh!

sarahchen6 Sep 25, 2025

Uh oh!

AlexeyKuznetsov-DD Sep 25, 2025

Uh oh!

bric3 Sep 25, 2025

Uh oh!

sarahchen6 Sep 25, 2025

Uh oh!

Uh oh!

Uh oh!

Update smoke test tasks to use lazy API #9617

Update smoke test tasks to use lazy API #9617

Uh oh!

Conversation

sarahchen6 commented Sep 25, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-official bot commented Sep 25, 2025 • edited by datadog-datadog-prod-us1 bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Sep 25, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

sarahchen6 Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

AlexeyKuznetsov-DD Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

sarahchen6 Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

AlexeyKuznetsov-DD Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

bric3 Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

sarahchen6 Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

datadog-official bot commented Sep 25, 2025 •

edited by datadog-datadog-prod-us1 bot

Loading