Fixed flaky test #9218

kr-igor · 2025-07-22T17:42:37Z

What Does This Do

Fix for a flaky test

Details

Elements in the collection were not sorted properly. Now this test should work as expected.
The issue was introduced in this PR.

github-actions · 2025-07-22T17:43:03Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

AlexeyKuznetsov-DD

LGTM, do not forget to add labels and proper PR description by template

pr-commenter · 2025-07-22T18:16:34Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	kr-igor/fix-flaky-test
git_commit_date	1753206619	1753215515
git_commit_sha	`7b6259d`	`69efa01`
release_version	1.51.1-SNAPSHOT~7b6259d446	1.51.1-SNAPSHOT~69efa01716

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1753217451	1753217451
ci_job_id	1042184538	1042184538
ci_pipeline_id	71377968	71377968
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-gzbj0d6i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-gzbj0d6i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 12 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.039 s) : 0, 1039345
Total [baseline] (10.69 s) : 0, 10689561
Agent [candidate] (1.04 s) : 0, 1039680
Total [candidate] (10.662 s) : 0, 10661526
section appsec
Agent [baseline] (1.223 s) : 0, 1222684
Total [baseline] (10.811 s) : 0, 10811109
Agent [candidate] (1.22 s) : 0, 1219539
Total [candidate] (10.756 s) : 0, 10756000
section iast
Agent [baseline] (1.173 s) : 0, 1173332
Total [baseline] (10.881 s) : 0, 10880865
Agent [candidate] (1.176 s) : 0, 1175671
Total [candidate] (10.905 s) : 0, 10905463
section profiling
Agent [baseline] (1.192 s) : 0, 1191762
Total [baseline] (10.899 s) : 0, 10899405
Agent [candidate] (1.185 s) : 0, 1184657
Total [candidate] (10.878 s) : 0, 10878138

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.039 s	-
Agent	appsec	1.223 s	183.339 ms (17.6%)
Agent	iast	1.173 s	133.986 ms (12.9%)
Agent	profiling	1.192 s	152.417 ms (14.7%)
Total	tracing	10.69 s	-
Total	appsec	10.811 s	121.548 ms (1.1%)
Total	iast	10.881 s	191.304 ms (1.8%)
Total	profiling	10.899 s	209.844 ms (2.0%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.04 s	-
Agent	appsec	1.22 s	179.859 ms (17.3%)
Agent	iast	1.176 s	135.991 ms (13.1%)
Agent	profiling	1.185 s	144.977 ms (13.9%)
Total	tracing	10.662 s	-
Total	appsec	10.756 s	94.474 ms (0.9%)
Total	iast	10.905 s	243.937 ms (2.3%)
Total	profiling	10.878 s	216.613 ms (2.0%)

gantt
    title petclinic - break down per module: candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.422 ms) : 0, 1422
crashtracking [candidate] (1.43 ms) : 0, 1430
BytebuddyAgent [baseline] (729.528 ms) : 0, 729528
BytebuddyAgent [candidate] (729.855 ms) : 0, 729855
GlobalTracer [baseline] (242.215 ms) : 0, 242215
GlobalTracer [candidate] (241.985 ms) : 0, 241985
AppSec [baseline] (30.372 ms) : 0, 30372
AppSec [candidate] (30.528 ms) : 0, 30528
Debugger [baseline] (5.982 ms) : 0, 5982
Debugger [candidate] (5.996 ms) : 0, 5996
Remote Config [baseline] (640.252 µs) : 0, 640
Remote Config [candidate] (642.719 µs) : 0, 643
Telemetry [baseline] (8.17 ms) : 0, 8170
Telemetry [candidate] (8.181 ms) : 0, 8181
section appsec
crashtracking [baseline] (1.428 ms) : 0, 1428
crashtracking [candidate] (1.431 ms) : 0, 1431
BytebuddyAgent [baseline] (754.776 ms) : 0, 754776
BytebuddyAgent [candidate] (752.956 ms) : 0, 752956
GlobalTracer [baseline] (235.975 ms) : 0, 235975
GlobalTracer [candidate] (235.061 ms) : 0, 235061
AppSec [baseline] (168.865 ms) : 0, 168865
AppSec [candidate] (170.081 ms) : 0, 170081
Debugger [baseline] (7.216 ms) : 0, 7216
Debugger [candidate] (6.389 ms) : 0, 6389
Remote Config [baseline] (599.452 µs) : 0, 599
Remote Config [candidate] (599.61 µs) : 0, 600
Telemetry [baseline] (8.85 ms) : 0, 8850
Telemetry [candidate] (8.074 ms) : 0, 8074
IAST [baseline] (23.621 ms) : 0, 23621
IAST [candidate] (23.62 ms) : 0, 23620
section iast
crashtracking [baseline] (1.425 ms) : 0, 1425
crashtracking [candidate] (1.445 ms) : 0, 1445
BytebuddyAgent [baseline] (846.909 ms) : 0, 846909
BytebuddyAgent [candidate] (848.898 ms) : 0, 848898
GlobalTracer [baseline] (232.247 ms) : 0, 232247
GlobalTracer [candidate] (231.636 ms) : 0, 231636
AppSec [baseline] (26.582 ms) : 0, 26582
AppSec [candidate] (29.114 ms) : 0, 29114
Debugger [baseline] (8.383 ms) : 0, 8383
Debugger [candidate] (6.706 ms) : 0, 6706
Remote Config [baseline] (589.623 µs) : 0, 590
Remote Config [candidate] (589.598 µs) : 0, 590
Telemetry [baseline] (7.895 ms) : 0, 7895
Telemetry [candidate] (7.985 ms) : 0, 7985
IAST [baseline] (28.298 ms) : 0, 28298
IAST [candidate] (28.186 ms) : 0, 28186
section profiling
ProfilingAgent [baseline] (105.765 ms) : 0, 105765
ProfilingAgent [candidate] (104.365 ms) : 0, 104365
crashtracking [baseline] (1.411 ms) : 0, 1411
crashtracking [candidate] (1.4 ms) : 0, 1400
BytebuddyAgent [baseline] (763.937 ms) : 0, 763937
BytebuddyAgent [candidate] (759.724 ms) : 0, 759724
GlobalTracer [baseline] (222.232 ms) : 0, 222232
GlobalTracer [candidate] (220.922 ms) : 0, 220922
AppSec [baseline] (30.751 ms) : 0, 30751
AppSec [candidate] (30.27 ms) : 0, 30270
Debugger [baseline] (6.338 ms) : 0, 6338
Debugger [candidate] (6.282 ms) : 0, 6282
Remote Config [baseline] (691.639 µs) : 0, 692
Remote Config [candidate] (666.293 µs) : 0, 666
Telemetry [baseline] (11.843 ms) : 0, 11843
Telemetry [candidate] (12.476 ms) : 0, 12476
Profiling [baseline] (105.789 ms) : 0, 105789
Profiling [candidate] (104.386 ms) : 0, 104386

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.04 s) : 0, 1040318
Total [baseline] (8.645 s) : 0, 8645309
Agent [candidate] (1.049 s) : 0, 1048660
Total [candidate] (8.619 s) : 0, 8618800
section iast
Agent [baseline] (1.173 s) : 0, 1172829
Total [baseline] (9.332 s) : 0, 9332105
Agent [candidate] (1.174 s) : 0, 1173597
Total [candidate] (9.272 s) : 0, 9271877

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.04 s	-
Agent	iast	1.173 s	132.51 ms (12.7%)
Total	tracing	8.645 s	-
Total	iast	9.332 s	686.796 ms (7.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.049 s	-
Agent	iast	1.174 s	124.937 ms (11.9%)
Total	tracing	8.619 s	-
Total	iast	9.272 s	653.077 ms (7.6%)

gantt
    title insecure-bank - break down per module: candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.446 ms) : 0, 1446
crashtracking [candidate] (1.44 ms) : 0, 1440
BytebuddyAgent [baseline] (731.107 ms) : 0, 731107
BytebuddyAgent [candidate] (735.029 ms) : 0, 735029
GlobalTracer [baseline] (241.568 ms) : 0, 241568
GlobalTracer [candidate] (244.444 ms) : 0, 244444
AppSec [baseline] (30.299 ms) : 0, 30299
AppSec [candidate] (30.842 ms) : 0, 30842
Debugger [baseline] (5.997 ms) : 0, 5997
Debugger [candidate] (6.066 ms) : 0, 6066
Remote Config [baseline] (638.038 µs) : 0, 638
Remote Config [candidate] (656.927 µs) : 0, 657
Telemetry [baseline] (8.195 ms) : 0, 8195
Telemetry [candidate] (9.08 ms) : 0, 9080
section iast
crashtracking [baseline] (1.429 ms) : 0, 1429
crashtracking [candidate] (1.427 ms) : 0, 1427
BytebuddyAgent [baseline] (846.52 ms) : 0, 846520
BytebuddyAgent [candidate] (847.396 ms) : 0, 847396
GlobalTracer [baseline] (231.195 ms) : 0, 231195
GlobalTracer [candidate] (231.525 ms) : 0, 231525
AppSec [baseline] (27.46 ms) : 0, 27460
AppSec [candidate] (28.117 ms) : 0, 28117
Debugger [baseline] (8.346 ms) : 0, 8346
Debugger [candidate] (5.781 ms) : 0, 5781
Remote Config [baseline] (583.554 µs) : 0, 584
Remote Config [candidate] (584.698 µs) : 0, 585
Telemetry [baseline] (8.815 ms) : 0, 8815
Telemetry [candidate] (7.901 ms) : 0, 7901
IAST [baseline] (27.504 ms) : 0, 27504
IAST [candidate] (29.919 ms) : 0, 29919

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	kr-igor/fix-flaky-test
git_commit_date	1753206619	1753215515
git_commit_sha	`7b6259d`	`69efa01`
release_version	1.51.1-SNAPSHOT~7b6259d446	1.51.1-SNAPSHOT~69efa01716

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1753217197	1753217197
ci_job_id	1042184539	1042184539
ci_pipeline_id	71377968	71377968
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-bl6f9f1b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-bl6f9f1b 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 1 performance regressions! Performance is the same for 8 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:appsec:high_load	better [-2.623ms; -1.695ms] or [-5.339%; -3.450%]	unstable [-2.635op/s; +11.310op/s] or [-2.766%; +11.873%]	46.972ms	99.600op/s	49.131ms	95.263op/s
scenario:load:petclinic:profiling:high_load	better [-2.693ms; -1.703ms] or [-5.449%; -3.445%]	unstable [-2.945op/s; +11.745op/s] or [-3.110%; +12.402%]	47.228ms	99.100op/s	49.426ms	94.700op/s
scenario:load:petclinic:tracing:high_load	worse [+1.126ms; +1.912ms] or [+2.610%; +4.434%]	unstable [-11.267op/s; +3.917op/s] or [-10.387%; +3.611%]	44.639ms	104.800op/s	43.120ms	108.475op/s
scenario:load:petclinic:code_origins:high_load	better [-2.146ms; -1.303ms] or [-4.651%; -2.824%]	unstable [-3.403op/s; +11.203op/s] or [-3.356%; +11.048%]	44.423ms	105.300op/s	46.148ms	101.400op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.955 ms) : 36649, 37261
.   : milestone, 36955,
appsec (49.131 ms) : 48691, 49570
.   : milestone, 49131,
code_origins (46.148 ms) : 45750, 46547
.   : milestone, 46148,
iast (44.223 ms) : 43826, 44619
.   : milestone, 44223,
profiling (49.426 ms) : 48952, 49900
.   : milestone, 49426,
tracing (43.12 ms) : 42771, 43470
.   : milestone, 43120,
section candidate
no_agent (37.047 ms) : 36755, 37338
.   : milestone, 37047,
appsec (46.972 ms) : 46549, 47395
.   : milestone, 46972,
code_origins (44.423 ms) : 44038, 44808
.   : milestone, 44423,
iast (44.944 ms) : 44566, 45323
.   : milestone, 44944,
profiling (47.228 ms) : 46782, 47675
.   : milestone, 47228,
tracing (44.639 ms) : 44259, 45020
.   : milestone, 44639,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.955 ms [36.649 ms, 37.261 ms]	-
appsec	49.131 ms [48.691 ms, 49.57 ms]	12.176 ms (32.9%)
code_origins	46.148 ms [45.75 ms, 46.547 ms]	9.193 ms (24.9%)
iast	44.223 ms [43.826 ms, 44.619 ms]	7.268 ms (19.7%)
profiling	49.426 ms [48.952 ms, 49.9 ms]	12.471 ms (33.7%)
tracing	43.12 ms [42.771 ms, 43.47 ms]	6.166 ms (16.7%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.047 ms [36.755 ms, 37.338 ms]	-
appsec	46.972 ms [46.549 ms, 47.395 ms]	9.925 ms (26.8%)
code_origins	44.423 ms [44.038 ms, 44.808 ms]	7.376 ms (19.9%)
iast	44.944 ms [44.566 ms, 45.323 ms]	7.898 ms (21.3%)
profiling	47.228 ms [46.782 ms, 47.675 ms]	10.181 ms (27.5%)
tracing	44.639 ms [44.259 ms, 45.02 ms]	7.593 ms (20.5%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.336 ms) : 4275, 4396
.   : milestone, 4336,
iast (9.571 ms) : 9402, 9740
.   : milestone, 9571,
iast_FULL (13.921 ms) : 13646, 14197
.   : milestone, 13921,
iast_GLOBAL (10.001 ms) : 9816, 10187
.   : milestone, 10001,
profiling (9.019 ms) : 8876, 9161
.   : milestone, 9019,
tracing (7.424 ms) : 7312, 7535
.   : milestone, 7424,
section candidate
no_agent (4.375 ms) : 4322, 4428
.   : milestone, 4375,
iast (9.474 ms) : 9313, 9636
.   : milestone, 9474,
iast_FULL (13.813 ms) : 13539, 14086
.   : milestone, 13813,
iast_GLOBAL (9.999 ms) : 9825, 10172
.   : milestone, 9999,
profiling (9.316 ms) : 9174, 9459
.   : milestone, 9316,
tracing (7.372 ms) : 7271, 7473
.   : milestone, 7372,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.336 ms [4.275 ms, 4.396 ms]	-
iast	9.571 ms [9.402 ms, 9.74 ms]	5.235 ms (120.7%)
iast_FULL	13.921 ms [13.646 ms, 14.197 ms]	9.586 ms (221.1%)
iast_GLOBAL	10.001 ms [9.816 ms, 10.187 ms]	5.666 ms (130.7%)
profiling	9.019 ms [8.876 ms, 9.161 ms]	4.683 ms (108.0%)
tracing	7.424 ms [7.312 ms, 7.535 ms]	3.088 ms (71.2%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.375 ms [4.322 ms, 4.428 ms]	-
iast	9.474 ms [9.313 ms, 9.636 ms]	5.099 ms (116.6%)
iast_FULL	13.813 ms [13.539 ms, 14.086 ms]	9.437 ms (215.7%)
iast_GLOBAL	9.999 ms [9.825 ms, 10.172 ms]	5.624 ms (128.5%)
profiling	9.316 ms [9.174 ms, 9.459 ms]	4.941 ms (112.9%)
tracing	7.372 ms [7.271 ms, 7.473 ms]	2.997 ms (68.5%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	kr-igor/fix-flaky-test
git_commit_date	1753206619	1753215515
git_commit_sha	`7b6259d`	`69efa01`
release_version	1.51.1-SNAPSHOT~7b6259d446	1.51.1-SNAPSHOT~69efa01716

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1753217725	1753217725
ci_job_id	1042184540	1042184540
ci_pipeline_id	71377968	71377968
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-hs3nlbib 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-hs3nlbib 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (3.616 ms) : 3401, 3831
.   : milestone, 3616,
iast (2.187 ms) : 2125, 2250
.   : milestone, 2187,
iast_GLOBAL (2.232 ms) : 2169, 2295
.   : milestone, 2232,
profiling (2.053 ms) : 2001, 2105
.   : milestone, 2053,
tracing (2.001 ms) : 1952, 2049
.   : milestone, 2001,
section candidate
no_agent (1.476 ms) : 1464, 1488
.   : milestone, 1476,
appsec (3.669 ms) : 3454, 3885
.   : milestone, 3669,
iast (2.19 ms) : 2127, 2252
.   : milestone, 2190,
iast_GLOBAL (2.25 ms) : 2187, 2314
.   : milestone, 2250,
profiling (2.485 ms) : 2311, 2658
.   : milestone, 2485,
tracing (2.008 ms) : 1959, 2057
.   : milestone, 2008,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.472 ms [1.46 ms, 1.483 ms]	-
appsec	3.616 ms [3.401 ms, 3.831 ms]	2.144 ms (145.7%)
iast	2.187 ms [2.125 ms, 2.25 ms]	715.569 µs (48.6%)
iast_GLOBAL	2.232 ms [2.169 ms, 2.295 ms]	760.056 µs (51.6%)
profiling	2.053 ms [2.001 ms, 2.105 ms]	581.849 µs (39.5%)
tracing	2.001 ms [1.952 ms, 2.049 ms]	529.247 µs (36.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.476 ms [1.464 ms, 1.488 ms]	-
appsec	3.669 ms [3.454 ms, 3.885 ms]	2.193 ms (148.6%)
iast	2.19 ms [2.127 ms, 2.252 ms]	713.654 µs (48.3%)
iast_GLOBAL	2.25 ms [2.187 ms, 2.314 ms]	774.185 µs (52.5%)
profiling	2.485 ms [2.311 ms, 2.658 ms]	1.009 ms (68.3%)
tracing	2.008 ms [1.959 ms, 2.057 ms]	531.969 µs (36.0%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~69efa01716, baseline=1.51.1-SNAPSHOT~7b6259d446
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.538 s) : 15538000, 15538000
.   : milestone, 15538000,
appsec (14.937 s) : 14937000, 14937000
.   : milestone, 14937000,
iast (18.011 s) : 18011000, 18011000
.   : milestone, 18011000,
iast_GLOBAL (17.808 s) : 17808000, 17808000
.   : milestone, 17808000,
profiling (15.36 s) : 15360000, 15360000
.   : milestone, 15360000,
tracing (14.699 s) : 14699000, 14699000
.   : milestone, 14699000,
section candidate
no_agent (15.503 s) : 15503000, 15503000
.   : milestone, 15503000,
appsec (14.991 s) : 14991000, 14991000
.   : milestone, 14991000,
iast (18.875 s) : 18875000, 18875000
.   : milestone, 18875000,
iast_GLOBAL (18.006 s) : 18006000, 18006000
.   : milestone, 18006000,
profiling (15.573 s) : 15573000, 15573000
.   : milestone, 15573000,
tracing (14.927 s) : 14927000, 14927000
.   : milestone, 14927000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.538 s [15.538 s, 15.538 s]	-
appsec	14.937 s [14.937 s, 14.937 s]	-601.0 ms (-3.9%)
iast	18.011 s [18.011 s, 18.011 s]	2.473 s (15.9%)
iast_GLOBAL	17.808 s [17.808 s, 17.808 s]	2.27 s (14.6%)
profiling	15.36 s [15.36 s, 15.36 s]	-178.0 ms (-1.1%)
tracing	14.699 s [14.699 s, 14.699 s]	-839.0 ms (-5.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.503 s [15.503 s, 15.503 s]	-
appsec	14.991 s [14.991 s, 14.991 s]	-512.0 ms (-3.3%)
iast	18.875 s [18.875 s, 18.875 s]	3.372 s (21.8%)
iast_GLOBAL	18.006 s [18.006 s, 18.006 s]	2.503 s (16.1%)
profiling	15.573 s [15.573 s, 15.573 s]	70.0 ms (0.5%)
tracing	14.927 s [14.927 s, 14.927 s]	-576.0 ms (-3.7%)

pr-commenter · 2025-07-22T18:18:25Z

Kafka / producer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	kr-igor/fix-flaky-test
git_commit_date	1753205644	1753215515
git_commit_sha	`f21ceea`	`69efa01`

See matching parameters

	Baseline	Candidate
ci_job_date	1753216787	1753216787
ci_job_id	1042184543	1042184543
ci_pipeline_id	71377968	71377968
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce	same

pr-commenter · 2025-07-22T21:10:16Z

Kafka / consumer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	kr-igor/fix-flaky-test
git_commit_date	1753205644	1753215515
git_commit_sha	`f21ceea`	`69efa01`

See matching parameters

	Baseline	Candidate
ci_job_date	1753216787	1753216787
ci_job_id	1042184544	1042184544
ci_pipeline_id	71377968	71377968
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.25	11.0.25
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant	-Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.25+9-post-Ubuntu-1ubuntu122.04	11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same

Fixed flaky test

557cbfc

kr-igor marked this pull request as ready for review July 22, 2025 17:42

kr-igor requested a review from a team as a code owner July 22, 2025 17:42

kr-igor added comp: testing Testing comp: data streams Data Streams Monitoring labels Jul 22, 2025

AlexeyKuznetsov-DD approved these changes Jul 22, 2025

View reviewed changes

kr-igor added the type: bug Bug report and fix label Jul 22, 2025

bm1549 mentioned this pull request Jul 22, 2025

Match Hands Off Config selectors on process_arguments value #9201

Merged

AlexeyKuznetsov-DD added the tag: no release notes Changes to exclude from release notes label Jul 22, 2025

Removed kafka cluster id check, as it's sometimes equal to empty string

69efa01

kr-igor merged commit 2469447 into master Jul 22, 2025
505 checks passed

kr-igor deleted the kr-igor/fix-flaky-test branch July 22, 2025 21:33

github-actions bot added this to the 1.52.0 milestone Jul 22, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fixed flaky test #9218

Fixed flaky test #9218

Uh oh!

kr-igor commented Jul 22, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 22, 2025 •

edited

Loading

Uh oh!

AlexeyKuznetsov-DD left a comment

Uh oh!

pr-commenter bot commented Jul 22, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Jul 22, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Jul 22, 2025

Uh oh!

Uh oh!

Uh oh!

Fixed flaky test #9218

Fixed flaky test #9218

Uh oh!

Conversation

kr-igor commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Details

Uh oh!

github-actions bot commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexeyKuznetsov-DD left a comment

Choose a reason for hiding this comment

Uh oh!

pr-commenter bot commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

pr-commenter bot commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Kafka / producer-benchmark

Parameters

Summary

Uh oh!

pr-commenter bot commented Jul 22, 2025

Kafka / consumer-benchmark

Parameters

Summary

Uh oh!

Uh oh!

Uh oh!

kr-igor commented Jul 22, 2025 •

edited

Loading

github-actions bot commented Jul 22, 2025 •

edited

Loading

pr-commenter bot commented Jul 22, 2025 •

edited

Loading

pr-commenter bot commented Jul 22, 2025 •

edited

Loading