Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add line probe exploration tests #8741

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add line probe exploration tests #8741

wants to merge 1 commit into from

Conversation

jpbempel
Copy link
Member

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@pr-commenter
Copy link

pr-commenter bot commented Apr 28, 2025 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1746610454 1746610823
end_time 2025-05-07T09:35:32 2025-05-07T09:41:41
git_branch master jpbempel/itw-line-probe
git_commit_sha a964c3d ccf6196
start_time 2025-05-07T09:34:15 2025-05-07T09:40:24
See matching parameters
Baseline Candidate
ci_job_id 927433452 927433452
ci_pipeline_id 64366791 64366791
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1746609121 1746609121

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 5 metrics, 5 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop worse
[+678.469µs; +686.254µs] or [+6.801%; +6.879%]		 worse
[+691.378µs; +754.766µs] or [+6.830%; +7.457%]		 worse
[+652.943µs; +757.826µs] or [+6.387%; +7.413%]		 worse
[+543.552µs; +982.530µs] or [+5.133%; +9.279%]		 worse
[-6.871op/s; -5.603op/s] or [-7.042%; -5.743%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-15876.002ns; +16989.797ns] or [-5.834%; +6.243%]		 unstable
[-28.352µs; +23.969µs] or [-9.090%; +7.684%]		 unstable
[-39.478µs; +33.733µs] or [-12.116%; +10.353%]		 unstable
[-85.803µs; +93.826µs] or [-13.899%; +15.199%]		 same
scenario:basic same same same unstable
[-53.619µs; +37.389µs] or [-9.326%; +6.503%]		 same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (311.919 µs) : 287, 337
.   : milestone, 312,
basic (310.35 µs) : 302, 319
.   : milestone, 310,
loop (10.122 ms) : 10100, 10144
.   : milestone, 10122,
section candidate
noprobe (309.727 µs) : 285, 334
.   : milestone, 310,
basic (306.042 µs) : 295, 317
.   : milestone, 306,
loop (10.845 ms) : 10810, 10880
.   : milestone, 10845,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 311.919 µs [286.701 µs, 337.136 µs]
basic 310.35 µs [301.792 µs, 318.908 µs]
loop 10.122 ms [10.1 ms, 10.144 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 309.727 µs [285.092 µs, 334.362 µs]
basic 306.042 µs [295.122 µs, 316.961 µs]
loop 10.845 ms [10.81 ms, 10.88 ms]

@pr-commenter
Copy link

pr-commenter bot commented Apr 28, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/itw-line-probe
git_commit_date 1746607993 1746609121
git_commit_sha a964c3d ccf6196
release_version 1.50.0-SNAPSHOT~a964c3d339 1.49.0-SNAPSHOT~ccf6196684
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1746612181 1746612181
ci_job_id 927433444 927433444
ci_pipeline_id 64366791 64366791
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-ps6td5ta-project-304-concurrent-0-4123isyv 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-ps6td5ta-project-304-concurrent-0-4123isyv 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 64 metrics, 7 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~ccf6196684, baseline=1.50.0-SNAPSHOT~a964c3d339

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.008 s) : 0, 1008443
Total [baseline] (10.478 s) : 0, 10477805
Agent [candidate] (1.007 s) : 0, 1006640
Total [candidate] (10.55 s) : 0, 10550336
section appsec
Agent [baseline] (1.155 s) : 0, 1155072
Total [baseline] (10.684 s) : 0, 10683555
Agent [candidate] (1.149 s) : 0, 1149278
Total [candidate] (10.675 s) : 0, 10675239
section iast
Agent [baseline] (1.138 s) : 0, 1138256
Total [baseline] (10.883 s) : 0, 10883308
Agent [candidate] (1.145 s) : 0, 1144964
Total [candidate] (10.854 s) : 0, 10853593
section profiling
Agent [baseline] (1.258 s) : 0, 1258447
Total [baseline] (10.787 s) : 0, 10787469
Agent [candidate] (1.256 s) : 0, 1255623
Total [candidate] (10.811 s) : 0, 10810935
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.008 s -
Agent appsec 1.155 s 146.629 ms (14.5%)
Agent iast 1.138 s 129.813 ms (12.9%)
Agent profiling 1.258 s 250.004 ms (24.8%)
Total tracing 10.478 s -
Total appsec 10.684 s 205.75 ms (2.0%)
Total iast 10.883 s 405.503 ms (3.9%)
Total profiling 10.787 s 309.664 ms (3.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.007 s -
Agent appsec 1.149 s 142.638 ms (14.2%)
Agent iast 1.145 s 138.324 ms (13.7%)
Agent profiling 1.256 s 248.983 ms (24.7%)
Total tracing 10.55 s -
Total appsec 10.675 s 124.903 ms (1.2%)
Total iast 10.854 s 303.257 ms (2.9%)
Total profiling 10.811 s 260.599 ms (2.5%) 
gantt
    title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~ccf6196684, baseline=1.50.0-SNAPSHOT~a964c3d339

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (673.351 ms) : 0, 673351
BytebuddyAgent [candidate] (670.814 ms) : 0, 670814
GlobalTracer [baseline] (240.255 ms) : 0, 240255
GlobalTracer [candidate] (240.828 ms) : 0, 240828
AppSec [baseline] (54.64 ms) : 0, 54640
AppSec [candidate] (55.213 ms) : 0, 55213
Debugger [baseline] (6.877 ms) : 0, 6877
Debugger [candidate] (6.229 ms) : 0, 6229
Remote Config [baseline] (697.064 µs) : 0, 697
Remote Config [candidate] (720.724 µs) : 0, 721
Telemetry [baseline] (9.186 ms) : 0, 9186
Telemetry [candidate] (9.392 ms) : 0, 9392
section appsec
BytebuddyAgent [baseline] (691.503 ms) : 0, 691503
BytebuddyAgent [candidate] (688.388 ms) : 0, 688388
GlobalTracer [baseline] (238.43 ms) : 0, 238430
GlobalTracer [candidate] (236.789 ms) : 0, 236789
AppSec [baseline] (176.367 ms) : 0, 176367
AppSec [candidate] (175.903 ms) : 0, 175903
Debugger [baseline] (6.242 ms) : 0, 6242
Debugger [candidate] (5.877 ms) : 0, 5877
Remote Config [baseline] (639.256 µs) : 0, 639
Remote Config [candidate] (638.219 µs) : 0, 638
Telemetry [baseline] (7.418 ms) : 0, 7418
Telemetry [candidate] (7.407 ms) : 0, 7407
IAST [baseline] (21.876 ms) : 0, 21876
IAST [candidate] (21.644 ms) : 0, 21644
section iast
BytebuddyAgent [baseline] (790.173 ms) : 0, 790173
BytebuddyAgent [candidate] (796.692 ms) : 0, 796692
GlobalTracer [baseline] (230.933 ms) : 0, 230933
GlobalTracer [candidate] (230.726 ms) : 0, 230726
AppSec [baseline] (56.327 ms) : 0, 56327
AppSec [candidate] (56.366 ms) : 0, 56366
Debugger [baseline] (5.932 ms) : 0, 5932
Debugger [candidate] (5.984 ms) : 0, 5984
Remote Config [baseline] (597.119 µs) : 0, 597
Remote Config [candidate] (604.673 µs) : 0, 605
Telemetry [baseline] (7.905 ms) : 0, 7905
Telemetry [candidate] (8.017 ms) : 0, 8017
IAST [baseline] (22.929 ms) : 0, 22929
IAST [candidate] (22.99 ms) : 0, 22990
section profiling
BytebuddyAgent [baseline] (662.2 ms) : 0, 662200
BytebuddyAgent [candidate] (661.684 ms) : 0, 661684
GlobalTracer [baseline] (380.071 ms) : 0, 380071
GlobalTracer [candidate] (379.451 ms) : 0, 379451
AppSec [baseline] (54.734 ms) : 0, 54734
AppSec [candidate] (53.709 ms) : 0, 53709
Debugger [baseline] (6.153 ms) : 0, 6153
Debugger [candidate] (6.116 ms) : 0, 6116
Remote Config [baseline] (677.725 µs) : 0, 678
Remote Config [candidate] (651.815 µs) : 0, 652
Telemetry [baseline] (8.187 ms) : 0, 8187
Telemetry [candidate] (8.14 ms) : 0, 8140
ProfilingAgent [baseline] (96.218 ms) : 0, 96218
ProfilingAgent [candidate] (95.605 ms) : 0, 95605
Profiling [baseline] (96.242 ms) : 0, 96242
Profiling [candidate] (95.63 ms) : 0, 95630
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~ccf6196684, baseline=1.50.0-SNAPSHOT~a964c3d339

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.022 s) : 0, 1022338
Total [baseline] (8.645 s) : 0, 8645397
Agent [candidate] (1.021 s) : 0, 1021109
Total [candidate] (8.691 s) : 0, 8691395
section iast
Agent [baseline] (1.138 s) : 0, 1137665
Total [baseline] (9.215 s) : 0, 9215172
Agent [candidate] (1.147 s) : 0, 1146555
Total [candidate] (9.252 s) : 0, 9252189
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.138 s) : 0, 1138318
Total [baseline] (9.219 s) : 0, 9219128
Agent [candidate] (1.14 s) : 0, 1140195
Total [candidate] (9.204 s) : 0, 9203554
section iast_TELEMETRY_OFF
Agent [baseline] (1.134 s) : 0, 1133748
Total [baseline] (9.232 s) : 0, 9231719
Agent [candidate] (1.132 s) : 0, 1131943
Total [candidate] (9.202 s) : 0, 9202342
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.022 s -
Agent iast 1.138 s 115.328 ms (11.3%)
Agent iast_HARDCODED_SECRET_DISABLED 1.138 s 115.98 ms (11.3%)
Agent iast_TELEMETRY_OFF 1.134 s 111.411 ms (10.9%)
Total tracing 8.645 s -
Total iast 9.215 s 569.775 ms (6.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.219 s 573.731 ms (6.6%)
Total iast_TELEMETRY_OFF 9.232 s 586.322 ms (6.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.021 s -
Agent iast 1.147 s 125.446 ms (12.3%)
Agent iast_HARDCODED_SECRET_DISABLED 1.14 s 119.086 ms (11.7%)
Agent iast_TELEMETRY_OFF 1.132 s 110.834 ms (10.9%)
Total tracing 8.691 s -
Total iast 9.252 s 560.794 ms (6.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.204 s 512.159 ms (5.9%)
Total iast_TELEMETRY_OFF 9.202 s 510.947 ms (5.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~ccf6196684, baseline=1.50.0-SNAPSHOT~a964c3d339

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (679.713 ms) : 0, 679713
BytebuddyAgent [candidate] (680.185 ms) : 0, 680185
GlobalTracer [baseline] (242.655 ms) : 0, 242655
GlobalTracer [candidate] (243.411 ms) : 0, 243411
AppSec [baseline] (54.999 ms) : 0, 54999
AppSec [candidate] (55.317 ms) : 0, 55317
Debugger [baseline] (6.928 ms) : 0, 6928
Debugger [candidate] (7.02 ms) : 0, 7020
Remote Config [baseline] (699.915 µs) : 0, 700
Remote Config [candidate] (715.51 µs) : 0, 716
Telemetry [baseline] (13.626 ms) : 0, 13626
Telemetry [candidate] (10.647 ms) : 0, 10647
section iast
BytebuddyAgent [baseline] (789.91 ms) : 0, 789910
BytebuddyAgent [candidate] (796.372 ms) : 0, 796372
GlobalTracer [baseline] (230.413 ms) : 0, 230413
GlobalTracer [candidate] (231.803 ms) : 0, 231803
AppSec [baseline] (56.729 ms) : 0, 56729
AppSec [candidate] (57.048 ms) : 0, 57048
Debugger [baseline] (5.913 ms) : 0, 5913
Debugger [candidate] (5.991 ms) : 0, 5991
Remote Config [baseline] (609.148 µs) : 0, 609
Remote Config [candidate] (617.803 µs) : 0, 618
Telemetry [baseline] (7.947 ms) : 0, 7947
Telemetry [candidate] (8.093 ms) : 0, 8093
IAST [baseline] (22.755 ms) : 0, 22755
IAST [candidate] (22.971 ms) : 0, 22971
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (790.492 ms) : 0, 790492
BytebuddyAgent [candidate] (791.396 ms) : 0, 791396
GlobalTracer [baseline] (230.316 ms) : 0, 230316
GlobalTracer [candidate] (231.242 ms) : 0, 231242
AppSec [baseline] (56.629 ms) : 0, 56629
AppSec [candidate] (56.637 ms) : 0, 56637
Debugger [baseline] (5.95 ms) : 0, 5950
Debugger [candidate] (5.936 ms) : 0, 5936
Remote Config [baseline] (587.753 µs) : 0, 588
Remote Config [candidate] (602.327 µs) : 0, 602
Telemetry [baseline] (7.966 ms) : 0, 7966
Telemetry [candidate] (7.945 ms) : 0, 7945
IAST [baseline] (22.853 ms) : 0, 22853
IAST [candidate] (22.936 ms) : 0, 22936
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (787.09 ms) : 0, 787090
BytebuddyAgent [candidate] (785.718 ms) : 0, 785718
GlobalTracer [baseline] (230.268 ms) : 0, 230268
GlobalTracer [candidate] (229.954 ms) : 0, 229954
AppSec [baseline] (56.52 ms) : 0, 56520
AppSec [candidate] (56.395 ms) : 0, 56395
Debugger [baseline] (5.926 ms) : 0, 5926
Debugger [candidate] (5.907 ms) : 0, 5907
Remote Config [baseline] (598.293 µs) : 0, 598
Remote Config [candidate] (613.441 µs) : 0, 613
Telemetry [baseline] (7.817 ms) : 0, 7817
Telemetry [candidate] (7.805 ms) : 0, 7805
IAST [baseline] (22.136 ms) : 0, 22136
IAST [candidate] (22.125 ms) : 0, 22125
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-05-07T09:33:23 2025-05-07T09:41:06
git_branch master jpbempel/itw-line-probe
git_commit_date 1746607993 1746609121
git_commit_sha a964c3d ccf6196
release_version 1.50.0-SNAPSHOT~a964c3d339 1.49.0-SNAPSHOT~ccf6196684
start_time 2025-05-07T09:33:09 2025-05-07T09:40:52
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1746611262 1746611262
ci_job_id 927433446 927433446
ci_pipeline_id 64366791 64366791
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-ps6td5ta-project-304-concurrent-1-002dgx7v 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-ps6td5ta-project-304-concurrent-1-002dgx7v 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~ccf6196684, baseline=1.50.0-SNAPSHOT~a964c3d339
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.37 ms) : 1350, 1390
.   : milestone, 1370,
appsec (1.727 ms) : 1703, 1751
.   : milestone, 1727,
appsec_no_iast (1.716 ms) : 1692, 1739
.   : milestone, 1716,
code_origins (1.69 ms) : 1663, 1716
.   : milestone, 1690,
iast (1.513 ms) : 1488, 1538
.   : milestone, 1513,
profiling (1.519 ms) : 1495, 1543
.   : milestone, 1519,
tracing (1.485 ms) : 1460, 1511
.   : milestone, 1485,
section candidate
no_agent (1.346 ms) : 1326, 1366
.   : milestone, 1346,
appsec (1.738 ms) : 1714, 1762
.   : milestone, 1738,
appsec_no_iast (1.726 ms) : 1701, 1751
.   : milestone, 1726,
code_origins (1.667 ms) : 1640, 1694
.   : milestone, 1667,
iast (1.508 ms) : 1484, 1532
.   : milestone, 1508,
profiling (1.516 ms) : 1492, 1539
.   : milestone, 1516,
tracing (1.493 ms) : 1468, 1518
.   : milestone, 1493,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.37 ms [1.35 ms, 1.39 ms] -
appsec 1.727 ms [1.703 ms, 1.751 ms] 356.85 µs (26.0%)
appsec_no_iast 1.716 ms [1.692 ms, 1.739 ms] 345.349 µs (25.2%)
code_origins 1.69 ms [1.663 ms, 1.716 ms] 319.508 µs (23.3%)
iast 1.513 ms [1.488 ms, 1.538 ms] 142.737 µs (10.4%)
profiling 1.519 ms [1.495 ms, 1.543 ms] 148.983 µs (10.9%)
tracing 1.485 ms [1.46 ms, 1.511 ms] 114.895 µs (8.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.346 ms [1.326 ms, 1.366 ms] -
appsec 1.738 ms [1.714 ms, 1.762 ms] 391.539 µs (29.1%)
appsec_no_iast 1.726 ms [1.701 ms, 1.751 ms] 379.631 µs (28.2%)
code_origins 1.667 ms [1.64 ms, 1.694 ms] 320.971 µs (23.8%)
iast 1.508 ms [1.484 ms, 1.532 ms] 161.893 µs (12.0%)
profiling 1.516 ms [1.492 ms, 1.539 ms] 169.528 µs (12.6%)
tracing 1.493 ms [1.468 ms, 1.518 ms] 146.822 µs (10.9%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~ccf6196684, baseline=1.50.0-SNAPSHOT~a964c3d339
    dateFormat X
    axisFormat %s
section baseline
no_agent (374.355 µs) : 355, 394
.   : milestone, 374,
iast (510.21 µs) : 487, 533
.   : milestone, 510,
iast_FULL (727.762 µs) : 704, 751
.   : milestone, 728,
iast_GLOBAL (571.169 µs) : 547, 595
.   : milestone, 571,
iast_HARDCODED_SECRET_DISABLED (522.256 µs) : 499, 545
.   : milestone, 522,
iast_INACTIVE (457.796 µs) : 436, 479
.   : milestone, 458,
iast_TELEMETRY_OFF (508.778 µs) : 485, 532
.   : milestone, 509,
tracing (454.488 µs) : 432, 477
.   : milestone, 454,
section candidate
no_agent (379.492 µs) : 359, 400
.   : milestone, 379,
iast (512.988 µs) : 489, 537
.   : milestone, 513,
iast_FULL (738.687 µs) : 715, 762
.   : milestone, 739,
iast_GLOBAL (553.91 µs) : 531, 577
.   : milestone, 554,
iast_HARDCODED_SECRET_DISABLED (522.473 µs) : 500, 545
.   : milestone, 522,
iast_INACTIVE (462.003 µs) : 440, 484
.   : milestone, 462,
iast_TELEMETRY_OFF (510.213 µs) : 488, 532
.   : milestone, 510,
tracing (454.197 µs) : 433, 476
.   : milestone, 454,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 374.355 µs [354.807 µs, 393.903 µs] -
iast 510.21 µs [487.242 µs, 533.178 µs] 135.855 µs (36.3%)
iast_FULL 727.762 µs [704.197 µs, 751.328 µs] 353.408 µs (94.4%)
iast_GLOBAL 571.169 µs [547.146 µs, 595.191 µs] 196.814 µs (52.6%)
iast_HARDCODED_SECRET_DISABLED 522.256 µs [499.398 µs, 545.113 µs] 147.901 µs (39.5%)
iast_INACTIVE 457.796 µs [436.237 µs, 479.356 µs] 83.442 µs (22.3%)
iast_TELEMETRY_OFF 508.778 µs [485.459 µs, 532.097 µs] 134.423 µs (35.9%)
tracing 454.488 µs [432.185 µs, 476.791 µs] 80.133 µs (21.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 379.492 µs [359.215 µs, 399.769 µs] -
iast 512.988 µs [489.447 µs, 536.529 µs] 133.496 µs (35.2%)
iast_FULL 738.687 µs [715.334 µs, 762.039 µs] 359.195 µs (94.7%)
iast_GLOBAL 553.91 µs [530.89 µs, 576.93 µs] 174.418 µs (46.0%)
iast_HARDCODED_SECRET_DISABLED 522.473 µs [499.568 µs, 545.377 µs] 142.981 µs (37.7%)
iast_INACTIVE 462.003 µs [440.4 µs, 483.605 µs] 82.511 µs (21.7%)
iast_TELEMETRY_OFF 510.213 µs [488.0 µs, 532.425 µs] 130.721 µs (34.4%)
tracing 454.197 µs [432.607 µs, 475.788 µs] 74.705 µs (19.7%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/itw-line-probe
git_commit_date 1746610136 1746609121
git_commit_sha 13eba48 ccf6196
release_version 1.50.0-SNAPSHOT~13eba48d61 1.49.0-SNAPSHOT~ccf6196684
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1746611888 1746611888
ci_job_id 927433447 927433447
ci_pipeline_id 64366791 64366791
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-ps6td5ta-project-304-concurrent-2-di0xgm1g 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-ps6td5ta-project-304-concurrent-2-di0xgm1g 6.8.0-1027-aws #29~22.04.1-Ubuntu SMP Sun Mar 30 07:45:38 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~ccf6196684, baseline=1.50.0-SNAPSHOT~13eba48d61
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.49 s) : 15490000, 15490000
.   : milestone, 15490000,
appsec (15.02 s) : 15020000, 15020000
.   : milestone, 15020000,
iast (18.997 s) : 18997000, 18997000
.   : milestone, 18997000,
iast_GLOBAL (17.709 s) : 17709000, 17709000
.   : milestone, 17709000,
profiling (15.628 s) : 15628000, 15628000
.   : milestone, 15628000,
tracing (15.131 s) : 15131000, 15131000
.   : milestone, 15131000,
section candidate
no_agent (14.75 s) : 14750000, 14750000
.   : milestone, 14750000,
appsec (15.202 s) : 15202000, 15202000
.   : milestone, 15202000,
iast (18.304 s) : 18304000, 18304000
.   : milestone, 18304000,
iast_GLOBAL (18.15 s) : 18150000, 18150000
.   : milestone, 18150000,
profiling (15.039 s) : 15039000, 15039000
.   : milestone, 15039000,
tracing (15.231 s) : 15231000, 15231000
.   : milestone, 15231000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.49 s [15.49 s, 15.49 s] -
appsec 15.02 s [15.02 s, 15.02 s] -470.0 ms (-3.0%)
iast 18.997 s [18.997 s, 18.997 s] 3.507 s (22.6%)
iast_GLOBAL 17.709 s [17.709 s, 17.709 s] 2.219 s (14.3%)
profiling 15.628 s [15.628 s, 15.628 s] 138.0 ms (0.9%)
tracing 15.131 s [15.131 s, 15.131 s] -359.0 ms (-2.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.75 s [14.75 s, 14.75 s] -
appsec 15.202 s [15.202 s, 15.202 s] 452.0 ms (3.1%)
iast 18.304 s [18.304 s, 18.304 s] 3.554 s (24.1%)
iast_GLOBAL 18.15 s [18.15 s, 18.15 s] 3.4 s (23.1%)
profiling 15.039 s [15.039 s, 15.039 s] 289.0 ms (2.0%)
tracing 15.231 s [15.231 s, 15.231 s] 481.0 ms (3.3%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~ccf6196684, baseline=1.50.0-SNAPSHOT~13eba48d61
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.47 ms) : 1459, 1482
.   : milestone, 1470,
appsec (2.397 ms) : 2348, 2447
.   : milestone, 2397,
iast (2.185 ms) : 2122, 2247
.   : milestone, 2185,
iast_GLOBAL (2.233 ms) : 2170, 2296
.   : milestone, 2233,
profiling (2.04 ms) : 1989, 2091
.   : milestone, 2040,
tracing (2.028 ms) : 1978, 2077
.   : milestone, 2028,
section candidate
no_agent (1.47 ms) : 1458, 1481
.   : milestone, 1470,
appsec (2.399 ms) : 2349, 2448
.   : milestone, 2399,
iast (2.18 ms) : 2118, 2242
.   : milestone, 2180,
iast_GLOBAL (2.232 ms) : 2170, 2294
.   : milestone, 2232,
profiling (2.039 ms) : 1989, 2089
.   : milestone, 2039,
tracing (2.001 ms) : 1954, 2049
.   : milestone, 2001,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.459 ms, 1.482 ms] -
appsec 2.397 ms [2.348 ms, 2.447 ms] 927.111 µs (63.1%)
iast 2.185 ms [2.122 ms, 2.247 ms] 714.514 µs (48.6%)
iast_GLOBAL 2.233 ms [2.17 ms, 2.296 ms] 762.624 µs (51.9%)
profiling 2.04 ms [1.989 ms, 2.091 ms] 570.092 µs (38.8%)
tracing 2.028 ms [1.978 ms, 2.077 ms] 557.418 µs (37.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.458 ms, 1.481 ms] -
appsec 2.399 ms [2.349 ms, 2.448 ms] 929.109 µs (63.2%)
iast 2.18 ms [2.118 ms, 2.242 ms] 710.374 µs (48.3%)
iast_GLOBAL 2.232 ms [2.17 ms, 2.294 ms] 762.338 µs (51.9%)
profiling 2.039 ms [1.989 ms, 2.089 ms] 569.491 µs (38.8%)
tracing 2.001 ms [1.954 ms, 2.049 ms] 531.668 µs (36.2%)

@jpbempel jpbempel added the tag: no release notes Changes to exclude from release notes label Apr 28, 2025
@jpbempel jpbempel force-pushed the jpbempel/itw-line-probe branch 2 times, most recently from b995155 to 625db29 Compare May 5, 2025 08:58
@jpbempel jpbempel force-pushed the jpbempel/itw-line-probe branch 4 times, most recently from bbb2818 to 3423769 Compare May 6, 2025 08:53
@jpbempel
Add line probe exploration tests
ccf6196 
Refactor InstrumentTheWorld config to take a string indicating the
usage of method or line probe for the process.
Rename jobs to indicate line or method probes
@jpbempel jpbempel force-pushed the jpbempel/itw-line-probe branch from 3423769 to ccf6196 Compare May 7, 2025 09:12
@jpbempel jpbempel marked this pull request as ready for review May 7, 2025 11:03
@jpbempel jpbempel requested review from a team as code owners May 7, 2025 11:03
@jpbempel jpbempel requested review from evanchooly and mcculls and removed request for a team May 7, 2025 11:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@evanchooly evanchooly Awaiting requested review from evanchooly evanchooly is a code owner automatically assigned from DataDog/debugger-java

@mcculls mcculls Awaiting requested review from mcculls mcculls is a code owner automatically assigned from DataDog/apm-java

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation comp: testing Testing tag: no release notes Changes to exclude from release notes type: enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@jpbempel