Add methods to capture embedding and retrieval spans #9297

nayeem-kamal · 2025-08-04T05:49:02Z

What Does This Do

Adds methods to the manual SDK for LLMObs to allow for capturing embedding and retrieval spans.

Motivation

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

github-actions · 2025-08-04T05:49:14Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

datadog-datadog-prod-us1 · 2025-08-04T06:07:24Z

Code coverage: total 56.93%, patch 0.00% (view details)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 1a5545b | Docs | Was this helpful? Give us feedback!}

pr-commenter · 2025-08-04T06:22:26Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nayeem-kamal/embeddings-retrievals
git_commit_date	1754075408	1754286451
git_commit_sha	`15176df`	`1a5545b`
release_version	1.51.1-SNAPSHOT~15176df7b8	1.51.1-SNAPSHOT~1a5545bead

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1754288209	1754288209
ci_job_id	1061736451	1061736451
ci_pipeline_id	72541390	72541390
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-llbkv94e 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-llbkv94e 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 50 metrics, 9 unstable metrics.

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.51.1-SNAPSHOT~1a5545bead, baseline=1.51.1-SNAPSHOT~15176df7b8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.041 s) : 0, 1041070
Total [baseline] (8.615 s) : 0, 8615207
Agent [candidate] (1.052 s) : 0, 1052278
Total [candidate] (8.648 s) : 0, 8648041
section iast
Agent [baseline] (1.172 s) : 0, 1172461
Total [baseline] (9.329 s) : 0, 9328739
Agent [candidate] (1.171 s) : 0, 1171284
Total [candidate] (9.334 s) : 0, 9333980

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.041 s	-
Agent	iast	1.172 s	131.391 ms (12.6%)
Total	tracing	8.615 s	-
Total	iast	9.329 s	713.532 ms (8.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.052 s	-
Agent	iast	1.171 s	119.006 ms (11.3%)
Total	tracing	8.648 s	-
Total	iast	9.334 s	685.939 ms (7.9%)

gantt
    title insecure-bank - break down per module: candidate=1.51.1-SNAPSHOT~1a5545bead, baseline=1.51.1-SNAPSHOT~15176df7b8

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.429 ms) : 0, 1429
crashtracking [candidate] (1.435 ms) : 0, 1435
BytebuddyAgent [baseline] (729.191 ms) : 0, 729191
BytebuddyAgent [candidate] (735.602 ms) : 0, 735602
GlobalTracer [baseline] (241.547 ms) : 0, 241547
GlobalTracer [candidate] (243.165 ms) : 0, 243165
AppSec [baseline] (30.036 ms) : 0, 30036
AppSec [candidate] (30.302 ms) : 0, 30302
Debugger [baseline] (6.05 ms) : 0, 6050
Debugger [candidate] (6.09 ms) : 0, 6090
Remote Config [baseline] (642.206 µs) : 0, 642
Remote Config [candidate] (648.768 µs) : 0, 649
Telemetry [baseline] (11.249 ms) : 0, 11249
Telemetry [candidate] (14.042 ms) : 0, 14042
section iast
crashtracking [baseline] (1.427 ms) : 0, 1427
crashtracking [candidate] (1.435 ms) : 0, 1435
BytebuddyAgent [baseline] (846.439 ms) : 0, 846439
BytebuddyAgent [candidate] (845.86 ms) : 0, 845860
GlobalTracer [baseline] (231.516 ms) : 0, 231516
GlobalTracer [candidate] (231.767 ms) : 0, 231767
AppSec [baseline] (26.013 ms) : 0, 26013
AppSec [candidate] (26.935 ms) : 0, 26935
Debugger [baseline] (5.816 ms) : 0, 5816
Debugger [candidate] (5.761 ms) : 0, 5761
Remote Config [baseline] (590.973 µs) : 0, 591
Remote Config [candidate] (579.853 µs) : 0, 580
Telemetry [baseline] (8.044 ms) : 0, 8044
Telemetry [candidate] (8.093 ms) : 0, 8093
IAST [baseline] (31.697 ms) : 0, 31697
IAST [candidate] (29.975 ms) : 0, 29975

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.51.1-SNAPSHOT~1a5545bead, baseline=1.51.1-SNAPSHOT~15176df7b8

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.05 s) : 0, 1049846
Total [baseline] (10.729 s) : 0, 10728904
Agent [candidate] (1.044 s) : 0, 1044357
Total [candidate] (10.791 s) : 0, 10791058
section appsec
Agent [baseline] (1.216 s) : 0, 1216116
Total [baseline] (10.762 s) : 0, 10761600
Agent [candidate] (1.219 s) : 0, 1218615
Total [candidate] (10.867 s) : 0, 10867207
section iast
Agent [baseline] (1.182 s) : 0, 1181913
Total [baseline] (10.857 s) : 0, 10856882
Agent [candidate] (1.176 s) : 0, 1175695
Total [candidate] (10.867 s) : 0, 10866647
section profiling
Agent [baseline] (1.2 s) : 0, 1199691
Total [baseline] (10.991 s) : 0, 10991004
Agent [candidate] (1.198 s) : 0, 1198400
Total [candidate] (10.892 s) : 0, 10891569

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.05 s	-
Agent	appsec	1.216 s	166.269 ms (15.8%)
Agent	iast	1.182 s	132.067 ms (12.6%)
Agent	profiling	1.2 s	149.845 ms (14.3%)
Total	tracing	10.729 s	-
Total	appsec	10.762 s	32.696 ms (0.3%)
Total	iast	10.857 s	127.978 ms (1.2%)
Total	profiling	10.991 s	262.1 ms (2.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.044 s	-
Agent	appsec	1.219 s	174.257 ms (16.7%)
Agent	iast	1.176 s	131.338 ms (12.6%)
Agent	profiling	1.198 s	154.043 ms (14.8%)
Total	tracing	10.791 s	-
Total	appsec	10.867 s	76.149 ms (0.7%)
Total	iast	10.867 s	75.59 ms (0.7%)
Total	profiling	10.892 s	100.511 ms (0.9%)

gantt
    title petclinic - break down per module: candidate=1.51.1-SNAPSHOT~1a5545bead, baseline=1.51.1-SNAPSHOT~15176df7b8

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.433 ms) : 0, 1433
crashtracking [candidate] (1.433 ms) : 0, 1433
BytebuddyAgent [baseline] (736.748 ms) : 0, 736748
BytebuddyAgent [candidate] (731.272 ms) : 0, 731272
GlobalTracer [baseline] (243.014 ms) : 0, 243014
GlobalTracer [candidate] (243.067 ms) : 0, 243067
AppSec [baseline] (30.289 ms) : 0, 30289
AppSec [candidate] (30.331 ms) : 0, 30331
Debugger [baseline] (6.07 ms) : 0, 6070
Debugger [candidate] (6.069 ms) : 0, 6069
Remote Config [baseline] (655.173 µs) : 0, 655
Remote Config [candidate] (653.101 µs) : 0, 653
Telemetry [baseline] (10.484 ms) : 0, 10484
Telemetry [candidate] (10.492 ms) : 0, 10492
section appsec
crashtracking [baseline] (1.428 ms) : 0, 1428
crashtracking [candidate] (1.431 ms) : 0, 1431
BytebuddyAgent [baseline] (750.372 ms) : 0, 750372
BytebuddyAgent [candidate] (752.416 ms) : 0, 752416
GlobalTracer [baseline] (234.561 ms) : 0, 234561
GlobalTracer [candidate] (234.672 ms) : 0, 234672
AppSec [baseline] (167.076 ms) : 0, 167076
AppSec [candidate] (169.487 ms) : 0, 169487
Debugger [baseline] (9.377 ms) : 0, 9377
Debugger [candidate] (7.162 ms) : 0, 7162
Remote Config [baseline] (610.554 µs) : 0, 611
Remote Config [candidate] (602.931 µs) : 0, 603
Telemetry [baseline] (8.201 ms) : 0, 8201
Telemetry [candidate] (8.287 ms) : 0, 8287
IAST [baseline] (23.513 ms) : 0, 23513
IAST [candidate] (23.515 ms) : 0, 23515
section iast
crashtracking [baseline] (1.445 ms) : 0, 1445
crashtracking [candidate] (1.439 ms) : 0, 1439
BytebuddyAgent [baseline] (854.751 ms) : 0, 854751
BytebuddyAgent [candidate] (848.833 ms) : 0, 848833
GlobalTracer [baseline] (232.646 ms) : 0, 232646
GlobalTracer [candidate] (233.083 ms) : 0, 233083
AppSec [baseline] (25.249 ms) : 0, 25249
AppSec [candidate] (27.68 ms) : 0, 27680
Debugger [baseline] (6.733 ms) : 0, 6733
Debugger [candidate] (6.655 ms) : 0, 6655
Remote Config [baseline] (584.531 µs) : 0, 585
Remote Config [candidate] (585.582 µs) : 0, 586
Telemetry [baseline] (8.102 ms) : 0, 8102
Telemetry [candidate] (8.12 ms) : 0, 8120
IAST [baseline] (31.358 ms) : 0, 31358
IAST [candidate] (28.196 ms) : 0, 28196
section profiling
crashtracking [baseline] (1.412 ms) : 0, 1412
crashtracking [candidate] (1.411 ms) : 0, 1411
BytebuddyAgent [baseline] (764.453 ms) : 0, 764453
BytebuddyAgent [candidate] (764.314 ms) : 0, 764314
GlobalTracer [baseline] (222.496 ms) : 0, 222496
GlobalTracer [candidate] (222.693 ms) : 0, 222693
AppSec [baseline] (30.214 ms) : 0, 30214
AppSec [candidate] (30.116 ms) : 0, 30116
Debugger [baseline] (6.313 ms) : 0, 6313
Debugger [candidate] (6.314 ms) : 0, 6314
Remote Config [baseline] (683.387 µs) : 0, 683
Remote Config [candidate] (677.702 µs) : 0, 678
Telemetry [baseline] (15.25 ms) : 0, 15250
Telemetry [candidate] (15.8 ms) : 0, 15800
ProfilingAgent [baseline] (109.285 ms) : 0, 109285
ProfilingAgent [candidate] (107.459 ms) : 0, 107459
Profiling [baseline] (109.941 ms) : 0, 109941
Profiling [candidate] (108.099 ms) : 0, 108099

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nayeem-kamal/embeddings-retrievals
git_commit_date	1754075408	1754286451
git_commit_sha	`15176df`	`1a5545b`
release_version	1.51.1-SNAPSHOT~15176df7b8	1.51.1-SNAPSHOT~1a5545bead

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1754287891	1754287891
ci_job_id	1061736452	1061736452
ci_pipeline_id	72541390	72541390
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-fm3v96rk 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-fm3v96rk 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 12 unstable metrics.

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~1a5545bead, baseline=1.51.1-SNAPSHOT~15176df7b8
    dateFormat X
    axisFormat %s
section baseline
no_agent (36.859 ms) : 36564, 37155
.   : milestone, 36859,
appsec (47.719 ms) : 47321, 48118
.   : milestone, 47719,
code_origins (46.689 ms) : 46280, 47098
.   : milestone, 46689,
iast (43.597 ms) : 43212, 43981
.   : milestone, 43597,
profiling (46.462 ms) : 46011, 46913
.   : milestone, 46462,
tracing (43.113 ms) : 42753, 43473
.   : milestone, 43113,
section candidate
no_agent (37.639 ms) : 37330, 37948
.   : milestone, 37639,
appsec (48.82 ms) : 48378, 49261
.   : milestone, 48820,
code_origins (46.047 ms) : 45660, 46434
.   : milestone, 46047,
iast (44.539 ms) : 44154, 44925
.   : milestone, 44539,
profiling (47.382 ms) : 46970, 47794
.   : milestone, 47382,
tracing (43.784 ms) : 43415, 44153
.   : milestone, 43784,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	36.859 ms [36.564 ms, 37.155 ms]	-
appsec	47.719 ms [47.321 ms, 48.118 ms]	10.86 ms (29.5%)
code_origins	46.689 ms [46.28 ms, 47.098 ms]	9.83 ms (26.7%)
iast	43.597 ms [43.212 ms, 43.981 ms]	6.738 ms (18.3%)
profiling	46.462 ms [46.011 ms, 46.913 ms]	9.603 ms (26.1%)
tracing	43.113 ms [42.753 ms, 43.473 ms]	6.254 ms (17.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.639 ms [37.33 ms, 37.948 ms]	-
appsec	48.82 ms [48.378 ms, 49.261 ms]	11.181 ms (29.7%)
code_origins	46.047 ms [45.66 ms, 46.434 ms]	8.408 ms (22.3%)
iast	44.539 ms [44.154 ms, 44.925 ms]	6.9 ms (18.3%)
profiling	47.382 ms [46.97 ms, 47.794 ms]	9.743 ms (25.9%)
tracing	43.784 ms [43.415 ms, 44.153 ms]	6.145 ms (16.3%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.51.1-SNAPSHOT~1a5545bead, baseline=1.51.1-SNAPSHOT~15176df7b8
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.286 ms) : 4237, 4336
.   : milestone, 4286,
iast (9.323 ms) : 9170, 9475
.   : milestone, 9323,
iast_FULL (13.879 ms) : 13609, 14149
.   : milestone, 13879,
iast_GLOBAL (10.44 ms) : 10245, 10635
.   : milestone, 10440,
profiling (8.851 ms) : 8716, 8986
.   : milestone, 8851,
tracing (7.418 ms) : 7307, 7529
.   : milestone, 7418,
section candidate
no_agent (4.357 ms) : 4303, 4412
.   : milestone, 4357,
iast (9.583 ms) : 9426, 9739
.   : milestone, 9583,
iast_FULL (13.721 ms) : 13452, 13990
.   : milestone, 13721,
iast_GLOBAL (10.272 ms) : 10091, 10453
.   : milestone, 10272,
profiling (8.872 ms) : 8716, 9028
.   : milestone, 8872,
tracing (7.637 ms) : 7527, 7747
.   : milestone, 7637,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.286 ms [4.237 ms, 4.336 ms]	-
iast	9.323 ms [9.17 ms, 9.475 ms]	5.036 ms (117.5%)
iast_FULL	13.879 ms [13.609 ms, 14.149 ms]	9.593 ms (223.8%)
iast_GLOBAL	10.44 ms [10.245 ms, 10.635 ms]	6.153 ms (143.6%)
profiling	8.851 ms [8.716 ms, 8.986 ms]	4.565 ms (106.5%)
tracing	7.418 ms [7.307 ms, 7.529 ms]	3.132 ms (73.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.357 ms [4.303 ms, 4.412 ms]	-
iast	9.583 ms [9.426 ms, 9.739 ms]	5.225 ms (119.9%)
iast_FULL	13.721 ms [13.452 ms, 13.99 ms]	9.363 ms (214.9%)
iast_GLOBAL	10.272 ms [10.091 ms, 10.453 ms]	5.914 ms (135.7%)
profiling	8.872 ms [8.716 ms, 9.028 ms]	4.515 ms (103.6%)
tracing	7.637 ms [7.527 ms, 7.747 ms]	3.28 ms (75.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nayeem-kamal/embeddings-retrievals
git_commit_date	1754075408	1754286451
git_commit_sha	`15176df`	`1a5545b`
release_version	1.51.1-SNAPSHOT~15176df7b8	1.51.1-SNAPSHOT~1a5545bead

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1754288453	1754288453
ci_job_id	1061736453	1061736453
ci_pipeline_id	72541390	72541390
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-ppk4pl44 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-ppk4pl44 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~1a5545bead, baseline=1.51.1-SNAPSHOT~15176df7b8
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.433 s) : 15433000, 15433000
.   : milestone, 15433000,
appsec (15.032 s) : 15032000, 15032000
.   : milestone, 15032000,
iast (18.749 s) : 18749000, 18749000
.   : milestone, 18749000,
iast_GLOBAL (18.077 s) : 18077000, 18077000
.   : milestone, 18077000,
profiling (15.589 s) : 15589000, 15589000
.   : milestone, 15589000,
tracing (14.869 s) : 14869000, 14869000
.   : milestone, 14869000,
section candidate
no_agent (15.721 s) : 15721000, 15721000
.   : milestone, 15721000,
appsec (14.666 s) : 14666000, 14666000
.   : milestone, 14666000,
iast (18.195 s) : 18195000, 18195000
.   : milestone, 18195000,
iast_GLOBAL (17.965 s) : 17965000, 17965000
.   : milestone, 17965000,
profiling (15.421 s) : 15421000, 15421000
.   : milestone, 15421000,
tracing (14.858 s) : 14858000, 14858000
.   : milestone, 14858000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.433 s [15.433 s, 15.433 s]	-
appsec	15.032 s [15.032 s, 15.032 s]	-401.0 ms (-2.6%)
iast	18.749 s [18.749 s, 18.749 s]	3.316 s (21.5%)
iast_GLOBAL	18.077 s [18.077 s, 18.077 s]	2.644 s (17.1%)
profiling	15.589 s [15.589 s, 15.589 s]	156.0 ms (1.0%)
tracing	14.869 s [14.869 s, 14.869 s]	-564.0 ms (-3.7%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.721 s [15.721 s, 15.721 s]	-
appsec	14.666 s [14.666 s, 14.666 s]	-1.055 s (-6.7%)
iast	18.195 s [18.195 s, 18.195 s]	2.474 s (15.7%)
iast_GLOBAL	17.965 s [17.965 s, 17.965 s]	2.244 s (14.3%)
profiling	15.421 s [15.421 s, 15.421 s]	-300.0 ms (-1.9%)
tracing	14.858 s [14.858 s, 14.858 s]	-863.0 ms (-5.5%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.51.1-SNAPSHOT~1a5545bead, baseline=1.51.1-SNAPSHOT~15176df7b8
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.475 ms) : 1463, 1486
.   : milestone, 1475,
appsec (3.672 ms) : 3452, 3891
.   : milestone, 3672,
iast (2.206 ms) : 2143, 2269
.   : milestone, 2206,
iast_GLOBAL (2.245 ms) : 2182, 2308
.   : milestone, 2245,
profiling (2.06 ms) : 2008, 2112
.   : milestone, 2060,
tracing (2.027 ms) : 1978, 2076
.   : milestone, 2027,
section candidate
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (3.654 ms) : 3438, 3871
.   : milestone, 3654,
iast (2.206 ms) : 2143, 2269
.   : milestone, 2206,
iast_GLOBAL (2.258 ms) : 2194, 2321
.   : milestone, 2258,
profiling (2.071 ms) : 2018, 2124
.   : milestone, 2071,
tracing (2.018 ms) : 1969, 2067
.   : milestone, 2018,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.475 ms [1.463 ms, 1.486 ms]	-
appsec	3.672 ms [3.452 ms, 3.891 ms]	2.197 ms (149.0%)
iast	2.206 ms [2.143 ms, 2.269 ms]	731.373 µs (49.6%)
iast_GLOBAL	2.245 ms [2.182 ms, 2.308 ms]	770.065 µs (52.2%)
profiling	2.06 ms [2.008 ms, 2.112 ms]	585.341 µs (39.7%)
tracing	2.027 ms [1.978 ms, 2.076 ms]	552.191 µs (37.4%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.48 ms [1.469 ms, 1.492 ms]	-
appsec	3.654 ms [3.438 ms, 3.871 ms]	2.174 ms (146.9%)
iast	2.206 ms [2.143 ms, 2.269 ms]	725.801 µs (49.0%)
iast_GLOBAL	2.258 ms [2.194 ms, 2.321 ms]	777.794 µs (52.6%)
profiling	2.071 ms [2.018 ms, 2.124 ms]	590.602 µs (39.9%)
tracing	2.018 ms [1.969 ms, 2.067 ms]	537.897 µs (36.3%)

dd-java-agent/agent-llmobs/src/main/java/datadog/trace/llmobs/LLMObsSystem.java

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [redis.clients:jedis](https://github.com/redis/jedis) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `6.0.0` -> `6.1.0` | | [com.google.api.grpc:proto-google-common-protos](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.59.2` -> `2.60.0` | | [com.google.cloud:google-cloud-core-http](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.58.2` -> `2.59.0` | | [com.google.cloud:google-cloud-core](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.58.2` -> `2.59.0` | | [com.google.api:gax](https://github.com/googleapis/sdk-platform-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `2.68.2` -> `2.69.0` | | [com.squareup.wire](https://github.com/square/wire) | plugin | misk/gradle/libs.versions.toml | gradle | patch | `5.3.5` -> `5.3.6` | | [com.squareup.wire:wire-schema](https://github.com/square/wire) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `5.3.5` -> `5.3.6` | | [com.squareup.wire:wire-runtime](https://github.com/square/wire) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `5.3.5` -> `5.3.6` | | [com.squareup.wire:wire-reflector](https://github.com/square/wire) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `5.3.5` -> `5.3.6` | | [com.squareup.wire:wire-moshi-adapter](https://github.com/square/wire) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `5.3.5` -> `5.3.6` | | [com.squareup.wire:wire-grpc-client](https://github.com/square/wire) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `5.3.5` -> `5.3.6` | | [com.squareup.wire:wire-bom](https://github.com/square/wire) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `5.3.5` -> `5.3.6` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.51.2` -> `1.52.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.14` -> `2.32.15` | | [software.amazon.awssdk:sqs](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.14` -> `2.32.15` | | [software.amazon.awssdk:regions](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.14` -> `2.32.15` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.14` -> `2.32.15` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.14` -> `2.32.15` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.14` -> `2.32.15` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.14` -> `2.32.15` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.32.14` -> `2.32.15` | --- ### Release Notes <details> <summary>redis/jedis (redis.clients:jedis)</summary> ### [`v6.1.0`](https://github.com/redis/jedis/releases/tag/v6.1.0): 6.1.0 ### Changes #### 🚀 New Features - Add support for SVS-VAMANA vector indexing ([#4222](redis/jedis#4222)) - Clarify why new stream entries aren't deleted with XDELEX ([#4218](redis/jedis#4218)) - Add support for new stream commands ([#4211](redis/jedis#4211)) - Add Support for New BITOP Operations in Redis 8.2 ([#4188](redis/jedis#4188)) ([#4190](redis/jedis#4190)) - Add binary stream support for XREAD and XREADGROUP ([#3566](redis/jedis#3566)) ([#4152](redis/jedis#4152)) - Run pipeline in current thread if all the keys on same node ([#4149](redis/jedis#4149)) #### 🐛 Bug Fixes - Restore binary compatibility of SetParams ([#4225](redis/jedis#4225)) - Fix memory leak in JedisClusterInfoCache - replica nodes not cleared ([#4205](redis/jedis#4205)) - Fix:JedisCluster throws NullPointerException when maxAttempts is set to 0 ([#4186](redis/jedis#4186)) #### 🧰 Maintenance - DOC-5471 time series doc examples ([#4210](redis/jedis#4210)) - Bump jackson.version from 2.19.1 to 2.19.2 ([#4208](redis/jedis#4208)) - Fix flaky test ClientCommandsTest.killSkipmeYesNo ([#4206](redis/jedis#4206)) - Bump org.junit:junit-bom from 5.13.2 to 5.13.3 ([#4198](redis/jedis#4198)) - Migrate publishing to Maven Central Portal ([#4199](redis/jedis#4199)) - Bump org.apache.maven.plugins:maven-gpg-plugin from 3.2.7 to 3.2.8 ([#4197](redis/jedis#4197)) - Bump org.junit:junit-bom from 5.13.1 to 5.13.2 ([#4192](redis/jedis#4192)) - DOC-5227 added probabilistic data type examples ([#4184](redis/jedis#4184)) - Bump jackson.version from 2.19.0 to 2.19.1 ([#4180](redis/jedis#4180)) - Update test infra to use latest Redis ([#4179](redis/jedis#4179)) - Bump org.junit:junit-bom from 5.13.0-RC1 to 5.13.1 ([#4174](redis/jedis#4174)) - Bump org.json:json from [`2025010`](redis/jedis@20250107) to [`2025051`](redis/jedis@20250517) ([#4171](redis/jedis#4171)) - Bump org.apache.httpcomponents.client5:httpclient5-fluent from 5.4.4 to 5.5 ([#4170](redis/jedis#4170)) - Fix flaky tests in DocumentTest ([#3617](redis/jedis#3617)) - Add retryable command execution example ([#3780](redis/jedis#3780)) - Bump jackson.version from 2.18.3 to 2.19.0 ([#4160](redis/jedis#4160)) - Bump com.google.code.gson:gson from 2.12.1 to 2.13.1 ([#4161](redis/jedis#4161)) #### Contributors We'd like to thank all the contributors who worked on this release! [@219sansim](https://github.com/219sansim), [@YoHanKi](https://github.com/YoHanKi), [@andy-stark-redis](https://github.com/andy-stark-redis), [@ggivo](https://github.com/ggivo), [@jujn](https://github.com/jujn), [@thachlp](https://github.com/thachlp), [@uglide](https://github.com/uglide) and [@xrayw](https://github.com/xrayw) </details> <details> <summary>googleapis/sdk-platform-java (com.google.api.grpc:proto-google-common-protos)</summary> ### [`v2.60.0`](https://github.com/googleapis/sdk-platform-java/blob/HEAD/CHANGELOG.md#2600-2025-06-23) ##### Features - handle auto pagination for BigQuery v2 ([#3829](googleapis/sdk-platform-java#3829)) ([025c84c](googleapis/sdk-platform-java@025c84c)) ##### Dependencies - update google auth library dependencies to v1.37.1 ([#3846](googleapis/sdk-platform-java#3846)) ([ea1d9e5](googleapis/sdk-platform-java@ea1d9e5)) - update google http client dependencies to v1.47.1 ([#3848](googleapis/sdk-platform-java#3848)) ([a9a39d7](googleapis/sdk-platform-java@a9a39d7)) </details> <details> <summary>square/wire (com.squareup.wire)</summary> ### [`v5.3.6`](https://github.com/square/wire/blob/HEAD/CHANGELOG.md#Version-536) [Compare Source](square/wire@5.3.5...5.3.6) *2025-08-05* ##### CLI - New CLI option `--ignore_unused_roots_and_prunes` ([#3354](square/wire#3354)) ##### JVM - Fix: Handle negative hexadecimal in default values ([#3355](square/wire#3355)) - Optimization: Avoid copying of repeated and map types when mutableTypes are being used ([#3352](square/wire#3352) by \[Rahul Ravikumar]\[tikurahul]) ##### Swift - Fix: Properly disambiguate OneOf enum if it has the same name as enclosing type ([#3350](square/wire#3350) by \[Dimitris Koutsogiorgas]\[dnkoutso]) </details> <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.52.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.52.0): 1.52.0 ### Components #### Application Security Management (WAF) - ✨ Only report ASM\_DD, ASM\_DATA and ASM capabilities when AppSec is enabled ([#9260](DataDog/dd-trace-java#9260) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - 🐛 Fix NPE in AppSecConfigServiceImpl ([#9165](DataDog/dd-trace-java#9165) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) - 🐛 Fix AppSec play.mvc.StatusHeader instrumentation for play 2.6 ([#9160](DataDog/dd-trace-java#9160) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) #### Build & Tooling - ✨📖 Automatically register crashtracking via native extensions ([#8851](DataDog/dd-trace-java#8851) - [@MattAlp](https://github.com/MattAlp)) #### Configuration at Runtime - ✨ Create activation origin config for telemetry ([#9064](DataDog/dd-trace-java#9064) - [@sezen-datadog](https://github.com/sezen-datadog)) #### Continuous Integration Visibility - ✨ Update GitLab provided tags ([#9275](DataDog/dd-trace-java#9275) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - 🐛 Fix base branch SHA usage in GitHub Actions ([#9257](DataDog/dd-trace-java#9257) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Add `ci.job.id` tag ([#9256](DataDog/dd-trace-java#9256) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Add new org to Weaver instrumentation ([#9235](DataDog/dd-trace-java#9235) - [@daniel-mohedano](https://github.com/daniel-mohedano) - thanks for the contribution!) - ✨ Improve Git commit info building ([#9210](DataDog/dd-trace-java#9210) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Update Attempt to Fix to v5 ([#9145](DataDog/dd-trace-java#9145) - [@daniel-mohedano](https://github.com/daniel-mohedano)) #### Crash tracking - ✨📖 Automatically register crashtracking via native extensions ([#8851](DataDog/dd-trace-java#8851) - [@MattAlp](https://github.com/MattAlp)) #### Data Streams Monitoring - ✨ Reduce DSM CPU overheard ([#9151](DataDog/dd-trace-java#9151) - [@kr-igor](https://github.com/kr-igor)) - ✨⚡ DSM optimizations for high throughput scenarios ([#9137](DataDog/dd-trace-java#9137) - [@kr-igor](https://github.com/kr-igor)) #### Database Monitoring - 🐛 Fix duplicate trace injection for SQL Server and Oracle DBM full propagation mode ([#9224](DataDog/dd-trace-java#9224) - [@lu-zhengda](https://github.com/lu-zhengda)) #### Dynamic Instrumentation - 🐛 Add URI in string primitives ([#9285](DataDog/dd-trace-java#9285) - [@jpbempel](https://github.com/jpbempel)) - ✨ Increase SourceFile tracking max queue size ([#9271](DataDog/dd-trace-java#9271) - [@jpbempel](https://github.com/jpbempel)) - ✨ Add capping on SourceFile tracking queue ([#9245](DataDog/dd-trace-java#9245) - [@jpbempel](https://github.com/jpbempel)) - ✨ Add third-party filtering in SourceFile tracking ([#9205](DataDog/dd-trace-java#9205) - [@jpbempel](https://github.com/jpbempel)) #### ML Observability (LLMObs) - ✨ Add methods to capture embedding and retrieval spans ([#9297](DataDog/dd-trace-java#9297) - [@nayeem-kamal](https://github.com/nayeem-kamal)) #### Metrics - ✨ Change primary client stats configuration key ([#9196](DataDog/dd-trace-java#9196) - [@amarziali](https://github.com/amarziali)) - ✨ Calculate client stats also if the span kind is eligible ([#9157](DataDog/dd-trace-java#9157) - [@amarziali](https://github.com/amarziali)) - ✨ Backpropagate peer tags ([#9144](DataDog/dd-trace-java#9144) - [@bric3](https://github.com/bric3)) - 🐛 Make client stats reliable in case of downgrade ([#9136](DataDog/dd-trace-java#9136) - [@amarziali](https://github.com/amarziali)) #### Platform components - 🐛 Fix VM options parsing from /proc/fs ([#9255](DataDog/dd-trace-java#9255) - [@PerfectSlayer](https://github.com/PerfectSlayer)) #### Profiling - ✨ Switch profile compression to zstd default ([#9293](DataDog/dd-trace-java#9293) - [@jbachorik](https://github.com/jbachorik)) - ✨ Bump ddprof to 1.29.0 ([#9262](DataDog/dd-trace-java#9262) - [@zhengyu123](https://github.com/zhengyu123)) - Potential memory leak and race with the JVMTI wallclock sampler by [@zhengyu123](https://github.com/zhengyu123) in DataDog/java-profiler#234 - Downport async-profiler no-allocation changes by [@zhengyu123](https://github.com/zhengyu123) in DataDog/java-profiler#245 - Adopt openjdk safefetch by [@zhengyu123](https://github.com/zhengyu123) in DataDog/java-profiler#246 - Safe fetch 64-bit value and pointer by [@zhengyu123](https://github.com/zhengyu123) in DataDog/java-profiler#247 - Rebase on Async-Profiler 4.1 by [@jbachorik](https://github.com/jbachorik) in DataDog/java-profiler#252 - Patch upstream stackWalker.cpp not to fail on unaligned access by [@jbachorik](https://github.com/jbachorik) in DataDog/java-profiler#218 - Remap thread id to avoid bitmap contention by [@zhengyu123](https://github.com/zhengyu123) in DataDog/java-profiler#229 - Improve performance using Unsafe to activate/deactivate thread filter by [@zhengyu123](https://github.com/zhengyu123) in DataDog/java-profiler#230 - Unify context propagation by [@zhengyu123](https://github.com/zhengyu123) in DataDog/java-profiler#231 - 🐛 Fix the profiler stackdepth setting propagation in recent (22+) Java versions ([#9130](DataDog/dd-trace-java#9130) - [@jbachorik](https://github.com/jbachorik)) #### Realtime User Monitoring - ✨ Wrap servlet original PrintWriter on rum injector ([#9146](DataDog/dd-trace-java#9146) - [@amarziali](https://github.com/amarziali)) #### Telemetry - ✨ Create activation origin config for telemetry ([#9064](DataDog/dd-trace-java#9064) - [@sezen-datadog](https://github.com/sezen-datadog)) #### Tracer core - ✨⚡ Reduce span construction overhead by switching to optimized TagMap ([#8589](DataDog/dd-trace-java#8589) - [@dougqh](https://github.com/dougqh)) - 🐛 Match Hands Off Config selectors on process\_arguments value ([#9201](DataDog/dd-trace-java#9201) - [@paullegranddc](https://github.com/paullegranddc)) - ✨ Move JSON generation to sender thread to improve startup time. ([#9197](DataDog/dd-trace-java#9197) - [@AlexeyKuznetsov-DD](https://github.com/AlexeyKuznetsov-DD)) - ✨ Improve agent to avoid loading global config on main thread ([#9190](DataDog/dd-trace-java#9190) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨ add injection metadata fields to telemetry forwarder ([#9185](DataDog/dd-trace-java#9185) - [@sydney-tung](https://github.com/sydney-tung)) - 🐛 Avoid race conditions on feature discovery during Writer creation ([#9173](DataDog/dd-trace-java#9173) - [@daniel-mohedano](https://github.com/daniel-mohedano)) - ✨ Surface potential root cause when agent initialization errors ([#9170](DataDog/dd-trace-java#9170) - [@AlexeyKuznetsov-DD](https://github.com/AlexeyKuznetsov-DD)) - 💡 Support adding W3C baggage as span tags ([#9169](DataDog/dd-trace-java#9169) - [@rachelyangdog](https://github.com/rachelyangdog)) - ✨⚡ Align our default classloader excludes with OTel ([#9161](DataDog/dd-trace-java#9161) - [@mcculls](https://github.com/mcculls)) - ✨ Backpropagate container tags hash coming from the info endpoint ([#9156](DataDog/dd-trace-java#9156) - [@amarziali](https://github.com/amarziali)) - 🐛 Avoid race conditions and multiple agent discovery feature states ([#9135](DataDog/dd-trace-java#9135) - [@amarziali](https://github.com/amarziali)) ### Instrumentations #### AWS SDK instrumentation - ✨ Enhance Service Representation for Serverless ([#9203](DataDog/dd-trace-java#9203) - [@zarirhamza](https://github.com/zarirhamza)) #### gRPC instrumentation - 🐛 Add check to prevent injection of repeated GRPC headers ([#9246](DataDog/dd-trace-java#9246) - [@mhlidd](https://github.com/mhlidd)) #### JDBC instrumentation - 🐛 Fix duplicate trace injection for SQL Server and Oracle DBM full propagation mode ([#9224](DataDog/dd-trace-java#9224) - [@lu-zhengda](https://github.com/lu-zhengda)) #### Play Framework instrumentation - 🐛 Fix AppSec play.mvc.StatusHeader instrumentation for play 2.6 ([#9160](DataDog/dd-trace-java#9160) - [@manuel-alvarez-alvarez](https://github.com/manuel-alvarez-alvarez)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: 9da36329ffa552291a640381780b608ef6513e29

added embedding and retrieval spans

1a5545b

nayeem-kamal requested review from a team as code owners August 4, 2025 05:49

nayeem-kamal requested a review from dougqh August 4, 2025 05:49

mcculls added type: enhancement Enhancements and improvements inst: others All other instrumentations tag: no release notes Changes to exclude from release notes and removed tag: no release notes Changes to exclude from release notes labels Aug 4, 2025

mcculls requested a review from gary-huang August 4, 2025 08:45

mcculls added comp: mlobs ML Observability (LLMObs) and removed inst: others All other instrumentations labels Aug 4, 2025

mcculls changed the title ~~Implement embedding and retrieval spans~~ Add methods to capture embedding and retrieval spans Aug 4, 2025

mcculls added the comp: api Tracer public API label Aug 4, 2025

gary-huang approved these changes Aug 4, 2025

View reviewed changes

AlexeyKuznetsov-DD requested changes Aug 4, 2025

View reviewed changes

dd-java-agent/agent-llmobs/src/main/java/datadog/trace/llmobs/LLMObsSystem.java Show resolved Hide resolved

AlexeyKuznetsov-DD approved these changes Aug 4, 2025

View reviewed changes

nayeem-kamal merged commit d7684b9 into master Aug 4, 2025
509 of 511 checks passed

nayeem-kamal deleted the nayeem-kamal/embeddings-retrievals branch August 4, 2025 14:29

github-actions bot added this to the 1.52.0 milestone Aug 4, 2025

ClintAtEsper mentioned this pull request Aug 15, 2025

LLMObs retrieval and embedding span creation methods are not static #9372

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add methods to capture embedding and retrieval spans #9297

Add methods to capture embedding and retrieval spans #9297

Uh oh!

nayeem-kamal commented Aug 4, 2025

Uh oh!

github-actions bot commented Aug 4, 2025

Uh oh!

datadog-datadog-prod-us1 bot commented Aug 4, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Aug 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Add methods to capture embedding and retrieval spans #9297

Add methods to capture embedding and retrieval spans #9297

Uh oh!

Conversation

nayeem-kamal commented Aug 4, 2025

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

github-actions bot commented Aug 4, 2025

Uh oh!

datadog-datadog-prod-us1 bot commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Aug 4, 2025

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

datadog-datadog-prod-us1 bot commented Aug 4, 2025 •

edited

Loading