ConfigProvider iterates over all sources and reports all non-null values to telemetry #9404

mtoffl01 · 2025-08-21T18:55:59Z

What Does This Do

Previously, telemetry was only being reported for the ConfigSource that set the final value of the configuration. This PR introduces reporting telemetry for all ConfigSources that have non-null values set, regardless of whether it sets the final configuration value or not.

Summary of Changes:

ConfigCollector
- Stores config telemetry payloads of Map<ConfigOrigin, Map<String (configName), ConfigSetting>>
- Deprecates old put method in favor of a new put method with seq_id field
- New putDefault method is created to ensure originally reported defaults are not overwritten. Use cases:
  - ConfigProvider methods getEnum, getIntegerRange, getSet and getList invoke getString, which erroneously reports defaults as null under the hood
  - when a Config is already processed by the ConfigProvider, then a function like isExplicitlyDisabled is called and invokes getBoolean without the proper default value, which previously would override the DEFAULT telemetry payload with null.
ConfigSetting
- New field seqId and new of function that takes it in as a parameter
ConfigProvider
- All methods querying ConfigSources start from lowest priority to highest priority. A local seqId is used to track priority of ConfigSources that are set. In the case of parsing errors, there may be re-reporting of the last valid ConfigSource to ensure the highest priority and valid ConfigSource is reported with the highest seqId
- ConfigValueResolver
  - Helper class to help with re-reporting configurations. Includes key, value, seqId, and ConfigOrigin.
  - All ConfigProvider methods now store the final return value through ConfigValueResolver
- ConfigMergeResolver
  - Helper class to help with merging maps from multiple sources

Motivation

RFC

Additional Notes

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: [PROJ-IDENT]

datadog-datadog-prod-us1 · 2025-08-21T19:18:49Z

🎯 Code Coverage
• Patch Coverage: 63.64%
• Total Coverage: 59.64% (-0.03%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 7508fc7 | Docs | Was this helpful? Give us feedback!}

pr-commenter · 2025-08-21T19:29:20Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mtoff/4-config-sources
git_commit_date	1758557794	1758557773
git_commit_sha	`cbe8eb4`	`7508fc7`
release_version	1.54.0-SNAPSHOT~cbe8eb4cf4	1.53.0-SNAPSHOT~7508fc78b6

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1758559569	1758559569
ci_job_id	1141399072	1141399072
ci_pipeline_id	77183783	77183783
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-q4bh7lw9 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-q4bh7lw9 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 1 performance improvements and 11 performance regressions! Performance is the same for 37 metrics, 10 unstable metrics.

scenario	Δ mean execution_time	candidate mean execution_time	baseline mean execution_time
scenario:startup:insecure-bank:iast:Agent.start	worse [+31.550ms; +44.189ms] or [+2.727%; +3.820%]	1.195s	1.157s
scenario:startup:insecure-bank:iast:BytebuddyAgent	worse [+40.426ms; +49.435ms] or [+4.986%; +6.097%]	855.683ms	810.752ms
scenario:startup:insecure-bank:tracing:Agent.start	worse [+25.145ms; +53.577ms] or [+2.453%; +5.226%]	1.065s	1.025s
scenario:startup:insecure-bank:tracing:BytebuddyAgent	worse [+34.200ms; +53.594ms] or [+4.951%; +7.759%]	734.637ms	690.740ms
scenario:startup:insecure-bank:tracing:AppSec	better [-1.937ms; -0.874ms] or [-6.059%; -2.733%]	30.563ms	31.968ms
scenario:startup:petclinic:appsec:Agent.start	worse [+32.860ms; +43.436ms] or [+2.738%; +3.619%]	1.238s	1.200s
scenario:startup:petclinic:appsec:BytebuddyAgent	worse [+41.560ms; +48.697ms] or [+5.830%; +6.831%]	758.016ms	712.887ms
scenario:startup:petclinic:iast:Agent.start	worse [+30.595ms; +51.330ms] or [+2.646%; +4.439%]	1.197s	1.156s
scenario:startup:petclinic:iast:BytebuddyAgent	worse [+38.935ms; +60.561ms] or [+4.808%; +7.479%]	859.504ms	809.757ms
scenario:startup:petclinic:profiling:BytebuddyAgent	worse [+18.046ms; +43.891ms] or [+2.467%; +6.000%]	762.454ms	731.485ms
scenario:startup:petclinic:tracing:Agent.start	worse [+28.114ms; +67.858ms] or [+2.735%; +6.601%]	1.076s	1.028s
scenario:startup:petclinic:tracing:BytebuddyAgent	worse [+36.537ms; +65.792ms] or [+5.275%; +9.499%]	743.786ms	692.622ms

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~7508fc78b6, baseline=1.54.0-SNAPSHOT~cbe8eb4cf4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.025 s) : 0, 1025234
Total [baseline] (8.688 s) : 0, 8688070
Agent [candidate] (1.065 s) : 0, 1064596
Total [candidate] (8.615 s) : 0, 8615385
section iast
Agent [baseline] (1.157 s) : 0, 1156847
Total [baseline] (9.352 s) : 0, 9351874
Agent [candidate] (1.195 s) : 0, 1194717
Total [candidate] (9.29 s) : 0, 9290445

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.025 s	-
Agent	iast	1.157 s	131.613 ms (12.8%)
Total	tracing	8.688 s	-
Total	iast	9.352 s	663.804 ms (7.6%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.065 s	-
Agent	iast	1.195 s	130.121 ms (12.2%)
Total	tracing	8.615 s	-
Total	iast	9.29 s	675.06 ms (7.8%)

gantt
    title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~7508fc78b6, baseline=1.54.0-SNAPSHOT~cbe8eb4cf4

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.47 ms) : 0, 1470
crashtracking [candidate] (1.456 ms) : 0, 1456
BytebuddyAgent [baseline] (690.74 ms) : 0, 690740
BytebuddyAgent [candidate] (734.637 ms) : 0, 734637
GlobalTracer [baseline] (259.556 ms) : 0, 259556
GlobalTracer [candidate] (253.534 ms) : 0, 253534
AppSec [baseline] (31.968 ms) : 0, 31968
AppSec [candidate] (30.563 ms) : 0, 30563
Debugger [baseline] (6.377 ms) : 0, 6377
Debugger [candidate] (6.346 ms) : 0, 6346
Remote Config [baseline] (683.903 µs) : 0, 684
Remote Config [candidate] (677.32 µs) : 0, 677
Telemetry [baseline] (13.383 ms) : 0, 13383
Telemetry [candidate] (16.368 ms) : 0, 16368
section iast
crashtracking [baseline] (1.468 ms) : 0, 1468
crashtracking [candidate] (1.445 ms) : 0, 1445
BytebuddyAgent [baseline] (810.752 ms) : 0, 810752
BytebuddyAgent [candidate] (855.683 ms) : 0, 855683
GlobalTracer [baseline] (249.879 ms) : 0, 249879
GlobalTracer [candidate] (244.87 ms) : 0, 244870
AppSec [baseline] (27.923 ms) : 0, 27923
AppSec [candidate] (27.25 ms) : 0, 27250
Debugger [baseline] (6.124 ms) : 0, 6124
Debugger [candidate] (6.026 ms) : 0, 6026
Remote Config [baseline] (598.176 µs) : 0, 598
Remote Config [candidate] (608.659 µs) : 0, 609
Telemetry [baseline] (8.176 ms) : 0, 8176
Telemetry [candidate] (8.292 ms) : 0, 8292
IAST [baseline] (30.902 ms) : 0, 30902
IAST [candidate] (29.665 ms) : 0, 29665

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~7508fc78b6, baseline=1.54.0-SNAPSHOT~cbe8eb4cf4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.028 s) : 0, 1028051
Total [baseline] (10.703 s) : 0, 10703000
Agent [candidate] (1.076 s) : 0, 1076037
Total [candidate] (10.796 s) : 0, 10795600
section appsec
Agent [baseline] (1.2 s) : 0, 1200320
Total [baseline] (11.082 s) : 0, 11082365
Agent [candidate] (1.238 s) : 0, 1238467
Total [candidate] (11.085 s) : 0, 11085279
section iast
Agent [baseline] (1.156 s) : 0, 1156457
Total [baseline] (11.085 s) : 0, 11085460
Agent [candidate] (1.197 s) : 0, 1197420
Total [candidate] (11.019 s) : 0, 11018710
section profiling
Agent [baseline] (1.184 s) : 0, 1184451
Total [baseline] (11.146 s) : 0, 11145695
Agent [candidate] (1.211 s) : 0, 1210679
Total [candidate] (10.933 s) : 0, 10933023

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.028 s	-
Agent	appsec	1.2 s	172.269 ms (16.8%)
Agent	iast	1.156 s	128.406 ms (12.5%)
Agent	profiling	1.184 s	156.4 ms (15.2%)
Total	tracing	10.703 s	-
Total	appsec	11.082 s	379.364 ms (3.5%)
Total	iast	11.085 s	382.46 ms (3.6%)
Total	profiling	11.146 s	442.695 ms (4.1%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.076 s	-
Agent	appsec	1.238 s	162.431 ms (15.1%)
Agent	iast	1.197 s	121.383 ms (11.3%)
Agent	profiling	1.211 s	134.643 ms (12.5%)
Total	tracing	10.796 s	-
Total	appsec	11.085 s	289.68 ms (2.7%)
Total	iast	11.019 s	223.11 ms (2.1%)
Total	profiling	10.933 s	137.423 ms (1.3%)

gantt
    title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~7508fc78b6, baseline=1.54.0-SNAPSHOT~cbe8eb4cf4

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.466 ms) : 0, 1466
crashtracking [candidate] (1.457 ms) : 0, 1457
BytebuddyAgent [baseline] (692.622 ms) : 0, 692622
BytebuddyAgent [candidate] (743.786 ms) : 0, 743786
GlobalTracer [baseline] (260.9 ms) : 0, 260900
GlobalTracer [candidate] (255.184 ms) : 0, 255184
AppSec [baseline] (32.024 ms) : 0, 32024
AppSec [candidate] (30.851 ms) : 0, 30851
Debugger [baseline] (6.399 ms) : 0, 6399
Debugger [candidate] (6.402 ms) : 0, 6402
Remote Config [baseline] (690.275 µs) : 0, 690
Remote Config [candidate] (684.171 µs) : 0, 684
Telemetry [baseline] (12.822 ms) : 0, 12822
Telemetry [candidate] (16.449 ms) : 0, 16449
section appsec
crashtracking [baseline] (1.459 ms) : 0, 1459
crashtracking [candidate] (1.45 ms) : 0, 1450
BytebuddyAgent [baseline] (712.887 ms) : 0, 712887
BytebuddyAgent [candidate] (758.016 ms) : 0, 758016
GlobalTracer [baseline] (251.835 ms) : 0, 251835
GlobalTracer [candidate] (246.133 ms) : 0, 246133
AppSec [baseline] (171.343 ms) : 0, 171343
AppSec [candidate] (172.431 ms) : 0, 172431
Debugger [baseline] (6.066 ms) : 0, 6066
Debugger [candidate] (6.004 ms) : 0, 6004
Remote Config [baseline] (628.618 µs) : 0, 629
Remote Config [candidate] (636.995 µs) : 0, 637
Telemetry [baseline] (9.961 ms) : 0, 9961
Telemetry [candidate] (8.568 ms) : 0, 8568
IAST [baseline] (25.025 ms) : 0, 25025
IAST [candidate] (24.099 ms) : 0, 24099
section iast
crashtracking [baseline] (1.47 ms) : 0, 1470
crashtracking [candidate] (1.477 ms) : 0, 1477
BytebuddyAgent [baseline] (809.757 ms) : 0, 809757
BytebuddyAgent [candidate] (859.504 ms) : 0, 859504
GlobalTracer [baseline] (250.02 ms) : 0, 250020
GlobalTracer [candidate] (244.786 ms) : 0, 244786
AppSec [baseline] (29.365 ms) : 0, 29365
AppSec [candidate] (26.236 ms) : 0, 26236
Debugger [baseline] (6.106 ms) : 0, 6106
Debugger [candidate] (6.029 ms) : 0, 6029
Remote Config [baseline] (603.613 µs) : 0, 604
Remote Config [candidate] (618.877 µs) : 0, 619
Telemetry [baseline] (8.043 ms) : 0, 8043
Telemetry [candidate] (8.277 ms) : 0, 8277
IAST [baseline] (29.988 ms) : 0, 29988
IAST [candidate] (29.453 ms) : 0, 29453
section profiling
crashtracking [baseline] (1.458 ms) : 0, 1458
crashtracking [candidate] (1.442 ms) : 0, 1442
BytebuddyAgent [baseline] (731.485 ms) : 0, 731485
BytebuddyAgent [candidate] (762.454 ms) : 0, 762454
GlobalTracer [baseline] (240.141 ms) : 0, 240141
GlobalTracer [candidate] (233.04 ms) : 0, 233040
AppSec [baseline] (31.647 ms) : 0, 31647
AppSec [candidate] (30.422 ms) : 0, 30422
Debugger [baseline] (6.59 ms) : 0, 6590
Debugger [candidate] (11.937 ms) : 0, 11937
Remote Config [baseline] (716.198 µs) : 0, 716
Remote Config [candidate] (717.558 µs) : 0, 718
Telemetry [baseline] (16.95 ms) : 0, 16950
Telemetry [candidate] (11.08 ms) : 0, 11080
ProfilingAgent [baseline] (103.74 ms) : 0, 103740
ProfilingAgent [candidate] (107.313 ms) : 0, 107313
Profiling [baseline] (104.355 ms) : 0, 104355
Profiling [candidate] (107.949 ms) : 0, 107949

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mtoff/4-config-sources
git_commit_date	1758557794	1758557773
git_commit_sha	`cbe8eb4`	`7508fc7`
release_version	1.54.0-SNAPSHOT~cbe8eb4cf4	1.53.0-SNAPSHOT~7508fc78b6

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1758559312	1758559312
ci_job_id	1141399073	1141399073
ci_pipeline_id	77183783	77183783
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-qf0779rn 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-qf0779rn 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 2 performance improvements and 3 performance regressions! Performance is the same for 7 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:no_agent:high_load	worse [+265.716µs; +382.162µs] or [+6.330%; +9.104%]	unstable [-200.732op/s; +43.607op/s] or [-18.338%; +3.984%]	4.521ms	1016.062op/s	4.198ms	1094.625op/s
scenario:load:insecure-bank:iast_FULL:high_load	better [-1302.691µs; -684.601µs] or [-8.653%; -4.547%]	unstable [-15.850op/s; +58.788op/s] or [-5.116%; +18.973%]	14.061ms	331.312op/s	15.055ms	309.844op/s
scenario:load:petclinic:appsec:high_load	better [-5.791ms; -4.842ms] or [-11.080%; -9.265%]	unstable [+3.419op/s; +16.781op/s] or [+3.815%; +18.729%]	46.947ms	99.700op/s	52.263ms	89.600op/s
scenario:load:petclinic:iast:high_load	worse [+1.184ms; +2.029ms] or [+2.689%; +4.608%]	unstable [-11.423op/s; +3.973op/s] or [-10.751%; +3.739%]	45.650ms	102.525op/s	44.043ms	106.250op/s
scenario:load:petclinic:profiling:high_load	worse [+3.119ms; +4.100ms] or [+6.488%; +8.528%]	unstable [-14.456op/s; -1.534op/s] or [-14.660%; -1.556%]	51.680ms	90.612op/s	48.071ms	98.608op/s

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~7508fc78b6, baseline=1.54.0-SNAPSHOT~cbe8eb4cf4
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.419 ms) : 37118, 37721
.   : milestone, 37419,
appsec (52.263 ms) : 51789, 52737
.   : milestone, 52263,
code_origins (44.731 ms) : 44358, 45105
.   : milestone, 44731,
iast (44.043 ms) : 43665, 44421
.   : milestone, 44043,
profiling (48.071 ms) : 47622, 48520
.   : milestone, 48071,
tracing (43.075 ms) : 42715, 43434
.   : milestone, 43075,
section candidate
no_agent (38.309 ms) : 37997, 38621
.   : milestone, 38309,
appsec (46.947 ms) : 46542, 47352
.   : milestone, 46947,
code_origins (45.12 ms) : 44733, 45507
.   : milestone, 45120,
iast (45.65 ms) : 45243, 46057
.   : milestone, 45650,
profiling (51.68 ms) : 51217, 52143
.   : milestone, 51680,
tracing (43.071 ms) : 42715, 43428
.   : milestone, 43071,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.419 ms [37.118 ms, 37.721 ms]	-
appsec	52.263 ms [51.789 ms, 52.737 ms]	14.844 ms (39.7%)
code_origins	44.731 ms [44.358 ms, 45.105 ms]	7.312 ms (19.5%)
iast	44.043 ms [43.665 ms, 44.421 ms]	6.624 ms (17.7%)
profiling	48.071 ms [47.622 ms, 48.52 ms]	10.651 ms (28.5%)
tracing	43.075 ms [42.715 ms, 43.434 ms]	5.656 ms (15.1%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	38.309 ms [37.997 ms, 38.621 ms]	-
appsec	46.947 ms [46.542 ms, 47.352 ms]	8.638 ms (22.5%)
code_origins	45.12 ms [44.733 ms, 45.507 ms]	6.811 ms (17.8%)
iast	45.65 ms [45.243 ms, 46.057 ms]	7.341 ms (19.2%)
profiling	51.68 ms [51.217 ms, 52.143 ms]	13.371 ms (34.9%)
tracing	43.071 ms [42.715 ms, 43.428 ms]	4.763 ms (12.4%)

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~7508fc78b6, baseline=1.54.0-SNAPSHOT~cbe8eb4cf4
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.198 ms) : 4141, 4254
.   : milestone, 4198,
iast (9.664 ms) : 9501, 9828
.   : milestone, 9664,
iast_FULL (15.055 ms) : 14756, 15354
.   : milestone, 15055,
iast_GLOBAL (10.435 ms) : 10252, 10617
.   : milestone, 10435,
profiling (8.846 ms) : 8700, 8993
.   : milestone, 8846,
tracing (7.808 ms) : 7696, 7921
.   : milestone, 7808,
section candidate
no_agent (4.521 ms) : 4470, 4573
.   : milestone, 4521,
iast (9.731 ms) : 9569, 9894
.   : milestone, 9731,
iast_FULL (14.061 ms) : 13786, 14336
.   : milestone, 14061,
iast_GLOBAL (10.12 ms) : 9941, 10299
.   : milestone, 10120,
profiling (8.871 ms) : 8729, 9013
.   : milestone, 8871,
tracing (7.69 ms) : 7572, 7808
.   : milestone, 7690,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.198 ms [4.141 ms, 4.254 ms]	-
iast	9.664 ms [9.501 ms, 9.828 ms]	5.467 ms (130.2%)
iast_FULL	15.055 ms [14.756 ms, 15.354 ms]	10.857 ms (258.7%)
iast_GLOBAL	10.435 ms [10.252 ms, 10.617 ms]	6.237 ms (148.6%)
profiling	8.846 ms [8.7 ms, 8.993 ms]	4.649 ms (110.8%)
tracing	7.808 ms [7.696 ms, 7.921 ms]	3.611 ms (86.0%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.521 ms [4.47 ms, 4.573 ms]	-
iast	9.731 ms [9.569 ms, 9.894 ms]	5.21 ms (115.2%)
iast_FULL	14.061 ms [13.786 ms, 14.336 ms]	9.54 ms (211.0%)
iast_GLOBAL	10.12 ms [9.941 ms, 10.299 ms]	5.599 ms (123.8%)
profiling	8.871 ms [8.729 ms, 9.013 ms]	4.35 ms (96.2%)
tracing	7.69 ms [7.572 ms, 7.808 ms]	3.169 ms (70.1%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	mtoff/4-config-sources
git_commit_date	1758557794	1758557773
git_commit_sha	`cbe8eb4`	`7508fc7`
release_version	1.54.0-SNAPSHOT~cbe8eb4cf4	1.53.0-SNAPSHOT~7508fc78b6

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1758559809	1758559809
ci_job_id	1141399074	1141399074
ci_pipeline_id	77183783	77183783
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-ejaobwr3 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-ejaobwr3 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~7508fc78b6, baseline=1.54.0-SNAPSHOT~cbe8eb4cf4
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.307 s) : 15307000, 15307000
.   : milestone, 15307000,
appsec (15.243 s) : 15243000, 15243000
.   : milestone, 15243000,
iast (18.656 s) : 18656000, 18656000
.   : milestone, 18656000,
iast_GLOBAL (18.097 s) : 18097000, 18097000
.   : milestone, 18097000,
profiling (16.006 s) : 16006000, 16006000
.   : milestone, 16006000,
tracing (14.999 s) : 14999000, 14999000
.   : milestone, 14999000,
section candidate
no_agent (14.738 s) : 14738000, 14738000
.   : milestone, 14738000,
appsec (14.735 s) : 14735000, 14735000
.   : milestone, 14735000,
iast (18.402 s) : 18402000, 18402000
.   : milestone, 18402000,
iast_GLOBAL (18.224 s) : 18224000, 18224000
.   : milestone, 18224000,
profiling (15.415 s) : 15415000, 15415000
.   : milestone, 15415000,
tracing (15.021 s) : 15021000, 15021000
.   : milestone, 15021000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.307 s [15.307 s, 15.307 s]	-
appsec	15.243 s [15.243 s, 15.243 s]	-64.0 ms (-0.4%)
iast	18.656 s [18.656 s, 18.656 s]	3.349 s (21.9%)
iast_GLOBAL	18.097 s [18.097 s, 18.097 s]	2.79 s (18.2%)
profiling	16.006 s [16.006 s, 16.006 s]	699.0 ms (4.6%)
tracing	14.999 s [14.999 s, 14.999 s]	-308.0 ms (-2.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.738 s [14.738 s, 14.738 s]	-
appsec	14.735 s [14.735 s, 14.735 s]	-3.0 ms (-0.0%)
iast	18.402 s [18.402 s, 18.402 s]	3.664 s (24.9%)
iast_GLOBAL	18.224 s [18.224 s, 18.224 s]	3.486 s (23.7%)
profiling	15.415 s [15.415 s, 15.415 s]	677.0 ms (4.6%)
tracing	15.021 s [15.021 s, 15.021 s]	283.0 ms (1.9%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~7508fc78b6, baseline=1.54.0-SNAPSHOT~cbe8eb4cf4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.488 ms) : 1477, 1500
.   : milestone, 1488,
appsec (3.703 ms) : 3488, 3917
.   : milestone, 3703,
iast (2.218 ms) : 2155, 2281
.   : milestone, 2218,
iast_GLOBAL (2.262 ms) : 2199, 2325
.   : milestone, 2262,
profiling (2.06 ms) : 2009, 2110
.   : milestone, 2060,
tracing (2.041 ms) : 1991, 2090
.   : milestone, 2041,
section candidate
no_agent (1.482 ms) : 1471, 1494
.   : milestone, 1482,
appsec (3.717 ms) : 3498, 3935
.   : milestone, 3717,
iast (2.207 ms) : 2144, 2270
.   : milestone, 2207,
iast_GLOBAL (2.268 ms) : 2205, 2332
.   : milestone, 2268,
profiling (2.072 ms) : 2020, 2124
.   : milestone, 2072,
tracing (2.052 ms) : 2002, 2102
.   : milestone, 2052,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.488 ms [1.477 ms, 1.5 ms]	-
appsec	3.703 ms [3.488 ms, 3.917 ms]	2.215 ms (148.8%)
iast	2.218 ms [2.155 ms, 2.281 ms]	729.51 µs (49.0%)
iast_GLOBAL	2.262 ms [2.199 ms, 2.325 ms]	773.429 µs (52.0%)
profiling	2.06 ms [2.009 ms, 2.11 ms]	571.406 µs (38.4%)
tracing	2.041 ms [1.991 ms, 2.09 ms]	552.529 µs (37.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.482 ms [1.471 ms, 1.494 ms]	-
appsec	3.717 ms [3.498 ms, 3.935 ms]	2.234 ms (150.7%)
iast	2.207 ms [2.144 ms, 2.27 ms]	724.969 µs (48.9%)
iast_GLOBAL	2.268 ms [2.205 ms, 2.332 ms]	786.051 µs (53.0%)
profiling	2.072 ms [2.02 ms, 2.124 ms]	589.621 µs (39.8%)
tracing	2.052 ms [2.002 ms, 2.102 ms]	569.44 µs (38.4%)

…x configId test to accommodate new ConfigCollector data structure

…Source'

…edConfigSetting static function that takes in a map

…rtFinalResult

mhlidd · 2025-08-29T20:15:14Z

internal-api/src/main/java/datadog/trace/bootstrap/config/provider/ConfigProvider.java

+  }
+
+  /** Helper class to store resolved configuration values with their metadata */
+  static class ConfigValueResolver<T> {


This class is not private in order to expose it for a unit test that is needed to complete code coverage. The unit test verifies that reReportToCollector is not invoked when origin == null, which should never happen since the method to create a ConfigValueResolver should only occur when no re-reporting is expected. This check is only here as a safety check against poor usage of the function.

telemetry/src/test/groovy/datadog/telemetry/TelemetryServiceSpecification.groovy

mtoffl01 · 2025-09-17T19:07:18Z

utils/config-utils/src/main/java/datadog/trace/api/ConfigSetting.java

+  // No usages of this function
  public static ConfigSetting of(String key, Object value, ConfigOrigin origin, String configId) {


should we mark as deprecated?

If this method is not used anymore then I would remove it (this is internal code, so not covered by breaking changes)

…sources

mcculls · 2025-09-22T14:32:50Z

dd-java-agent/appsec/src/main/java/com/datadog/appsec/config/AppSecConfigServiceImpl.java

      newState = asm.enabled;
      // Report AppSec activation change via telemetry when modified via remote config
-      ConfigCollector.get().put(APPSEC_ENABLED, asm.enabled, ConfigOrigin.REMOTE);
+      ConfigCollector.get().put(APPSEC_ENABLED, asm.enabled, ConfigOrigin.REMOTE, ABSENT_SEQ_ID);


What do you think about adding an update method to ConfigCollector ? or maybe remotePut ?

The new method would just take a key and new value - the origin would always be ConfigOrigin.REMOTE and the sequence id would be absent.

This simplifies remote-config calls to ConfigCollector and also provides a clear code path for such calls...

mcculls · 2025-09-22T14:34:35Z

dd-smoke-tests/log-injection/src/main/java/datadog/smoketest/loginjection/BaseApplication.java

-    ConfigSetting configSetting = ConfigCollector.get().collect().get(LOGS_INJECTION_ENABLED);
+    ConfigSetting configSetting =
+        ConfigCollector.getAppliedConfigSetting(
+            LOGS_INJECTION_ENABLED, ConfigCollector.get().collect());


I'm looking into ways to remove this call, as smoke tests shouldn't be relying on internal-api

mcculls · 2025-09-22T14:36:11Z

internal-api/src/main/java/datadog/trace/api/Config.java

    if (value != null) {
-      ConfigCollector.get().put(name, value, ConfigOrigin.ENV);
+      // Report non-default sequence id for consistency
+      ConfigCollector.get().put(name, value, ConfigOrigin.ENV, DEFAULT_SEQ_ID + 1);


Suggestion: add a constant NON_DEFAULT_SEQ_ID which equals DEFAULT_SEQ_ID + 1

mcculls · 2025-09-22T14:37:01Z

internal-api/src/main/java/datadog/trace/api/Config.java

    if (value != null) {
-      ConfigCollector.get().put(name, value, ConfigOrigin.JVM_PROP);
+      // Report non-default sequence id for consistency
+      ConfigCollector.get().put(name, value, ConfigOrigin.JVM_PROP, DEFAULT_SEQ_ID + 1);


Replace with NON_DEFAULT_SEQ_ID ? (see suggestion above)

mcculls · 2025-09-22T14:38:43Z

utils/config-utils/src/main/java/datadog/trace/api/ConfigSetting.java

+
+  public final int seqId;
+  public static final int DEFAULT_SEQ_ID = 1;
+  public static final int ABSENT_SEQ_ID = 0;


Minor: I usually put constants at the top of the class, then fields, then methods

mcculls · 2025-09-22T14:42:55Z

utils/config-utils/src/main/java/datadog/trace/api/ConfigCollector.java

  }
+
+  // NOTE: Only used to preserve legacy behavior for with smoke tests
+  public static ConfigSetting getAppliedConfigSetting(


I'm looking into ways to remove this :)

…igin; migrate AppSecConfigServiceImpl to use this API

… calls to 'DEFAULT_SEQ_ID + 1' to use new constant; move constants to top of ConfigSetting class

internal-api/src/test/groovy/datadog/trace/api/ConfigCollectorTest.groovy

mcculls · 2025-09-22T15:12:20Z

internal-api/src/test/groovy/datadog/trace/api/ConfigCollectorTest.groovy

  def "non-default config settings get collected"() {
    setup:
    injectEnvConfig(ConfigStrings.toEnvVar(configKey), configValue)
+    def origin = ConfigOrigin.ENV


I would just inline the use of this below (afaict it's only used on one line)

mcculls · 2025-09-22T15:21:09Z

utils/config-utils/src/main/java/datadog/trace/api/ConfigCollector.java

+   *     ordering.
+   */
+  @Deprecated
  public void put(String key, Object value, ConfigOrigin origin) {


Can we migrate any remaining use onto the future-proof method and then remove this deprecated method? (since this is all internal api)

mcculls · 2025-09-22T15:25:12Z

utils/config-utils/src/main/java/datadog/trace/api/ConfigCollector.java

+  public void put(String key, Object value, ConfigOrigin origin, int seqId, String configId) {
+    ConfigSetting setting = ConfigSetting.of(key, value, origin, seqId, configId);
+    Map<String, ConfigSetting> configMap =
+        collected.computeIfAbsent(origin, k -> new ConcurrentHashMap<>());


Consider storing this as a static field, i.e.

private static final Function< ConfigOrigin, Map<String, ConfigSetting>> NEW_SUB_MAP = k -> new ConcurrentHashMap<>();

which you can then re-use here and in putDefault below.

mcculls · 2025-09-22T15:30:03Z

utils/config-utils/src/main/java/datadog/trace/api/ConfigCollector.java

+   * @param configMap map of configuration key-value pairs to add
+   * @param origin the configuration origin for all entries
+   */
+  public void putAll(Map<String, Object> configMap, ConfigOrigin origin) {


This is only used by DynamicConfig to report remote config updates - so I would change this to be

Suggested change

public void putAll(Map<String, Object> configMap, ConfigOrigin origin) {

public void putRemote(Map<String, Object> configMap) {

and use ConfigOrigin.REMOTE to get the right sub-map to update.

mcculls · 2025-09-22T15:33:05Z

utils/config-utils/src/main/java/datadog/trace/api/ConfigCollector.java

+   */
+  public void putAll(Map<String, Object> configMap, ConfigOrigin origin) {
+    for (Map.Entry<String, Object> entry : configMap.entrySet()) {
+      put(entry.getKey(), entry.getValue(), origin, ABSENT_SEQ_ID, null);


This breaks the atomicity of the previous implementation - i.e. with the new code if a remote config update happens at the same time the config is collected for telemetry then half of the remote config update could land in the outgoing batch and the rest would land in the subsequent batch. This could result in inconsistent data being shown in the UI.

I'll add an update to show how the atomicity can be kept with the new sub-map approach.

…Test.groovy Co-authored-by: Stuart McCulloch <[email protected]>

…gCollector, used by putDefault and put

…java into mtoff/4-config-sources

apply all changes from PR #9327, accomodated to master

08c80b4

mtoffl01 mentioned this pull request Aug 21, 2025

ConfigProvider iterates over all sources and reports all non-null val… #9327

Closed

mtoffl01 added 3 commits August 21, 2025 15:36

merge with configId changes made in master

c62688b

nits: add configId in ConfigProvider methods where it was missing; fi…

125744a

…x configId test to accommodate new ConfigCollector data structure

Fix 'test config id exists in ConfigCollector when using StableConfig…

8becbc5

…Source'

mtoffl01 force-pushed the mtoff/4-config-sources branch from 67d3e75 to 8becbc5 Compare August 21, 2025 20:44

mtoffl01 and others added 7 commits August 21, 2025 16:46

introduce reportDefault

db60faf

Call collect in BaseApplication.getLogInjectionEnabled; make getAppli…

750ba64

…edConfigSetting static function that takes in a map

Fix call to getAppliedConfigSetting in ConfigCollector test

3006e7c

Introduce ConfigValueResolver and ConfigMergeResolver

bee84bf

getString methods only report non-null values to telemetry

bfcfd7e

remove dependency on collectConfig for reReportToCollector and reRepo…

b411ea9

…rtFinalResult

Merge branch 'master' into mtoff/4-config-sources

d097300

mtoffl01 force-pushed the mtoff/4-config-sources branch from 369b020 to d097300 Compare August 22, 2025 20:44

mhlidd added 3 commits August 27, 2025 17:35

updating config collector to not override existing configs

1ca7fd9

Adding putDefault and adding unit tests

81db67d

relaxing reReportToCollector restraint and adding unit test

1b81e8a

mhlidd force-pushed the mtoff/4-config-sources branch from 535c685 to 1b81e8a Compare August 29, 2025 15:51

mhlidd reviewed Aug 29, 2025

View reviewed changes

telemetry/src/test/groovy/datadog/telemetry/TelemetryServiceSpecification.groovy Show resolved Hide resolved

cleanup

9b22670

mhlidd added the tag: do not merge Do not merge changes label Aug 29, 2025

mhlidd marked this pull request as ready for review August 29, 2025 20:41

mhlidd requested review from a team as code owners August 29, 2025 20:41

mhlidd requested review from sarahchen6 and removed request for a team August 29, 2025 20:41

mtoffl01 added 5 commits September 17, 2025 13:32

remove serializenulls comment in TelemetryRequestBody

779551d

remove 'no usages' comment above unused ConfigCollector put method

94f6485

Simplify javadoc for putAll

e796896

remove javadoc for ABSENT_SEQ_ID

3997f56

revert getStringInternal changes

9794631

mtoffl01 commented Sep 17, 2025

View reviewed changes

mtoffl01 added 2 commits September 17, 2025 16:21

Introduce new getStringInternal, for getting string from non-default …

84ab423

…sources

Deprecate ConfigCollector constructor without sequence ID

a9c1f7a

mcculls reviewed Sep 22, 2025

View reviewed changes

mtoffl01 added 2 commits September 22, 2025 11:01

putRemote: ConfigCollector method for 'putting' from Remote Config or…

ff39a53

…igin; migrate AppSecConfigServiceImpl to use this API

ConfigSetting.NON_DEFAULT_SEQ_ID: introduce new constant, migrate all…

0408e49

… calls to 'DEFAULT_SEQ_ID + 1' to use new constant; move constants to top of ConfigSetting class

mcculls reviewed Sep 22, 2025

View reviewed changes

internal-api/src/test/groovy/datadog/trace/api/ConfigCollectorTest.groovy Outdated Show resolved Hide resolved

ConfigCollector.put: Delete unused function

e3b307d

mcculls reviewed Sep 22, 2025

View reviewed changes

mtoffl01 and others added 7 commits September 22, 2025 11:39

Update internal-api/src/test/groovy/datadog/trace/api/ConfigCollector…

81187aa

…Test.groovy Co-authored-by: Stuart McCulloch <[email protected]>

replace 'def origin' assignment in ConfigCollectorTest with in-line use

0ffe413

ConfigCollector.put: Remove deprecated function with zero uses

b5e1414

NEW_SUB_MAP: Define reusable lambda function as static field in Confi…

b048c91

…gCollector, used by putDefault and put

updateAll: rename putAll to updateAll, and scope to Remote origin, only

0ef089d

Merge branch 'mtoff/4-config-sources' of github.com:DataDog/dd-trace-…

7ef2a9f

…java into mtoff/4-config-sources

Fix updateAll

7508fc7

		// No usages of this function
		public static ConfigSetting of(String key, Object value, ConfigOrigin origin, String configId) {

	public void putAll(Map<String, Object> configMap, ConfigOrigin origin) {
	public void putRemote(Map<String, Object> configMap) {

ConfigProvider iterates over all sources and reports all non-null values to telemetry #9404

Are you sure you want to change the base?

ConfigProvider iterates over all sources and reports all non-null values to telemetry #9404

Conversation

mtoffl01 commented Aug 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

datadog-datadog-prod-us1 bot commented Aug 21, 2025 • edited by datadog-official bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Aug 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcculls Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcculls Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcculls Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mtoffl01 commented Aug 21, 2025 •

edited

Loading

datadog-datadog-prod-us1 bot commented Aug 21, 2025 •

edited by datadog-official bot

Loading

pr-commenter bot commented Aug 21, 2025 •

edited

Loading

mcculls Sep 22, 2025 •

edited

Loading

mcculls Sep 22, 2025 •

edited

Loading

mcculls Sep 22, 2025 •

edited

Loading