ConfigProvider iterates over all sources and reports all non-null values to telemetry #9404
+1,023
−201
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
🎯 Code Coverage 🔗 Commit SHA: 5b5ad0d | Docs | Was this helpful? Give us feedback! |
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 4 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~5b5ad0d63e, baseline=1.54.0-SNAPSHOT~e992b65c80
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.004 s) : 0, 1003725
Total [baseline] (8.622 s) : 0, 8622358
Agent [candidate] (1.01 s) : 0, 1009642
Total [candidate] (8.646 s) : 0, 8645933
section iast
Agent [baseline] (1.148 s) : 0, 1147789
Total [baseline] (9.28 s) : 0, 9280276
Agent [candidate] (1.144 s) : 0, 1143836
Total [candidate] (9.28 s) : 0, 9280082
gantt
title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~5b5ad0d63e, baseline=1.54.0-SNAPSHOT~e992b65c80
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.463 ms) : 0, 1463
crashtracking [candidate] (1.468 ms) : 0, 1468
BytebuddyAgent [baseline] (686.741 ms) : 0, 686741
BytebuddyAgent [candidate] (690.903 ms) : 0, 690903
GlobalTracer [baseline] (247.442 ms) : 0, 247442
GlobalTracer [candidate] (248.881 ms) : 0, 248881
AppSec [baseline] (30.963 ms) : 0, 30963
AppSec [candidate] (31.235 ms) : 0, 31235
Debugger [baseline] (6.307 ms) : 0, 6307
Debugger [candidate] (6.443 ms) : 0, 6443
Remote Config [baseline] (678.336 µs) : 0, 678
Remote Config [candidate] (670.796 µs) : 0, 671
Telemetry [baseline] (9.064 ms) : 0, 9064
Telemetry [candidate] (9.024 ms) : 0, 9024
section iast
crashtracking [baseline] (1.463 ms) : 0, 1463
crashtracking [candidate] (1.45 ms) : 0, 1450
BytebuddyAgent [baseline] (811.194 ms) : 0, 811194
BytebuddyAgent [candidate] (809.142 ms) : 0, 809142
GlobalTracer [baseline] (238.913 ms) : 0, 238913
GlobalTracer [candidate] (237.078 ms) : 0, 237078
AppSec [baseline] (33.78 ms) : 0, 33780
AppSec [candidate] (33.923 ms) : 0, 33923
Debugger [baseline] (6.131 ms) : 0, 6131
Debugger [candidate] (6.048 ms) : 0, 6048
Remote Config [baseline] (588.017 µs) : 0, 588
Remote Config [candidate] (583.044 µs) : 0, 583
Telemetry [baseline] (8.414 ms) : 0, 8414
Telemetry [candidate] (8.261 ms) : 0, 8261
IAST [baseline] (26.142 ms) : 0, 26142
IAST [candidate] (26.181 ms) : 0, 26181
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~5b5ad0d63e, baseline=1.54.0-SNAPSHOT~e992b65c80
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.007 s) : 0, 1006614
Total [baseline] (10.664 s) : 0, 10663696
Agent [candidate] (1.016 s) : 0, 1015808
Total [candidate] (10.755 s) : 0, 10754656
section appsec
Agent [baseline] (1.186 s) : 0, 1186474
Total [baseline] (10.956 s) : 0, 10955930
Agent [candidate] (1.187 s) : 0, 1186686
Total [candidate] (10.964 s) : 0, 10963765
section iast
Agent [baseline] (1.15 s) : 0, 1150186
Total [baseline] (10.954 s) : 0, 10954200
Agent [candidate] (1.143 s) : 0, 1142597
Total [candidate] (10.977 s) : 0, 10976555
section profiling
Agent [baseline] (1.153 s) : 0, 1152893
Total [baseline] (11.034 s) : 0, 11033584
Agent [candidate] (1.153 s) : 0, 1153368
Total [candidate] (11.07 s) : 0, 11069707
gantt
title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~5b5ad0d63e, baseline=1.54.0-SNAPSHOT~e992b65c80
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.471 ms) : 0, 1471
crashtracking [candidate] (1.469 ms) : 0, 1469
BytebuddyAgent [baseline] (689.074 ms) : 0, 689074
BytebuddyAgent [candidate] (695.313 ms) : 0, 695313
GlobalTracer [baseline] (247.587 ms) : 0, 247587
GlobalTracer [candidate] (250.284 ms) : 0, 250284
AppSec [baseline] (31.183 ms) : 0, 31183
AppSec [candidate] (31.376 ms) : 0, 31376
Debugger [baseline] (6.344 ms) : 0, 6344
Debugger [candidate] (6.451 ms) : 0, 6451
Remote Config [baseline] (674.175 µs) : 0, 674
Remote Config [candidate] (672.549 µs) : 0, 673
Telemetry [baseline] (9.028 ms) : 0, 9028
Telemetry [candidate] (9.027 ms) : 0, 9027
section appsec
crashtracking [baseline] (1.459 ms) : 0, 1459
crashtracking [candidate] (1.451 ms) : 0, 1451
BytebuddyAgent [baseline] (711.198 ms) : 0, 711198
BytebuddyAgent [candidate] (712.429 ms) : 0, 712429
GlobalTracer [baseline] (239.61 ms) : 0, 239610
GlobalTracer [candidate] (240.008 ms) : 0, 240008
AppSec [baseline] (170.805 ms) : 0, 170805
AppSec [candidate] (171.995 ms) : 0, 171995
Debugger [baseline] (6.761 ms) : 0, 6761
Debugger [candidate] (5.953 ms) : 0, 5953
Remote Config [baseline] (626.922 µs) : 0, 627
Remote Config [candidate] (641.711 µs) : 0, 642
Telemetry [baseline] (10.009 ms) : 0, 10009
Telemetry [candidate] (8.43 ms) : 0, 8430
IAST [baseline] (24.877 ms) : 0, 24877
IAST [candidate] (24.696 ms) : 0, 24696
section iast
crashtracking [baseline] (1.457 ms) : 0, 1457
crashtracking [candidate] (1.443 ms) : 0, 1443
BytebuddyAgent [baseline] (813.255 ms) : 0, 813255
BytebuddyAgent [candidate] (809.125 ms) : 0, 809125
GlobalTracer [baseline] (239.117 ms) : 0, 239117
GlobalTracer [candidate] (236.609 ms) : 0, 236609
AppSec [baseline] (33.934 ms) : 0, 33934
AppSec [candidate] (33.566 ms) : 0, 33566
Debugger [baseline] (6.121 ms) : 0, 6121
Debugger [candidate] (5.983 ms) : 0, 5983
Remote Config [baseline] (603.261 µs) : 0, 603
Remote Config [candidate] (578.162 µs) : 0, 578
Telemetry [baseline] (8.197 ms) : 0, 8197
Telemetry [candidate] (8.195 ms) : 0, 8195
IAST [baseline] (26.309 ms) : 0, 26309
IAST [candidate] (25.89 ms) : 0, 25890
section profiling
ProfilingAgent [baseline] (101.053 ms) : 0, 101053
ProfilingAgent [candidate] (101.254 ms) : 0, 101254
crashtracking [baseline] (1.451 ms) : 0, 1451
crashtracking [candidate] (1.45 ms) : 0, 1450
BytebuddyAgent [baseline] (719.977 ms) : 0, 719977
BytebuddyAgent [candidate] (719.063 ms) : 0, 719063
GlobalTracer [baseline] (224.253 ms) : 0, 224253
GlobalTracer [candidate] (224.767 ms) : 0, 224767
AppSec [baseline] (31.396 ms) : 0, 31396
AppSec [candidate] (31.266 ms) : 0, 31266
Debugger [baseline] (8.03 ms) : 0, 8030
Debugger [candidate] (7.287 ms) : 0, 7287
Remote Config [baseline] (708.737 µs) : 0, 709
Remote Config [candidate] (717.38 µs) : 0, 717
Telemetry [baseline] (14.789 ms) : 0, 14789
Telemetry [candidate] (15.549 ms) : 0, 15549
Profiling [baseline] (101.632 ms) : 0, 101632
Profiling [candidate] (101.85 ms) : 0, 101850
LoadParameters
See matching parameters
SummaryFound 1 performance improvements and 2 performance regressions! Performance is the same for 9 metrics, 12 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~5b5ad0d63e, baseline=1.54.0-SNAPSHOT~27641dbc53
dateFormat X
axisFormat %s
section baseline
no_agent (37.177 ms) : 36876, 37477
. : milestone, 37177,
appsec (48.061 ms) : 47633, 48489
. : milestone, 48061,
code_origins (44.227 ms) : 43844, 44611
. : milestone, 44227,
iast (44.777 ms) : 44380, 45175
. : milestone, 44777,
profiling (49.005 ms) : 48558, 49451
. : milestone, 49005,
tracing (44.254 ms) : 43882, 44627
. : milestone, 44254,
section candidate
no_agent (37.818 ms) : 37511, 38124
. : milestone, 37818,
appsec (48.924 ms) : 48481, 49366
. : milestone, 48924,
code_origins (43.1 ms) : 42731, 43468
. : milestone, 43100,
iast (44.552 ms) : 44175, 44928
. : milestone, 44552,
profiling (50.799 ms) : 50319, 51279
. : milestone, 50799,
tracing (45.089 ms) : 44696, 45481
. : milestone, 45089,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~5b5ad0d63e, baseline=1.54.0-SNAPSHOT~27641dbc53
dateFormat X
axisFormat %s
section baseline
no_agent (4.348 ms) : 4296, 4401
. : milestone, 4348,
iast (9.763 ms) : 9599, 9926
. : milestone, 9763,
iast_FULL (14.101 ms) : 13810, 14392
. : milestone, 14101,
iast_GLOBAL (10.981 ms) : 10780, 11182
. : milestone, 10981,
profiling (9.048 ms) : 8899, 9196
. : milestone, 9048,
tracing (7.585 ms) : 7474, 7696
. : milestone, 7585,
section candidate
no_agent (4.372 ms) : 4318, 4426
. : milestone, 4372,
iast (9.721 ms) : 9558, 9885
. : milestone, 9721,
iast_FULL (14.344 ms) : 14058, 14629
. : milestone, 14344,
iast_GLOBAL (10.397 ms) : 10213, 10582
. : milestone, 10397,
profiling (9.564 ms) : 9415, 9712
. : milestone, 9564,
tracing (7.747 ms) : 7632, 7861
. : milestone, 7747,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~5b5ad0d63e, baseline=1.54.0-SNAPSHOT~e992b65c80
dateFormat X
axisFormat %s
section baseline
no_agent (1.474 ms) : 1462, 1485
. : milestone, 1474,
appsec (3.736 ms) : 3519, 3954
. : milestone, 3736,
iast (2.21 ms) : 2146, 2273
. : milestone, 2210,
iast_GLOBAL (2.252 ms) : 2188, 2316
. : milestone, 2252,
profiling (2.065 ms) : 2014, 2117
. : milestone, 2065,
tracing (2.02 ms) : 1971, 2070
. : milestone, 2020,
section candidate
no_agent (1.474 ms) : 1463, 1486
. : milestone, 1474,
appsec (3.742 ms) : 3521, 3962
. : milestone, 3742,
iast (2.212 ms) : 2148, 2276
. : milestone, 2212,
iast_GLOBAL (2.241 ms) : 2178, 2305
. : milestone, 2241,
profiling (2.072 ms) : 2019, 2125
. : milestone, 2072,
tracing (2.032 ms) : 1982, 2081
. : milestone, 2032,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~5b5ad0d63e, baseline=1.54.0-SNAPSHOT~e992b65c80
dateFormat X
axisFormat %s
section baseline
no_agent (15.454 s) : 15454000, 15454000
. : milestone, 15454000,
appsec (15.133 s) : 15133000, 15133000
. : milestone, 15133000,
iast (18.341 s) : 18341000, 18341000
. : milestone, 18341000,
iast_GLOBAL (18.025 s) : 18025000, 18025000
. : milestone, 18025000,
profiling (15.529 s) : 15529000, 15529000
. : milestone, 15529000,
tracing (15.136 s) : 15136000, 15136000
. : milestone, 15136000,
section candidate
no_agent (15.026 s) : 15026000, 15026000
. : milestone, 15026000,
appsec (15.127 s) : 15127000, 15127000
. : milestone, 15127000,
iast (18.324 s) : 18324000, 18324000
. : milestone, 18324000,
iast_GLOBAL (18.293 s) : 18293000, 18293000
. : milestone, 18293000,
profiling (15.487 s) : 15487000, 15487000
. : milestone, 15487000,
tracing (15.037 s) : 15037000, 15037000
. : milestone, 15037000,
|
…x configId test to accommodate new ConfigCollector data structure
67d3e75 to
8becbc5
Compare
…edConfigSetting static function that takes in a map
369b020 to
d097300
Compare
535c685 to
1b81e8a
Compare
mhlidd
reviewed
Aug 29, 2025
| } | ||
|
|
||
| /** Helper class to store resolved configuration values with their metadata */ | ||
| static class ConfigValueResolver<T> { |
There was a problem hiding this comment.
This class is not private in order to expose it for a unit test that is needed to complete code coverage. The unit test verifies that reReportToCollector is not invoked when origin == null, which should never happen since the method to create a ConfigValueResolver should only occur when no re-reporting is expected. This check is only here as a safety check against poor usage of the function.
mhlidd
reviewed
Aug 29, 2025
telemetry/src/test/groovy/datadog/telemetry/TelemetryServiceSpecification.groovy
Show resolved
Hide resolved
mcculls
reviewed
Sep 22, 2025
| * @param configMap map of configuration key-value pairs to add | ||
| * @param origin the configuration origin for all entries | ||
| */ | ||
| public void putAll(Map<String, Object> configMap, ConfigOrigin origin) { |
There was a problem hiding this comment.
This is only used by DynamicConfig to report remote config updates - so I would change this to be
Suggested change
| public void putAll(Map<String, Object> configMap, ConfigOrigin origin) { | |
| public void putRemote(Map<String, Object> configMap) { |
and use ConfigOrigin.REMOTE to get the right sub-map to update.
mcculls
reviewed
Sep 22, 2025
| */ | ||
| public void putAll(Map<String, Object> configMap, ConfigOrigin origin) { | ||
| for (Map.Entry<String, Object> entry : configMap.entrySet()) { | ||
| put(entry.getKey(), entry.getValue(), origin, ABSENT_SEQ_ID, null); |
There was a problem hiding this comment.
This breaks the atomicity of the previous implementation - i.e. with the new code if a remote config update happens at the same time the config is collected for telemetry then half of the remote config update could land in the outgoing batch and the rest would land in the subsequent batch. This could result in inconsistent data being shown in the UI.
I'll add an update to show how the atomicity can be kept with the new sub-map approach.
…Test.groovy Co-authored-by: Stuart McCulloch <[email protected]>
…gCollector, used by putDefault and put
…java into mtoff/4-config-sources
6 tasks
mcculls
reviewed
Sep 23, 2025
utils/config-utils/src/main/java/datadog/trace/bootstrap/config/provider/ConfigProvider.java
Show resolved
Hide resolved
mcculls
reviewed
Sep 23, 2025
utils/config-utils/src/main/java/datadog/trace/bootstrap/config/provider/ConfigProvider.java
Show resolved
Hide resolved
mcculls
reviewed
Sep 23, 2025
utils/config-utils/src/main/java/datadog/trace/bootstrap/config/provider/ConfigProvider.java
Outdated
Show resolved
Hide resolved
mcculls
reviewed
Sep 23, 2025
utils/config-utils/src/main/java/datadog/trace/bootstrap/config/provider/ConfigProvider.java
Outdated
Show resolved
Hide resolved
mcculls
reviewed
Sep 23, 2025
| */ | ||
| void reReportFinalResult(String key, int finalSeqId) { | ||
| if (currentOrigin != ConfigOrigin.DEFAULT && !mergedValue.isEmpty()) { | ||
| ConfigCollector.get().put(key, mergedValue, ConfigOrigin.CALCULATED, finalSeqId); |
There was a problem hiding this comment.
What if currentOrigin is not DEFAULT and not CALCULATED, i.e. when there is only one contribution
Should we still be reporting it as CALCULATED or should we be using currentOrigin here?
mcculls
reviewed
Sep 23, 2025
utils/config-utils/src/main/java/datadog/trace/bootstrap/config/provider/ConfigProvider.java
Outdated
Show resolved
Hide resolved
…g/provider/ConfigProvider.java Co-authored-by: Stuart McCulloch <[email protected]>
…g/provider/ConfigProvider.java Co-authored-by: Stuart McCulloch <[email protected]>
…g/provider/ConfigProvider.java Co-authored-by: Stuart McCulloch <[email protected]>
…or all non-default values
…java into mtoff/4-config-sources
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Previously, telemetry was only being reported for the
ConfigSourcethat set the final value of the configuration. This PR introduces reporting telemetry for allConfigSourcesthat have non-null values set, regardless of whether it sets the final configuration value or not.Summary of Changes:
ConfigCollectorMap<ConfigOrigin, Map<String (configName), ConfigSetting>>putmethod in favor of a newputmethod withseq_idfieldputDefaultmethod is created to ensure originally reported defaults are not overwritten. Use cases:getEnum,getIntegerRange,getSetandgetListinvokegetString, which erroneously reports defaults as null under the hoodisExplicitlyDisabledis called and invokesgetBooleanwithout the proper default value, which previously would override theDEFAULTtelemetry payload withnull.ConfigSettingseqIdand newoffunction that takes it in as a parameterConfigProviderConfigSourcesstart from lowest priority to highest priority. A localseqIdis used to track priority ofConfigSourcesthat are set. In the case of parsing errors, there may be re-reporting of the last validConfigSourceto ensure the highest priority and valid ConfigSource is reported with the highestseqIdConfigValueResolverkey,value,seqId, andConfigOrigin.ConfigProvidermethods now store the final return value throughConfigValueResolverConfigMergeResolverMotivation
RFC
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any usefull labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]