-
Notifications
You must be signed in to change notification settings - Fork 301
Rework Kotlin coroutines instrumentation around coroutine context #8774
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mcculls
wants to merge
1
commit into
master
Choose a base branch
from
mcculls/rework-kotlin-instrumentation
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+276
−954
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 14 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~2bf7b31fd6, baseline=1.50.0-SNAPSHOT~5a5e4a54b8
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.024 s) : 0, 1023991
Total [baseline] (10.515 s) : 0, 10514987
Agent [candidate] (1.018 s) : 0, 1018267
Total [candidate] (10.452 s) : 0, 10452230
section appsec
Agent [baseline] (1.173 s) : 0, 1173455
Total [baseline] (10.724 s) : 0, 10723553
Agent [candidate] (1.163 s) : 0, 1162722
Total [candidate] (10.726 s) : 0, 10726473
section iast
Agent [baseline] (1.149 s) : 0, 1149058
Total [baseline] (10.893 s) : 0, 10893469
Agent [candidate] (1.148 s) : 0, 1148055
Total [candidate] (10.82 s) : 0, 10819685
section profiling
Agent [baseline] (1.276 s) : 0, 1276422
Total [baseline] (10.808 s) : 0, 10807503
Agent [candidate] (1.294 s) : 0, 1293544
Total [candidate] (10.978 s) : 0, 10977904
gantt
title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~2bf7b31fd6, baseline=1.50.0-SNAPSHOT~5a5e4a54b8
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (687.285 ms) : 0, 687285
BytebuddyAgent [candidate] (681.886 ms) : 0, 681886
GlobalTracer [baseline] (239.977 ms) : 0, 239977
GlobalTracer [candidate] (239.85 ms) : 0, 239850
AppSec [baseline] (54.464 ms) : 0, 54464
AppSec [candidate] (54.99 ms) : 0, 54990
Debugger [baseline] (9.014 ms) : 0, 9014
Debugger [candidate] (8.316 ms) : 0, 8316
Remote Config [baseline] (683.609 µs) : 0, 684
Remote Config [candidate] (692.166 µs) : 0, 692
Telemetry [baseline] (9.012 ms) : 0, 9012
Telemetry [candidate] (9.029 ms) : 0, 9029
section appsec
BytebuddyAgent [baseline] (708.846 ms) : 0, 708846
BytebuddyAgent [candidate] (701.384 ms) : 0, 701384
GlobalTracer [baseline] (239.265 ms) : 0, 239265
GlobalTracer [candidate] (237.202 ms) : 0, 237202
AppSec [baseline] (176.295 ms) : 0, 176295
AppSec [candidate] (175.822 ms) : 0, 175822
Debugger [baseline] (5.982 ms) : 0, 5982
Debugger [candidate] (5.932 ms) : 0, 5932
Remote Config [baseline] (635.684 µs) : 0, 636
Remote Config [candidate] (625.483 µs) : 0, 625
Telemetry [baseline] (7.826 ms) : 0, 7826
Telemetry [candidate] (7.397 ms) : 0, 7397
IAST [baseline] (21.874 ms) : 0, 21874
IAST [candidate] (21.653 ms) : 0, 21653
section iast
BytebuddyAgent [baseline] (802.303 ms) : 0, 802303
BytebuddyAgent [candidate] (801.327 ms) : 0, 801327
GlobalTracer [baseline] (231.054 ms) : 0, 231054
GlobalTracer [candidate] (230.462 ms) : 0, 230462
AppSec [baseline] (52.218 ms) : 0, 52218
AppSec [candidate] (50.2 ms) : 0, 50200
Debugger [baseline] (5.907 ms) : 0, 5907
Debugger [candidate] (5.928 ms) : 0, 5928
Remote Config [baseline] (595.824 µs) : 0, 596
Remote Config [candidate] (601.661 µs) : 0, 602
Telemetry [baseline] (7.877 ms) : 0, 7877
Telemetry [candidate] (7.882 ms) : 0, 7882
IAST [baseline] (24.902 ms) : 0, 24902
IAST [candidate] (26.695 ms) : 0, 26695
section profiling
BytebuddyAgent [baseline] (673.619 ms) : 0, 673619
BytebuddyAgent [candidate] (683.433 ms) : 0, 683433
GlobalTracer [baseline] (378.833 ms) : 0, 378833
GlobalTracer [candidate] (382.829 ms) : 0, 382829
AppSec [baseline] (55.032 ms) : 0, 55032
AppSec [candidate] (54.932 ms) : 0, 54932
Debugger [baseline] (6.17 ms) : 0, 6170
Debugger [candidate] (6.291 ms) : 0, 6291
Remote Config [baseline] (664.956 µs) : 0, 665
Remote Config [candidate] (683.074 µs) : 0, 683
Telemetry [baseline] (8.108 ms) : 0, 8108
Telemetry [candidate] (8.422 ms) : 0, 8422
ProfilingAgent [baseline] (103.418 ms) : 0, 103418
ProfilingAgent [candidate] (105.76 ms) : 0, 105760
Profiling [baseline] (103.442 ms) : 0, 103442
Profiling [candidate] (105.786 ms) : 0, 105786
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~2bf7b31fd6, baseline=1.50.0-SNAPSHOT~5a5e4a54b8
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.02 s) : 0, 1019613
Total [baseline] (8.668 s) : 0, 8668461
Agent [candidate] (1.021 s) : 0, 1020537
Total [candidate] (8.657 s) : 0, 8656818
section iast
Agent [baseline] (1.148 s) : 0, 1147982
Total [baseline] (9.266 s) : 0, 9265560
Agent [candidate] (1.149 s) : 0, 1149289
Total [candidate] (9.262 s) : 0, 9262471
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.148 s) : 0, 1147744
Total [baseline] (9.215 s) : 0, 9214989
Agent [candidate] (1.155 s) : 0, 1154606
Total [candidate] (9.213 s) : 0, 9213426
section iast_TELEMETRY_OFF
Agent [baseline] (1.143 s) : 0, 1143228
Total [baseline] (9.218 s) : 0, 9217630
Agent [candidate] (1.146 s) : 0, 1145881
Total [candidate] (9.248 s) : 0, 9247787
gantt
title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~2bf7b31fd6, baseline=1.50.0-SNAPSHOT~5a5e4a54b8
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (682.2 ms) : 0, 682200
BytebuddyAgent [candidate] (681.933 ms) : 0, 681933
GlobalTracer [baseline] (240.246 ms) : 0, 240246
GlobalTracer [candidate] (240.654 ms) : 0, 240654
AppSec [baseline] (55.434 ms) : 0, 55434
AppSec [candidate] (54.931 ms) : 0, 54931
Debugger [baseline] (8.401 ms) : 0, 8401
Debugger [candidate] (9.094 ms) : 0, 9094
Remote Config [baseline] (695.053 µs) : 0, 695
Remote Config [candidate] (684.311 µs) : 0, 684
Telemetry [baseline] (9.156 ms) : 0, 9156
Telemetry [candidate] (9.719 ms) : 0, 9719
section iast
BytebuddyAgent [baseline] (801.048 ms) : 0, 801048
BytebuddyAgent [candidate] (800.507 ms) : 0, 800507
GlobalTracer [baseline] (230.536 ms) : 0, 230536
GlobalTracer [candidate] (231.557 ms) : 0, 231557
IAST [baseline] (28.291 ms) : 0, 28291
IAST [candidate] (29.351 ms) : 0, 29351
AppSec [baseline] (49.579 ms) : 0, 49579
AppSec [candidate] (49.876 ms) : 0, 49876
Debugger [baseline] (5.881 ms) : 0, 5881
Debugger [candidate] (5.966 ms) : 0, 5966
Remote Config [baseline] (591.727 µs) : 0, 592
Remote Config [candidate] (588.557 µs) : 0, 589
Telemetry [baseline] (7.901 ms) : 0, 7901
Telemetry [candidate] (7.927 ms) : 0, 7927
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (801.243 ms) : 0, 801243
BytebuddyAgent [candidate] (805.676 ms) : 0, 805676
GlobalTracer [baseline] (230.113 ms) : 0, 230113
GlobalTracer [candidate] (232.175 ms) : 0, 232175
IAST [baseline] (25.893 ms) : 0, 25893
IAST [candidate] (26.673 ms) : 0, 26673
AppSec [baseline] (52.678 ms) : 0, 52678
AppSec [candidate] (52.132 ms) : 0, 52132
Debugger [baseline] (5.937 ms) : 0, 5937
Debugger [candidate] (5.949 ms) : 0, 5949
Remote Config [baseline] (587.236 µs) : 0, 587
Remote Config [candidate] (590.579 µs) : 0, 591
Telemetry [baseline] (7.88 ms) : 0, 7880
Telemetry [candidate] (7.99 ms) : 0, 7990
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (796.978 ms) : 0, 796978
BytebuddyAgent [candidate] (799.153 ms) : 0, 799153
GlobalTracer [baseline] (229.906 ms) : 0, 229906
GlobalTracer [candidate] (230.717 ms) : 0, 230717
IAST [baseline] (22.389 ms) : 0, 22389
IAST [candidate] (22.235 ms) : 0, 22235
AppSec [baseline] (56.167 ms) : 0, 56167
AppSec [candidate] (56.035 ms) : 0, 56035
Debugger [baseline] (5.967 ms) : 0, 5967
Debugger [candidate] (5.973 ms) : 0, 5973
Remote Config [baseline] (611.903 µs) : 0, 612
Remote Config [candidate] (613.551 µs) : 0, 614
Telemetry [baseline] (7.804 ms) : 0, 7804
Telemetry [candidate] (7.778 ms) : 0, 7778
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics. Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~2bf7b31fd6, baseline=1.50.0-SNAPSHOT~5a5e4a54b8
dateFormat X
axisFormat %s
section baseline
no_agent (384.73 µs) : 365, 405
. : milestone, 385,
iast (522.389 µs) : 500, 545
. : milestone, 522,
iast_FULL (740.839 µs) : 719, 763
. : milestone, 741,
iast_GLOBAL (570.672 µs) : 549, 592
. : milestone, 571,
iast_HARDCODED_SECRET_DISABLED (518.78 µs) : 497, 541
. : milestone, 519,
iast_INACTIVE (468.955 µs) : 447, 491
. : milestone, 469,
iast_TELEMETRY_OFF (511.968 µs) : 489, 535
. : milestone, 512,
tracing (463.682 µs) : 441, 486
. : milestone, 464,
section candidate
no_agent (389.049 µs) : 368, 410
. : milestone, 389,
iast (522.539 µs) : 499, 546
. : milestone, 523,
iast_FULL (742.048 µs) : 720, 764
. : milestone, 742,
iast_GLOBAL (568.985 µs) : 547, 591
. : milestone, 569,
iast_HARDCODED_SECRET_DISABLED (529.645 µs) : 507, 552
. : milestone, 530,
iast_INACTIVE (469.956 µs) : 448, 492
. : milestone, 470,
iast_TELEMETRY_OFF (514.725 µs) : 492, 538
. : milestone, 515,
tracing (467.27 µs) : 445, 489
. : milestone, 467,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~2bf7b31fd6, baseline=1.50.0-SNAPSHOT~5a5e4a54b8
dateFormat X
axisFormat %s
section baseline
no_agent (1.36 ms) : 1341, 1379
. : milestone, 1360,
appsec (1.738 ms) : 1714, 1762
. : milestone, 1738,
appsec_no_iast (1.74 ms) : 1717, 1764
. : milestone, 1740,
code_origins (1.68 ms) : 1652, 1707
. : milestone, 1680,
iast (1.525 ms) : 1501, 1548
. : milestone, 1525,
profiling (1.538 ms) : 1514, 1561
. : milestone, 1538,
tracing (1.507 ms) : 1482, 1532
. : milestone, 1507,
section candidate
no_agent (1.368 ms) : 1348, 1387
. : milestone, 1368,
appsec (1.742 ms) : 1719, 1766
. : milestone, 1742,
appsec_no_iast (1.745 ms) : 1722, 1769
. : milestone, 1745,
code_origins (1.676 ms) : 1649, 1703
. : milestone, 1676,
iast (1.532 ms) : 1508, 1555
. : milestone, 1532,
profiling (1.58 ms) : 1555, 1604
. : milestone, 1580,
tracing (1.504 ms) : 1479, 1528
. : milestone, 1504,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~2bf7b31fd6, baseline=1.50.0-SNAPSHOT~5a5e4a54b8
dateFormat X
axisFormat %s
section baseline
no_agent (1.476 ms) : 1465, 1488
. : milestone, 1476,
appsec (2.401 ms) : 2352, 2450
. : milestone, 2401,
iast (2.178 ms) : 2117, 2240
. : milestone, 2178,
iast_GLOBAL (2.225 ms) : 2163, 2287
. : milestone, 2225,
profiling (2.034 ms) : 1984, 2083
. : milestone, 2034,
tracing (2.008 ms) : 1960, 2056
. : milestone, 2008,
section candidate
no_agent (1.472 ms) : 1460, 1483
. : milestone, 1472,
appsec (2.397 ms) : 2348, 2446
. : milestone, 2397,
iast (2.188 ms) : 2126, 2250
. : milestone, 2188,
iast_GLOBAL (2.219 ms) : 2157, 2280
. : milestone, 2219,
profiling (2.484 ms) : 2316, 2652
. : milestone, 2484,
tracing (2.018 ms) : 1970, 2066
. : milestone, 2018,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~2bf7b31fd6, baseline=1.50.0-SNAPSHOT~5a5e4a54b8
dateFormat X
axisFormat %s
section baseline
no_agent (14.974 s) : 14974000, 14974000
. : milestone, 14974000,
appsec (14.81 s) : 14810000, 14810000
. : milestone, 14810000,
iast (18.808 s) : 18808000, 18808000
. : milestone, 18808000,
iast_GLOBAL (17.924 s) : 17924000, 17924000
. : milestone, 17924000,
profiling (15.047 s) : 15047000, 15047000
. : milestone, 15047000,
tracing (15.02 s) : 15020000, 15020000
. : milestone, 15020000,
section candidate
no_agent (15.375 s) : 15375000, 15375000
. : milestone, 15375000,
appsec (15.177 s) : 15177000, 15177000
. : milestone, 15177000,
iast (18.64 s) : 18640000, 18640000
. : milestone, 18640000,
iast_GLOBAL (18.209 s) : 18209000, 18209000
. : milestone, 18209000,
profiling (15.016 s) : 15016000, 15016000
. : milestone, 15016000,
tracing (14.913 s) : 14913000, 14913000
. : milestone, 14913000,
|
a77d8bd
to
06b989f
Compare
As an added benefit, this approach does not require field injection
06b989f
to
2bf7b31
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
The previous approach associated the scope state with the job (coroutine) instance using field-injection. However this doesn't work when the coroutine instance might be recreated or wrapped by something else.
This new approach creates a
DatadogThreadContextElement
for every coroutine context created. The scope state is then lazily captured within this element: either after non-lazy coroutines are constructed, when lazy coroutines start, or failing that when the element is first used to update the thread context.At the same time it captures the active span using a continuation to avoid the trace from completing before the coroutine is complete. Note there is no need to (re)activate the continuation when switching contexts because we have already captured the scope state.
Motivation
This simpler instrumentation will be easier to move over to the new
Context
APIThis also fixes the scenario described in https://github.com/arun-rama/KotlinCoroutineTracingIssueRepro/tree/main
Additional Notes
As an added benefit, this approach does not require field injection
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issueJira ticket: AIDM-602