-
Notifications
You must be signed in to change notification settings - Fork 300
Rework Kotlin coroutines instrumentation around coroutine context #8774
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mcculls
wants to merge
1
commit into
master
Choose a base branch
from
mcculls/rework-kotlin-instrumentation
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+281
−954
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 55 metrics, 16 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.50.0-SNAPSHOT~37ba1f604c9, baseline=1.50.0-SNAPSHOT~f494c33b01
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.023 s) : 0, 1022887
Total [baseline] (8.665 s) : 0, 8664515
Agent [candidate] (1.025 s) : 0, 1025194
Total [candidate] (8.644 s) : 0, 8643900
section iast
Agent [baseline] (1.149 s) : 0, 1148626
Total [baseline] (9.269 s) : 0, 9269333
Agent [candidate] (1.165 s) : 0, 1164653
Total [candidate] (9.286 s) : 0, 9286475
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.161 s) : 0, 1160792
Total [baseline] (9.26 s) : 0, 9260437
Agent [candidate] (1.157 s) : 0, 1157487
Total [candidate] (9.235 s) : 0, 9234777
section iast_TELEMETRY_OFF
Agent [baseline] (1.152 s) : 0, 1152491
Total [baseline] (9.284 s) : 0, 9283688
Agent [candidate] (1.146 s) : 0, 1146319
Total [candidate] (9.237 s) : 0, 9237344
gantt
title insecure-bank - break down per module: candidate=1.50.0-SNAPSHOT~37ba1f604c9, baseline=1.50.0-SNAPSHOT~f494c33b01
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (684.658 ms) : 0, 684658
BytebuddyAgent [candidate] (686.668 ms) : 0, 686668
GlobalTracer [baseline] (241.001 ms) : 0, 241001
GlobalTracer [candidate] (239.454 ms) : 0, 239454
AppSec [baseline] (55.289 ms) : 0, 55289
AppSec [candidate] (54.309 ms) : 0, 54309
Debugger [baseline] (8.498 ms) : 0, 8498
Debugger [candidate] (9.958 ms) : 0, 9958
Remote Config [baseline] (696.465 µs) : 0, 696
Remote Config [candidate] (691.662 µs) : 0, 692
Telemetry [baseline] (9.157 ms) : 0, 9157
Telemetry [candidate] (10.519 ms) : 0, 10519
section iast
BytebuddyAgent [baseline] (801.175 ms) : 0, 801175
BytebuddyAgent [candidate] (813.532 ms) : 0, 813532
GlobalTracer [baseline] (230.966 ms) : 0, 230966
GlobalTracer [candidate] (233.491 ms) : 0, 233491
IAST [baseline] (27.56 ms) : 0, 27560
IAST [candidate] (28.586 ms) : 0, 28586
AppSec [baseline] (50.981 ms) : 0, 50981
AppSec [candidate] (50.049 ms) : 0, 50049
Debugger [baseline] (5.941 ms) : 0, 5941
Debugger [candidate] (5.958 ms) : 0, 5958
Remote Config [baseline] (588.609 µs) : 0, 589
Remote Config [candidate] (588.383 µs) : 0, 588
Telemetry [baseline] (7.879 ms) : 0, 7879
Telemetry [candidate] (7.985 ms) : 0, 7985
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (810.25 ms) : 0, 810250
BytebuddyAgent [candidate] (808.488 ms) : 0, 808488
GlobalTracer [baseline] (232.532 ms) : 0, 232532
GlobalTracer [candidate] (231.977 ms) : 0, 231977
IAST [baseline] (31.213 ms) : 0, 31213
IAST [candidate] (27.013 ms) : 0, 27013
AppSec [baseline] (48.431 ms) : 0, 48431
AppSec [candidate] (51.944 ms) : 0, 51944
Debugger [baseline] (6.0 ms) : 0, 6000
Debugger [candidate] (5.944 ms) : 0, 5944
Remote Config [baseline] (601.901 µs) : 0, 602
Remote Config [candidate] (591.775 µs) : 0, 592
Telemetry [baseline] (8.056 ms) : 0, 8056
Telemetry [candidate] (7.875 ms) : 0, 7875
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (804.041 ms) : 0, 804041
BytebuddyAgent [candidate] (799.911 ms) : 0, 799911
GlobalTracer [baseline] (231.525 ms) : 0, 231525
GlobalTracer [candidate] (230.639 ms) : 0, 230639
IAST [baseline] (24.828 ms) : 0, 24828
IAST [candidate] (22.188 ms) : 0, 22188
AppSec [baseline] (53.944 ms) : 0, 53944
AppSec [candidate] (54.97 ms) : 0, 54970
Debugger [baseline] (6.078 ms) : 0, 6078
Debugger [candidate] (5.902 ms) : 0, 5902
Remote Config [baseline] (609.543 µs) : 0, 610
Remote Config [candidate] (581.5 µs) : 0, 581
Telemetry [baseline] (7.92 ms) : 0, 7920
Telemetry [candidate] (7.755 ms) : 0, 7755
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.50.0-SNAPSHOT~37ba1f604c9, baseline=1.50.0-SNAPSHOT~f494c33b01
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.017 s) : 0, 1017425
Total [baseline] (10.564 s) : 0, 10564348
Agent [candidate] (1.021 s) : 0, 1021095
Total [candidate] (10.518 s) : 0, 10518193
section appsec
Agent [baseline] (1.162 s) : 0, 1161793
Total [baseline] (10.709 s) : 0, 10708953
Agent [candidate] (1.173 s) : 0, 1173412
Total [candidate] (10.692 s) : 0, 10692401
section iast
Agent [baseline] (1.152 s) : 0, 1151977
Total [baseline] (10.906 s) : 0, 10906097
Agent [candidate] (1.151 s) : 0, 1150989
Total [candidate] (10.849 s) : 0, 10849061
section profiling
Agent [baseline] (1.282 s) : 0, 1282440
Total [baseline] (10.975 s) : 0, 10975121
Agent [candidate] (1.292 s) : 0, 1291996
Total [candidate] (10.837 s) : 0, 10836805
gantt
title petclinic - break down per module: candidate=1.50.0-SNAPSHOT~37ba1f604c9, baseline=1.50.0-SNAPSHOT~f494c33b01
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (681.846 ms) : 0, 681846
BytebuddyAgent [candidate] (683.782 ms) : 0, 683782
GlobalTracer [baseline] (239.743 ms) : 0, 239743
GlobalTracer [candidate] (240.509 ms) : 0, 240509
AppSec [baseline] (54.958 ms) : 0, 54958
AppSec [candidate] (54.276 ms) : 0, 54276
Debugger [baseline] (7.663 ms) : 0, 7663
Debugger [candidate] (7.653 ms) : 0, 7653
Remote Config [baseline] (700.979 µs) : 0, 701
Remote Config [candidate] (700.466 µs) : 0, 700
Telemetry [baseline] (9.047 ms) : 0, 9047
Telemetry [candidate] (10.701 ms) : 0, 10701
section appsec
BytebuddyAgent [baseline] (700.454 ms) : 0, 700454
BytebuddyAgent [candidate] (709.083 ms) : 0, 709083
GlobalTracer [baseline] (236.868 ms) : 0, 236868
GlobalTracer [candidate] (239.15 ms) : 0, 239150
AppSec [baseline] (174.926 ms) : 0, 174926
AppSec [candidate] (176.584 ms) : 0, 176584
Debugger [baseline] (5.964 ms) : 0, 5964
Debugger [candidate] (5.947 ms) : 0, 5947
Remote Config [baseline] (620.509 µs) : 0, 621
Remote Config [candidate] (627.502 µs) : 0, 628
Telemetry [baseline] (8.566 ms) : 0, 8566
Telemetry [candidate] (7.385 ms) : 0, 7385
IAST [baseline] (21.685 ms) : 0, 21685
IAST [candidate] (21.757 ms) : 0, 21757
section iast
BytebuddyAgent [baseline] (804.456 ms) : 0, 804456
BytebuddyAgent [candidate] (804.066 ms) : 0, 804066
GlobalTracer [baseline] (230.693 ms) : 0, 230693
GlobalTracer [candidate] (231.045 ms) : 0, 231045
AppSec [baseline] (49.539 ms) : 0, 49539
AppSec [candidate] (51.644 ms) : 0, 51644
Debugger [baseline] (5.93 ms) : 0, 5930
Debugger [candidate] (5.846 ms) : 0, 5846
Remote Config [baseline] (606.929 µs) : 0, 607
Remote Config [candidate] (591.612 µs) : 0, 592
Telemetry [baseline] (7.936 ms) : 0, 7936
Telemetry [candidate] (7.829 ms) : 0, 7829
IAST [baseline] (29.276 ms) : 0, 29276
IAST [candidate] (24.938 ms) : 0, 24938
section profiling
BytebuddyAgent [baseline] (674.433 ms) : 0, 674433
BytebuddyAgent [candidate] (679.228 ms) : 0, 679228
GlobalTracer [baseline] (378.856 ms) : 0, 378856
GlobalTracer [candidate] (381.291 ms) : 0, 381291
AppSec [baseline] (54.726 ms) : 0, 54726
AppSec [candidate] (54.411 ms) : 0, 54411
Debugger [baseline] (6.123 ms) : 0, 6123
Debugger [candidate] (6.228 ms) : 0, 6228
Remote Config [baseline] (660.299 µs) : 0, 660
Remote Config [candidate] (653.734 µs) : 0, 654
Telemetry [baseline] (8.092 ms) : 0, 8092
Telemetry [candidate] (8.258 ms) : 0, 8258
ProfilingAgent [baseline] (109.001 ms) : 0, 109001
ProfilingAgent [candidate] (110.95 ms) : 0, 110950
Profiling [baseline] (109.026 ms) : 0, 109026
Profiling [candidate] (110.977 ms) : 0, 110977
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics. Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~37ba1f604c9, baseline=1.50.0-SNAPSHOT~f494c33b01
dateFormat X
axisFormat %s
section baseline
no_agent (1.369 ms) : 1349, 1389
. : milestone, 1369,
appsec (1.74 ms) : 1717, 1764
. : milestone, 1740,
appsec_no_iast (1.747 ms) : 1724, 1770
. : milestone, 1747,
code_origins (1.687 ms) : 1660, 1713
. : milestone, 1687,
iast (1.526 ms) : 1502, 1549
. : milestone, 1526,
profiling (1.56 ms) : 1535, 1585
. : milestone, 1560,
tracing (1.505 ms) : 1482, 1528
. : milestone, 1505,
section candidate
no_agent (1.378 ms) : 1358, 1397
. : milestone, 1378,
appsec (1.758 ms) : 1735, 1782
. : milestone, 1758,
appsec_no_iast (1.728 ms) : 1705, 1751
. : milestone, 1728,
code_origins (1.69 ms) : 1663, 1717
. : milestone, 1690,
iast (1.542 ms) : 1518, 1567
. : milestone, 1542,
profiling (1.529 ms) : 1505, 1553
. : milestone, 1529,
tracing (1.514 ms) : 1491, 1536
. : milestone, 1514,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.50.0-SNAPSHOT~37ba1f604c9, baseline=1.50.0-SNAPSHOT~f494c33b01
dateFormat X
axisFormat %s
section baseline
no_agent (387.116 µs) : 367, 407
. : milestone, 387,
iast (522.562 µs) : 499, 546
. : milestone, 523,
iast_FULL (742.318 µs) : 720, 764
. : milestone, 742,
iast_GLOBAL (574.731 µs) : 553, 596
. : milestone, 575,
iast_HARDCODED_SECRET_DISABLED (533.659 µs) : 511, 556
. : milestone, 534,
iast_INACTIVE (477.418 µs) : 455, 500
. : milestone, 477,
iast_TELEMETRY_OFF (521.982 µs) : 500, 544
. : milestone, 522,
tracing (475.291 µs) : 453, 498
. : milestone, 475,
section candidate
no_agent (385.625 µs) : 366, 406
. : milestone, 386,
iast (531.692 µs) : 509, 554
. : milestone, 532,
iast_FULL (743.579 µs) : 720, 767
. : milestone, 744,
iast_GLOBAL (576.151 µs) : 554, 598
. : milestone, 576,
iast_HARDCODED_SECRET_DISABLED (531.656 µs) : 509, 554
. : milestone, 532,
iast_INACTIVE (477.367 µs) : 456, 499
. : milestone, 477,
iast_TELEMETRY_OFF (512.75 µs) : 490, 536
. : milestone, 513,
tracing (465.495 µs) : 443, 488
. : milestone, 465,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~37ba1f604c9, baseline=1.50.0-SNAPSHOT~f494c33b01
dateFormat X
axisFormat %s
section baseline
no_agent (14.949 s) : 14949000, 14949000
. : milestone, 14949000,
appsec (15.064 s) : 15064000, 15064000
. : milestone, 15064000,
iast (19.029 s) : 19029000, 19029000
. : milestone, 19029000,
iast_GLOBAL (18.069 s) : 18069000, 18069000
. : milestone, 18069000,
profiling (15.749 s) : 15749000, 15749000
. : milestone, 15749000,
tracing (15.05 s) : 15050000, 15050000
. : milestone, 15050000,
section candidate
no_agent (14.922 s) : 14922000, 14922000
. : milestone, 14922000,
appsec (15.091 s) : 15091000, 15091000
. : milestone, 15091000,
iast (19.059 s) : 19059000, 19059000
. : milestone, 19059000,
iast_GLOBAL (17.698 s) : 17698000, 17698000
. : milestone, 17698000,
profiling (15.07 s) : 15070000, 15070000
. : milestone, 15070000,
tracing (14.842 s) : 14842000, 14842000
. : milestone, 14842000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.50.0-SNAPSHOT~37ba1f604c9, baseline=1.50.0-SNAPSHOT~f494c33b01
dateFormat X
axisFormat %s
section baseline
no_agent (1.48 ms) : 1468, 1492
. : milestone, 1480,
appsec (2.433 ms) : 2384, 2483
. : milestone, 2433,
iast (2.198 ms) : 2135, 2260
. : milestone, 2198,
iast_GLOBAL (2.265 ms) : 2201, 2328
. : milestone, 2265,
profiling (2.034 ms) : 1984, 2083
. : milestone, 2034,
tracing (2.029 ms) : 1981, 2077
. : milestone, 2029,
section candidate
no_agent (1.484 ms) : 1472, 1495
. : milestone, 1484,
appsec (2.424 ms) : 2375, 2474
. : milestone, 2424,
iast (2.198 ms) : 2136, 2260
. : milestone, 2198,
iast_GLOBAL (2.244 ms) : 2182, 2307
. : milestone, 2244,
profiling (2.07 ms) : 2019, 2121
. : milestone, 2070,
tracing (2.018 ms) : 1970, 2066
. : milestone, 2018,
|
a77d8bd
to
06b989f
Compare
48b5305
to
37ba1f6
Compare
As an added benefit, this approach does not require field injection
37ba1f6
to
91445ae
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
The previous approach associated the scope state with the job (coroutine) instance using field-injection. However this doesn't work when the coroutine instance might be recreated or wrapped by something else.
This new approach creates a
DatadogThreadContextElement
for every coroutine context created. The scope state is then lazily captured within this element: either after non-lazy coroutines are constructed, when lazy coroutines start, or failing that when the element is first used to update the thread context.At the same time it captures the active span using a continuation to avoid the trace from completing before the coroutine is complete. Note there is no need to (re)activate the continuation when switching contexts because we have already captured the scope state.
Motivation
This simpler instrumentation will be easier to move over to the new
Context
APIThis also fixes the scenario described in https://github.com/arun-rama/KotlinCoroutineTracingIssueRepro/tree/main
Additional Notes
As an added benefit, this approach does not require field injection
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issueJira ticket: AIDM-602