Make Akka's ForkJoin pool instrumented class names configurable #8766
+62
−4
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amarziali
reviewed
May 5, 2025
| @@ -52,14 +52,17 @@ public Map<String, String> contextStore() { | |||
|
|
|||
| @Override | |||
| public String hierarchyMarkerType() { | |||
| return "akka.dispatch.forkjoin.ForkJoinTask"; | |||
| return "org.apache.pekko.dispatch.ExecutorServiceConfigurator"; | |||
There was a problem hiding this comment.
why this has been changed?
There was a problem hiding this comment.
Because the previously used class (akka.dispatch.forkjoin.ForkJoinTask) is not available in logs-backend, since logs-backend uses a newer Akka version and its own ForkJoinPool implementation.
The class name returned by this method is used as a hint: class loaders that do not contain this class will be skipped by the instrumentation.
The new value is the superclass of both the original fork join pool and the logs-backend backport. If either of those pools is present in a classloader, then this parent class is guaranteed to be present too.
There was a problem hiding this comment.
I feel that doing that we're going to break customer with older versions. Is putting null something acceptable? It will bypass the classloader hint for that instrumentation
There was a problem hiding this comment.
Makes sense. I made the custom class name configurable, if nothing is configured then the old value is used as the hint, and if a custom hierarchy is configured then null is used to bypass the hint.
amarziali
reviewed
May 5, 2025
| public String[] knownMatchingTypes() { | ||
| return new String[] { | ||
| "akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask", | ||
| "com.dd.logs.concurrent.akka.AkkaForkJoinTask", // logs-backend fork |
There was a problem hiding this comment.
are we putting internal dd things in that repo? This is not tested and perhaps it will change in time. I would rather have made it configurable (see ForConfiguredType) and have a configuration flag returning that. logs-backend can then enable it optionally
There was a problem hiding this comment.
This will only cover 2 out of 3 instrumentations, the 3rd one needs to instrument a type hierarchy (every class that implements an interface).
logs-backend are going to get rid of the fork in the future and replace it with the standard implementation, also covered with these instrumentations.
There was a problem hiding this comment.
I did this for 2 instrumentations in the end, and for the third one I implemented some ad-hoc logic to allow configuring the type hierarchy root.
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 66 metrics, 5 unstable metrics. Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.49.0-SNAPSHOT~ef0443286b, baseline=1.49.0-SNAPSHOT~0e18e0e887
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.005 s) : 0, 1005059
Total [baseline] (10.493 s) : 0, 10492600
Agent [candidate] (1.009 s) : 0, 1008532
Total [candidate] (10.502 s) : 0, 10502005
section appsec
Agent [baseline] (1.151 s) : 0, 1151263
Total [baseline] (10.73 s) : 0, 10730258
Agent [candidate] (1.159 s) : 0, 1158727
Total [candidate] (10.731 s) : 0, 10730840
section iast
Agent [baseline] (1.147 s) : 0, 1146506
Total [baseline] (10.842 s) : 0, 10841636
Agent [candidate] (1.139 s) : 0, 1138536
Total [candidate] (10.83 s) : 0, 10829840
section profiling
Agent [baseline] (1.259 s) : 0, 1258834
Total [baseline] (10.887 s) : 0, 10887111
Agent [candidate] (1.26 s) : 0, 1259847
Total [candidate] (10.828 s) : 0, 10827882
gantt
title petclinic - break down per module: candidate=1.49.0-SNAPSHOT~ef0443286b, baseline=1.49.0-SNAPSHOT~0e18e0e887
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (670.082 ms) : 0, 670082
BytebuddyAgent [candidate] (674.278 ms) : 0, 674278
GlobalTracer [baseline] (240.01 ms) : 0, 240010
GlobalTracer [candidate] (240.233 ms) : 0, 240233
AppSec [baseline] (54.732 ms) : 0, 54732
AppSec [candidate] (54.519 ms) : 0, 54519
Debugger [baseline] (6.203 ms) : 0, 6203
Debugger [candidate] (6.132 ms) : 0, 6132
Remote Config [baseline] (705.121 µs) : 0, 705
Remote Config [candidate] (687.724 µs) : 0, 688
Telemetry [baseline] (9.92 ms) : 0, 9920
Telemetry [candidate] (9.135 ms) : 0, 9135
section appsec
BytebuddyAgent [baseline] (690.568 ms) : 0, 690568
BytebuddyAgent [candidate] (695.015 ms) : 0, 695015
GlobalTracer [baseline] (237.017 ms) : 0, 237017
GlobalTracer [candidate] (238.378 ms) : 0, 238378
AppSec [baseline] (174.966 ms) : 0, 174966
AppSec [candidate] (174.839 ms) : 0, 174839
Debugger [baseline] (5.869 ms) : 0, 5869
Debugger [candidate] (5.925 ms) : 0, 5925
Remote Config [baseline] (635.187 µs) : 0, 635
Remote Config [candidate] (641.411 µs) : 0, 641
Telemetry [baseline] (7.742 ms) : 0, 7742
Telemetry [candidate] (9.337 ms) : 0, 9337
IAST [baseline] (22.051 ms) : 0, 22051
IAST [candidate] (22.043 ms) : 0, 22043
section iast
BytebuddyAgent [baseline] (796.526 ms) : 0, 796526
BytebuddyAgent [candidate] (790.828 ms) : 0, 790828
GlobalTracer [baseline] (232.114 ms) : 0, 232114
GlobalTracer [candidate] (230.243 ms) : 0, 230243
AppSec [baseline] (56.846 ms) : 0, 56846
AppSec [candidate] (56.648 ms) : 0, 56648
Debugger [baseline] (5.948 ms) : 0, 5948
Debugger [candidate] (5.936 ms) : 0, 5936
Remote Config [baseline] (592.51 µs) : 0, 593
Remote Config [candidate] (603.271 µs) : 0, 603
Telemetry [baseline] (7.956 ms) : 0, 7956
Telemetry [candidate] (7.885 ms) : 0, 7885
IAST [baseline] (22.976 ms) : 0, 22976
IAST [candidate] (22.897 ms) : 0, 22897
section profiling
BytebuddyAgent [baseline] (662.588 ms) : 0, 662588
BytebuddyAgent [candidate] (664.024 ms) : 0, 664024
GlobalTracer [baseline] (379.882 ms) : 0, 379882
GlobalTracer [candidate] (379.923 ms) : 0, 379923
AppSec [baseline] (54.828 ms) : 0, 54828
AppSec [candidate] (54.215 ms) : 0, 54215
Debugger [baseline] (6.152 ms) : 0, 6152
Debugger [candidate] (6.194 ms) : 0, 6194
Remote Config [baseline] (659.256 µs) : 0, 659
Remote Config [candidate] (654.567 µs) : 0, 655
Telemetry [baseline] (8.191 ms) : 0, 8191
Telemetry [candidate] (8.246 ms) : 0, 8246
ProfilingAgent [baseline] (96.306 ms) : 0, 96306
ProfilingAgent [candidate] (96.334 ms) : 0, 96334
Profiling [baseline] (96.329 ms) : 0, 96329
Profiling [candidate] (96.357 ms) : 0, 96357
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.49.0-SNAPSHOT~ef0443286b, baseline=1.49.0-SNAPSHOT~0e18e0e887
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.011 s) : 0, 1010735
Total [baseline] (8.668 s) : 0, 8667803
Agent [candidate] (1.011 s) : 0, 1010708
Total [candidate] (8.637 s) : 0, 8636788
section iast
Agent [baseline] (1.138 s) : 0, 1138292
Total [baseline] (9.202 s) : 0, 9202227
Agent [candidate] (1.139 s) : 0, 1138799
Total [candidate] (9.221 s) : 0, 9221169
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.134 s) : 0, 1134109
Total [baseline] (9.154 s) : 0, 9154105
Agent [candidate] (1.137 s) : 0, 1136537
Total [candidate] (9.203 s) : 0, 9202729
section iast_TELEMETRY_OFF
Agent [baseline] (1.135 s) : 0, 1134858
Total [baseline] (9.187 s) : 0, 9187419
Agent [candidate] (1.143 s) : 0, 1142861
Total [candidate] (9.222 s) : 0, 9222472
gantt
title insecure-bank - break down per module: candidate=1.49.0-SNAPSHOT~ef0443286b, baseline=1.49.0-SNAPSHOT~0e18e0e887
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (675.586 ms) : 0, 675586
BytebuddyAgent [candidate] (672.289 ms) : 0, 672289
GlobalTracer [baseline] (240.643 ms) : 0, 240643
GlobalTracer [candidate] (240.577 ms) : 0, 240577
AppSec [baseline] (54.874 ms) : 0, 54874
AppSec [candidate] (54.747 ms) : 0, 54747
Debugger [baseline] (6.201 ms) : 0, 6201
Debugger [candidate] (6.881 ms) : 0, 6881
Remote Config [baseline] (720.12 µs) : 0, 720
Remote Config [candidate] (698.462 µs) : 0, 698
Telemetry [baseline] (9.234 ms) : 0, 9234
Telemetry [candidate] (12.134 ms) : 0, 12134
section iast
BytebuddyAgent [baseline] (790.805 ms) : 0, 790805
BytebuddyAgent [candidate] (791.543 ms) : 0, 791543
GlobalTracer [baseline] (230.537 ms) : 0, 230537
GlobalTracer [candidate] (230.501 ms) : 0, 230501
IAST [baseline] (22.765 ms) : 0, 22765
IAST [candidate] (22.813 ms) : 0, 22813
AppSec [baseline] (56.292 ms) : 0, 56292
AppSec [candidate] (56.229 ms) : 0, 56229
Debugger [baseline] (5.944 ms) : 0, 5944
Debugger [candidate] (5.89 ms) : 0, 5890
Remote Config [baseline] (588.018 µs) : 0, 588
Remote Config [candidate] (581.053 µs) : 0, 581
Telemetry [baseline] (7.947 ms) : 0, 7947
Telemetry [candidate] (7.876 ms) : 0, 7876
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (787.476 ms) : 0, 787476
BytebuddyAgent [candidate] (788.943 ms) : 0, 788943
GlobalTracer [baseline] (229.406 ms) : 0, 229406
GlobalTracer [candidate] (229.768 ms) : 0, 229768
IAST [baseline] (22.923 ms) : 0, 22923
IAST [candidate] (22.985 ms) : 0, 22985
AppSec [baseline] (56.735 ms) : 0, 56735
AppSec [candidate] (56.995 ms) : 0, 56995
Debugger [baseline] (5.842 ms) : 0, 5842
Debugger [candidate] (5.918 ms) : 0, 5918
Remote Config [baseline] (581.51 µs) : 0, 582
Remote Config [candidate] (587.556 µs) : 0, 588
Telemetry [baseline] (7.826 ms) : 0, 7826
Telemetry [candidate] (7.862 ms) : 0, 7862
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (787.966 ms) : 0, 787966
BytebuddyAgent [candidate] (794.134 ms) : 0, 794134
GlobalTracer [baseline] (230.831 ms) : 0, 230831
GlobalTracer [candidate] (231.816 ms) : 0, 231816
IAST [baseline] (22.364 ms) : 0, 22364
IAST [candidate] (22.58 ms) : 0, 22580
AppSec [baseline] (56.025 ms) : 0, 56025
AppSec [candidate] (56.408 ms) : 0, 56408
Debugger [baseline] (5.94 ms) : 0, 5940
Debugger [candidate] (5.926 ms) : 0, 5926
Remote Config [baseline] (608.529 µs) : 0, 609
Remote Config [candidate] (609.982 µs) : 0, 610
Telemetry [baseline] (7.762 ms) : 0, 7762
Telemetry [candidate] (7.855 ms) : 0, 7855
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 18 unstable metrics. Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~ef0443286b, baseline=1.49.0-SNAPSHOT~0e18e0e887
dateFormat X
axisFormat %s
section baseline
no_agent (1.354 ms) : 1335, 1374
. : milestone, 1354,
appsec (1.733 ms) : 1710, 1756
. : milestone, 1733,
appsec_no_iast (1.724 ms) : 1701, 1747
. : milestone, 1724,
code_origins (1.682 ms) : 1655, 1709
. : milestone, 1682,
iast (1.52 ms) : 1496, 1543
. : milestone, 1520,
profiling (1.52 ms) : 1497, 1544
. : milestone, 1520,
tracing (1.484 ms) : 1459, 1510
. : milestone, 1484,
section candidate
no_agent (1.367 ms) : 1347, 1386
. : milestone, 1367,
appsec (1.741 ms) : 1718, 1765
. : milestone, 1741,
appsec_no_iast (1.748 ms) : 1724, 1772
. : milestone, 1748,
code_origins (1.688 ms) : 1662, 1715
. : milestone, 1688,
iast (1.505 ms) : 1481, 1528
. : milestone, 1505,
profiling (1.517 ms) : 1494, 1541
. : milestone, 1517,
tracing (1.488 ms) : 1464, 1513
. : milestone, 1488,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.49.0-SNAPSHOT~ef0443286b, baseline=1.49.0-SNAPSHOT~0e18e0e887
dateFormat X
axisFormat %s
section baseline
no_agent (388.543 µs) : 369, 408
. : milestone, 389,
iast (524.177 µs) : 500, 548
. : milestone, 524,
iast_FULL (743.735 µs) : 720, 767
. : milestone, 744,
iast_GLOBAL (574.483 µs) : 551, 598
. : milestone, 574,
iast_HARDCODED_SECRET_DISABLED (531.845 µs) : 509, 555
. : milestone, 532,
iast_INACTIVE (471.732 µs) : 449, 494
. : milestone, 472,
iast_TELEMETRY_OFF (518.021 µs) : 495, 541
. : milestone, 518,
tracing (462.162 µs) : 440, 484
. : milestone, 462,
section candidate
no_agent (384.904 µs) : 365, 405
. : milestone, 385,
iast (528.501 µs) : 505, 552
. : milestone, 529,
iast_FULL (736.223 µs) : 713, 760
. : milestone, 736,
iast_GLOBAL (582.271 µs) : 558, 606
. : milestone, 582,
iast_HARDCODED_SECRET_DISABLED (522.629 µs) : 500, 545
. : milestone, 523,
iast_INACTIVE (464.884 µs) : 443, 487
. : milestone, 465,
iast_TELEMETRY_OFF (515.989 µs) : 494, 538
. : milestone, 516,
tracing (459.221 µs) : 437, 481
. : milestone, 459,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~ef0443286b, baseline=1.49.0-SNAPSHOT~0e18e0e887
dateFormat X
axisFormat %s
section baseline
no_agent (1.476 ms) : 1464, 1488
. : milestone, 1476,
appsec (2.384 ms) : 2337, 2432
. : milestone, 2384,
iast (2.161 ms) : 2101, 2220
. : milestone, 2161,
iast_GLOBAL (2.215 ms) : 2155, 2276
. : milestone, 2215,
profiling (2.487 ms) : 2307, 2667
. : milestone, 2487,
tracing (2.004 ms) : 1958, 2051
. : milestone, 2004,
section candidate
no_agent (1.48 ms) : 1468, 1491
. : milestone, 1480,
appsec (2.389 ms) : 2342, 2437
. : milestone, 2389,
iast (2.167 ms) : 2107, 2227
. : milestone, 2167,
iast_GLOBAL (2.211 ms) : 2151, 2271
. : milestone, 2211,
profiling (2.041 ms) : 1992, 2089
. : milestone, 2041,
tracing (1.987 ms) : 1941, 2033
. : milestone, 1987,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.49.0-SNAPSHOT~ef0443286b, baseline=1.49.0-SNAPSHOT~0e18e0e887
dateFormat X
axisFormat %s
section baseline
no_agent (15.405 s) : 15405000, 15405000
. : milestone, 15405000,
appsec (14.976 s) : 14976000, 14976000
. : milestone, 14976000,
iast (18.512 s) : 18512000, 18512000
. : milestone, 18512000,
iast_GLOBAL (18.217 s) : 18217000, 18217000
. : milestone, 18217000,
profiling (15.262 s) : 15262000, 15262000
. : milestone, 15262000,
tracing (15.011 s) : 15011000, 15011000
. : milestone, 15011000,
section candidate
no_agent (15.368 s) : 15368000, 15368000
. : milestone, 15368000,
appsec (15.072 s) : 15072000, 15072000
. : milestone, 15072000,
iast (19.027 s) : 19027000, 19027000
. : milestone, 19027000,
iast_GLOBAL (17.967 s) : 17967000, 17967000
. : milestone, 17967000,
profiling (14.886 s) : 14886000, 14886000
. : milestone, 14886000,
tracing (14.873 s) : 14873000, 14873000
. : milestone, 14873000,
|
amarziali
reviewed
May 5, 2025
| @@ -72,6 +72,11 @@ public final class TraceInstrumentationConfig { | |||
|
|
|||
| public static final String JDBC_CONNECTION_CLASS_NAME = "trace.jdbc.connection.class.name"; | |||
|
|
|||
| public static final String AKKA_FORK_JOIN_TASK_NAME = "trace.akka.fork.join.task.name"; | |||
There was a problem hiding this comment.
it's really a nit (for the three configuration options). Since it's an additional class (it matches with a or) perhaps something like trace.akka.additional.fork.join.task.name is more clear
amarziali
approved these changes
May 5, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Update Akka's ForkJoinPool instrumentations to also instrument backported fork of the pool that is used by logs-backend (existing instrumentations can now be configured to instrument custom class names; logs-backend-specific class names will be provided in logs-backend tracer config).
Motivation
CI Visibility's per-test code coverage relies on spans propagation to capture coverage data from different threads.
Without this instrumentation coverage data for test cases that use Akka is incomplete.
Additional Notes
More context as to why logs-backend has its own ForkJoinPool implementation can be found in this PR.
Contributor Checklist
type:and (comp:orinst:) labels in addition to any usefull labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: SDTEST-1957