Eric.firth/fix md5 incompatibility with openssl changes issue #6333
Conversation
Overall package sizeSelf size: 11.9 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.7.0 | 35.02 MB | 35.02 MB | | @datadog/native-appsec | 10.1.0 | 20.37 MB | 20.37 MB | | @datadog/native-iast-taint-tracking | 4.0.0 | 11.72 MB | 11.73 MB | | @datadog/pprof | 5.9.0 | 9.77 MB | 10.1 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.5.4 | 2.95 MB | 5.6 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | jsonpath-plus | 10.3.0 | 617.18 kB | 1.08 MB | | import-in-the-middle | 1.14.2 | 122.36 kB | 850.93 kB | | lru-cache | 10.4.3 | 804.3 kB | 804.3 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | source-map | 0.7.6 | 185.63 kB | 185.63 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 7.0.5 | 63.38 kB | 63.38 kB | | istanbul-lib-coverage | 3.2.2 | 34.37 kB | 34.37 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | dc-polyfill | 0.1.10 | 26.73 kB | 26.73 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.3 | 23.74 kB | 23.74 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.4 | 3.96 kB | 3.96 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #6333 +/- ##
==========================================
+ Coverage 83.72% 83.76% +0.04%
==========================================
Files 476 477 +1
Lines 20001 20060 +59
==========================================
+ Hits 16745 16804 +59
Misses 3256 3256 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
BenchmarksBenchmark execution time: 2025-08-26 15:52:58 Comparing candidate commit e8a3e6f in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 1268 metrics, 55 unstable metrics. |
07fd2cc to
572baf4
Compare
572baf4 to
b55192b
Compare
| function shaHash (checkpointString) { | ||
| const hash = crypto.createHash('md5').update(checkpointString).digest('hex').slice(0, 16) | ||
| return Buffer.from(hash, 'hex') | ||
| const hash = crypto.createHash('sha256').update(checkpointString).digest('hex').slice(0, 8) |
There was a problem hiding this comment.
Is it fine to just switch to sha256? I do not know what requirements we have for the hash and if we may use any hashing algorithm or not.
If changing the algorithm is fine (md5 is of course not secure, while that should not matter for the use case here, as far as I can tell), I would suggest we look into using either XXH3 or BLAKE3. These are much faster.
In addition: if we only care about the first 8 characters, we likely loose some of the hash collision guarantees. Would it potentially make sense to increase the characters we want to look at?
There was a problem hiding this comment.
the guarantees we care about are pretty slim. we take the checkpoint string, which is just a bunch of tags concatted together and we want to hash it so that the same string gives the same uint64. collisions should be avoided but isn't the worst thing in the world. I did make a change since i accidentally had made collisions more likely.
At this point I wonder if its worth adding a new dependency in XXH3. Right now we want to fix dd-trace-js asap for users who are experiencing errors.
However, there is an initiative to make DSM on by default and drop collecting the stats at the edge (meaning at the agent level, whereas the tracer is just always on). If we pursue that for node (we are currently testing this on dotnet), I believe the dsm code will have a lot of performance testing to ensure we don't slow down produce/consume calls and that is when it would make sense for us to add XXH3.
…etter error message aws-sdk's use hardcoded dsm hash values with the better error messaging to discover new ones
ac5d429 to
8b6ced2
Compare
8b6ced2 to
6369e63
Compare
* Edit the dsmStatsExist tests so they fail in a way that gives you a better error message aws-sdk's use hardcoded dsm hash values with the better error messaging to discover new ones * Update the checkpointer to use sha256 instead of md5 as it is deprecated * empty
* Edit the dsmStatsExist tests so they fail in a way that gives you a better error message aws-sdk's use hardcoded dsm hash values with the better error messaging to discover new ones * Update the checkpointer to use sha256 instead of md5 as it is deprecated * empty
What does this PR do?
Error: Hash not found. Expected: 16719995429565380169, Found hashes: [15742857388623921431], instead of erroring likeError: Timeout of 10000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.and requiring logging to get the hash value you are looking forMotivation
Related JIRA tickets: DSMS-105, DSMS-104
Reference: Node.js / OpenSSL MD5 deprecation docs
Plugin Checklist
Additional Notes