Fix bug where tree_mask would get incorrectly set in certain cases #103
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CodSpeed Performance ReportMerging #103 will not alter performanceComparing Summary
|
530b2e0 to
da11f37
Compare
As reported in: paradigmxyz/reth#12080 paradigmxyz/reth#12129 the tree_mask would sometimes get a bit set on it even though the corresponding child wouldn't appear in the set of updated branch nodes. This was due to the bit getting incorrectly set for extension nodes which pointed to updated branch nodes. The solution is a bit of a hack.
da11f37 to
451cd09
Compare
451cd09 to
e202160
Compare
|
My work being done here is going beyond the tree_mask bug, I'm honestly not even sure that is a bug at this point, just a misunderstanding in the code documentation about tree mask is (vs how we use it). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hello! I noted the following open bug in reth and decided to use the opportunity to get a better understanding for the codebase and related libraries:
paradigmxyz/reth#12129 (related PR which provided a test case: paradigmxyz/reth#12080)
Most of my understanding of the problem and solution is based on my reading of the reth and alloy codebases and other materials, so I could well have made some wrong assumptions or misunderstood some fundamental point. Sorry if that's the case, but please let me know!
As I dug into the issue I found that the source of the problem was in the HashBuilder here in the trie library. As described in the original bug/pr, in certain cases the tree_mask would get a bit set on it even though the corresponding child wouldn't appear in the set of updated branch nodes. This was due to the bit getting incorrectly set for extension nodes which pointed to intermediate branch nodes.
For example, the following is a simplified reproduction case based on the one in the above PR. Given these leafs:
The node
0xb6is an extension node with short key5leading to the branch node0xb65, which does appear in the updated branch nodes result. The branch node at0xbwas incorrectly getting the bit for child 6 set in its tree_masks, due to this code in theupdate_masksmethod:trie/src/hash_builder/mod.rs
Lines 434 to 436 in c69e9f5
Unfortunately it was not enough to simply remove the offending line, as the tree_masks is getting used to determine if an updated branch node should be created at all, and so would cause the
0xb65branch node to not be included in the updated branch nodes set.trie/src/hash_builder/mod.rs
Line 401 in c69e9f5
My solution is to simply do a bitwise AND between the hash_mask and the tree_mask when creating the BranchNodeCompact. If I'm not mistaken the hash mask will be correctly set with a bit only for child branch nodes, regardless of if the node is an intermediate node, and so is always a superset of the desired tree_mask. If this isn't correct then another solution will need to be found.
I've included a bunch of extra tracing which I found helpful during debugging, as well as some test cases which check the solution. If any of these aren't up to coding guidelines or what-have-you I can of course change them.