incorrect behavior and out of bounds accesses with keys that are prefixes of other keys #56
Description
The code does not seem to handle leaf splitting correctly when inserting keys that are prefixes of other keys. Specifically in this part of recursive_insert():
if (IS_LEAF(n)) {
art_leaf *l = `LEAF_RAW(n);`
// Check if we are updating an existing value
if (!leaf_matches(l, key, key_len, depth)) {
*old = 1;
void *old_val = l->value;
if(replace) l->value = value;
return old_val;
}
// New value, we must split the leaf into a node4
art_node4 *new_node = (art_node4*)alloc_node(NODE4);
// Create a new leaf
art_leaf *l2 = make_leaf(key, key_len, value);
// Determine longest prefix
int longest_prefix = longest_common_prefix(l, l2, depth);
new_node->n.partial_len = longest_prefix;
memcpy(new_node->n.partial, key+depth, min(MAX_PREFIX_LEN, longest_prefix));
// Add the leafs to the new node4
*ref = (art_node*)new_node;
add_child4(new_node, ref, l->key[depth+longest_prefix], SET_LEAF(l));
add_child4(new_node, ref, l2->key[depth+longest_prefix], SET_LEAF(l2));
return NULL;
}
For example, if at depth 0 you have one key "hello" and trying to insert a new value with key "hello2", longest_prefix will be 5, and the first add_child4() call will cause an out of bounds read on l->key[depth+longest_prefix]. I do not see anything in the paper or documentation that seems to account for this behavior, or mention any kind of restriction related to this. There does not seem to be an easy fix that wouldn't have far-reaching consequences. Allowing inner nodes to have values would change the cache-related behavior of the data structure dramatically. Requiring keys to be terminated with a nul byte would cause issues with keys that legitimately end with 0. Restricting keys to be of the same length would make this an entirely different data structure and make a lot of the code useless/non-sensical, and so on...