fix: bump pointer/bump chunk debug assertion#313
Conversation
|
Another debug_assert problem caught by fuzzing. This time the pointer is asserted to be aligned before an aligned pointer is created. Minimal case: let mut bump = Bump::<16>::with_min_align();
let ptr = bump.alloc(0u8); |
fitzgen
left a comment
There was a problem hiding this comment.
Please include the minimal reproduction(s) as regression tests in tests/all/tests.rs or somewhere else where they fit.
Also, can you share the fuzz target you've written? Are you using libfuzzer and cargo fuzz or something else?
Anyways, I think this bug requires a different fix from the one implemented here; see my comment below. I haven't had time to dig in further myself yet, however.
| debug_assert!( | ||
| is_pointer_aligned_to(ptr, MIN_ALIGN), | ||
| "bump pointer {ptr:#p} should be aligned to the minimum alignment of {MIN_ALIGN:#x}" | ||
| ); |
There was a problem hiding this comment.
I don't think removing this assertion is correct; if it is failing, then we should fix the code such that it passes.
The Ordering::Equal case right below this will assume that ptr - aligned_size will result in an aligned pointer, and if this assertion doesn't hold then that won't be the case.
There was a problem hiding this comment.
I think it only fails when layout.align() < MIN_ALIGN, where the problem is that the debug_assert is done too early before it is even aligned!
The aligned_ptr that is returned passes both debug_assert's where it is aligned to layout.align() and MIN_ALIGN as expected.
There was a problem hiding this comment.
The intended invariant is that ptr should always be aligned to MIN_ALIGN, and then in the Ordering::Less case we do this:
Lines 1916 to 1928 in 0b12966
If ptr is not aligned going into that block of code, then its result will not be aligned either.
the problem is that the debug_assert is done too early before it is even aligned!
That block does not align ptr, it aligns the amount by which we adjust ptr. Adjusting by an aligned amount means that
- if the
ptrwas initially aligned, then the adjusted pointer is also aligned - if the
ptrwas not initially aligned, then the adjusted pointer is also not aligned
For example, if layout.size() = 1, and MIN_ALIGN = 8 then aligned_size will be 8 and
- if
ptr = 0x1008(which is aligned toMIN_ALIGN) then the resulting adjusted pointer will be0x1000(which is also aligned toMIN_ALIGN). - if
ptr = 0x1009(which is not aligned toMIN_ALIGN) then the resulting adjusted pointer will be0x1001(which is also not aligned toMIN_ALIGN).
So I think there is something else funky going on here, and removing this assertion is not the correct fix.
There was a problem hiding this comment.
Yes, you are right that if ptr was not aligned to begin with it still won't be after that alignment, sorry about that. It seems my misconception stems from the non-determinancy of where the chunk is placed.
It is very finicky in that if the other regression test is added then it doesn't trigger the debug_assert anymore (since the ptr is suddenly properly aligned). Equally, if the faulting debug_assert is removed, ptr is again magically aligned, masking the underlying issue 🤦♂️. I will dig deeper into it.
There was a problem hiding this comment.
Using rr could help here, if you're on x86-64 linux:
- break on the assertion failure
- set a watchpoint on the chunk footer's ptr field
- reverse continue
This should bring you to the exact location where the mis-aligned ptr is being written into the footer.
There was a problem hiding this comment.
Found the issue. This is caused by the EMPTY_CHUNK static singleton sometimes being placed at 0 mod 16 and sometimes 8 mod 16 (problematic), so not really an issue since the capacity is always 0 and hence the function will always return None.
I can either specify #[repr(align(16))] to EmptyChunkFooter, but I can't use CHUNK_ALIGN in there since it currently only supports literal numbers inside. Alternative solution is to change the condition of the debug_assert, something like: core::ptr::eq(ptr, &EMPTY_CHUNK as *const _ as _) || is_pointer_aligned_to(ptr, MIN_ALIGN).
There was a problem hiding this comment.
D'oh, I was pretty sure that we already had that repr(align(16)) there.
You can add a static assert that it matches CHUNK_ALIGN like this:
const _CHUNK_FOOTER_ALIGN_IS_CHUNK_ALIGN: () = {
assert!(mem::align_of::<ChunkFooter>() == CHUNK_ALIGN);
};There was a problem hiding this comment.
Mhm I'm not sure if it should apply to ChunkFooter or EmptyChunkFooter since there is already this assertion:
Lines 478 to 482 in 0b12966
which implies that ChunkFooter is aligned to MIN_ALIGN. My thinking is:
#[repr(align(16))]
struct EmptyChunkFooter(ChunkFooter);
const _EMPTY_CHUNK_FOOTER_ALIGN_IS_CHUNK_ALIGN: () = {
assert!(mem::align_of::<EmptyChunkFooter>() == CHUNK_ALIGN);
};But this means we lose out on #[repr(transparent)] and its guarantees.
There was a problem hiding this comment.
Mhm I'm not sure if it should apply to
ChunkFooterorEmptyChunkFootersince there is already this assertion:...
which implies that
ChunkFooteris aligned toMIN_ALIGN.
Wait -- that assertion implies that it is not over-aligned (which would be an issue in other parts of the code) but it doesn't imply that it is not under-aligned (which is the issue here).
I think changing that assertion to == instead of <= (along with the repr(align(16)) on ChunkFooter) would resolve things here.
Putting repr(align(16)) on EmptyChunkFooter only would still allow for a non-empty ChunkFooter to be aligned for its own alignment but not to be aligned for the bump's MIN_ALIGN, so I think we really do want the repr(align(16)) on ChunkFooter.
There was a problem hiding this comment.
Agreed. I have fixed that, and it seems there is already debug_assert to make sure the footer is properly aligned to CHUNK_ALIGN:
Line 900 in 0b12966
Everything should be good now.
|
The fuzzer I'm writing uses cargo-fuzz which in turn uses the libfuzzer backend. It essentially takes some random-ish inputs, and transform them into methods using the Arbitrary trait, where each method denotes some operation done on Bump, e.g. allocate new string or check an allocated content's content has not been changed. Unlike unit/property testing, it is throwing everything at it, trying to cover as many combinations of paths as possible. I am trying to improve its coverage which is just under 50% at the moment. I will add the regression tests soon. |
Caught this while trying to add fuzzing which I will PR later.
Minimal reproduction: