feat: Bounds checking for Flatbuf should be enabled in the default build #47
Conversation
|
Documentation preview URL: https://CurtHagenlocher.github.io/arrow-dotnet If the preview URL doesn't work, you may need to configure your fork repository for preview. |
adamreeve
left a comment
There was a problem hiding this comment.
Looks good to me thanks Curt.
| #if ENABLE_SPAN_T && UNSAFE_BYTEBUFFER | ||
| public unsafe string GetStringUTF8(int startPos, int len) | ||
| { | ||
| AssertOffsetAndLength(startPos, len); |
There was a problem hiding this comment.
For reference, this has been added in the upstream Flatbuffers project but not yet released: google/flatbuffers#8673
So we should make sure if we upgrade the bundled Flatbuffers code we do so after that change is released or add it in manually again. The new test should catch this though.
| <PropertyGroup> | ||
| <AllowUnsafeBlocks>true</AllowUnsafeBlocks> | ||
| <DefineConstants>$(DefineConstants);UNSAFE_BYTEBUFFER;BYTEBUFFER_NO_BOUNDS_CHECK;ENABLE_SPAN_T</DefineConstants> | ||
| <DefineConstants>$(DefineConstants);UNSAFE_BYTEBUFFER;ENABLE_SPAN_T</DefineConstants> |
There was a problem hiding this comment.
Just checking we're happy with leaving UNSAFE_BYTEBUFFER enabled? As far as I understand, bounds checks should prevent any invalid reads and this should be safe in theory, but it means bugs in the Flatbuffers implementation (like missing bounds checks) could still cause memory safety issues. It sounds like this can increase performance significantly though so is worth keeping enabled (https://flatbuffers.dev/languages/c_sharp/#conditional-compilation-symbols).
There was a problem hiding this comment.
I didn't try benchmarking without UNSAFE_BYTEBUFFER but I did audit all the code and confirmed that there was only one missing bounds check.
What's Changed
Enables bounds checking for Flatbuf as input can't generally be trusted.
Adds a test for a malformed column name length.
Makes some fixes required to be able to benchmark the change and demonstrate that there's no significant regression.
Closes #48.