Thanks to visit codestin.com
Credit goes to github.com

Skip to content

atomic_base64_to_binary_safe is not really ready for production use #763

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
pauldreik opened this issue Apr 25, 2025 · 3 comments · Fixed by #774
Closed

atomic_base64_to_binary_safe is not really ready for production use #763

pauldreik opened this issue Apr 25, 2025 · 3 comments · Fixed by #774
Labels
bug Something isn't working

Comments

@pauldreik
Copy link
Collaborator

I write this here so it is visible. While writing a span overload and a fuzzer for atomic_base64_to_binary_safe, it seems like there are several problems with the implementation

  • it might hang in an infinite loop
  • it may give a different result than calling base64_to_binary_safe directly (even if there is no multi threaded access)

I have a branch where I work on this: https://github.com/simdutf/simdutf/tree/issue751_span_and_fuzz
@pauldreik pauldreik mentioned this issue Apr 25, 2025
Closed
@lemire lemire added the bug Something isn't working label Apr 25, 2025
@lemire
Copy link
Member

lemire commented Apr 25, 2025

Marking as a 'bug'.

@pauldreik pauldreik mentioned this issue Apr 26, 2025
Merged
@pauldreik
Copy link
Collaborator Author

pauldreik commented Apr 30, 2025 •
edited
Loading

Unfortunately I found another failing case on the current master 19ff889 (which has the fixes linked above included). I had to manually guide the fuzzer since the fuzzer now uses a different temp buffer size than the ordinarily build, but this reproduces as a unit test in the normal build:

// in file atomic_base64_tests.cpp
TEST(causes_mismatch_in_count) {
  const std::vector<unsigned char> base64(9194, 0x20);
  compare_decode(base64, 8224, simdutf::base64_default,
                 simdutf::last_chunk_handling_options::loose, false);
};

this fails with:

Checking implementation icelake
.......
lhs: r1.count = 9194
rhs: r2.count = 5460
r1.count

@lemire
Copy link
Member

lemire commented Apr 30, 2025

@pauldreik I am delaying the release.

@lemire lemire mentioned this issue Apr 30, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fixes issue 763 simdutf/simdutf
2 participants
@lemire @pauldreik