Thanks to visit codestin.com
Credit goes to github.com

Skip to content

refactor: use zerocopy instead of bincode#38

Merged
Oakchris1955 merged 7 commits into
Oakchris1955:mainfrom
lukaslihotzki:zerocopy
Feb 18, 2026
Merged

refactor: use zerocopy instead of bincode#38
Oakchris1955 merged 7 commits into
Oakchris1955:mainfrom
lukaslihotzki:zerocopy

Conversation

@lukaslihotzki
Copy link
Copy Markdown
Contributor

I think that the zerocopy library is a good fit to replace bincode (#33). Benefits:

  • No longer depend on obsolete libraries. zerocopy is already in Cargo.lock, so this PR doesn't actually add a new dependency. Also, zerocopy has 2 maintainers in crates.io, so the chances it will be maintained in the future are somewhat higher.
  • No run-time errors for the actual conversions. This simplifies the error handling. When you have arrays (with fixed size), zerocopy::transmute! even checks the size during compile time. For slices, compile-time size checks are impossible. In this case, indexing (such as [..BPBFAT_SIZE]) was used before, panicking on too small slices. To preserve that panic behavior, I used .unwrap() with zerocopy. (Too small size is the only reason why read_from_prefix or similar would return the error variant.)
  • Performance should be improved, because zerocopy::transmute! compiles to a no-op, and read_from_prefix only does the size check.

Challenges:

  • bitfields and bitflags are not well-integrated to zerocopy. For bitflags, there was only a single use which I replaced with a manual zerocopy-compatible implementation. For bitfields, I added the Bitfield wrapper type that does the conversion. Having zerocopy support in bitfields (or the other way round) would be nice, but that doesn't exist.
  • With the zerocopy approach, reserved space needs to be defined in the struct, and cannot be handled in ser/de.

@lukaslihotzki
Copy link
Copy Markdown
Contributor Author

I forgot to add the file. Please re-approve.

@Oakchris1955
Copy link
Copy Markdown
Owner

There seems to be an issue with your code using the PhantomData struct from the std module instead from core. Please fix it and I will re-approve the next run.

@Oakchris1955
Copy link
Copy Markdown
Owner

Should be good to merge. I'd like to take a better look at it before merging tho, so I will probably merge this during the coming weekend.

@Oakchris1955 Oakchris1955 self-assigned this Feb 10, 2026
@Oakchris1955 Oakchris1955 self-requested a review February 14, 2026 16:14
@Oakchris1955
Copy link
Copy Markdown
Owner

I found a way to integrate bitflags with zerocopy thanks to this example. I will try to check if I can do that with the Bitfield struct too. I have a question to make first however, since I am not that familiar with zerocopy. All those #[repr(C)] are there because Rust reorders the struct fields and that messes with zerocopy, right?

@lukaslihotzki
Copy link
Copy Markdown
Contributor Author

Nice to see that bitflags can be used with zerocopy.

All those #[repr(C)] are there because Rust reorders the struct fields and that messes with zerocopy, right?

Yes, #[repr(C)] is needed to prevent field reordering, which would break simple-fatfs.

zerocopy itself supports reordered fields, and the guarantees that its traits provide (such as FromBytes means that any bit pattern is valid) still holds when reordering is present. However, the types cannot be used for proper serialization/deserialization when the field ordering is random. (It should work though as long as the data is only ever read and write by the same binary, but this isn't helpful for simple-fatfs.)

@Oakchris1955
Copy link
Copy Markdown
Owner

I think we are done here. I would like to get rid of the get methods on EntryCreationTime and EntryLastAccessed, but I don't see an easy day to do so. If you don't want to make any changes, I will merge this into main

@lukaslihotzki
Copy link
Copy Markdown
Contributor Author

I'm not sure if the 19e4581 broke big endian targets, because the ordering in the bitfield attribute (Lsb) stands for "least-significant bit" and that may not affect byte order. I'm currently trying to run the tests on an (emulated) big endian target.

@Oakchris1955
Copy link
Copy Markdown
Owner

If you are talking about the FAT32ExtendedFlags struct, the Lsb attribute is used by the bitfield trait. Both bincode and zerocopy should be treating this as a little-endian u16, and I don't see a reason for why this wouldn't be the case. Furthermore, I don't think simple-fatfs works correctly on big endian targets, due to the String::from_utf16 call in string_from_lfn treating the byte slice as native-endian

@lukaslihotzki
Copy link
Copy Markdown
Contributor Author

zerocopy does not do any endian conversion, except when using a zerocopy::byteorder type inside the struct (such as zerocopy::byteorder::little_endian::U16). zerocopy FromBytes and IntoBytes just expose the native struct bytes as a byte slice, which means that it will have native endianness if you use the native types (such as u16). So to ensure correct behavior on big endian targets, you need to use zerocopy::byteorder::little_endian::U16 inside the bitfield, which I implemented with the Bitfield wrapper.

I would like to have this crate working on big endian targets, but if it's currently broken, you can of course merge this PR as-is and then I may make a big endian fix PR in the future.

@lukaslihotzki
Copy link
Copy Markdown
Contributor Author

I have found a way how to ensure the correct endianness of bitfields with some extra bitfield macro attributes. Also, I fixed file_size to be little_endian::U32. With these changes, all tests pass on big endian.

By the way, LFN with UTF-16 seem to work fine. The endianness conversion is in https://github.com/Oakchris1955/simple-fatfs/blob/9bf0dccf30ac93d467995095c69eb0326d8c9b17/src/fat/direntry/ser_de.rs (line 59 for reading and line 110 for writing). string_from_lfn then gets a correct native u16 slice. To prevent such confusion, it may make sense to express endianness in the type system, using u16 for native endianness and little_endian::U16 for little endian values.

I think that this PR can be merged now. I have an additional change to replace the unsafe code with zerocopy, but I think this should be a future PR.

@Oakchris1955
Copy link
Copy Markdown
Owner

Oakchris1955 commented Feb 15, 2026

Before I merge,

With these changes, all tests pass on big endian.

Could you open an issue about big-endian tests and describe the process which you use to emulate a big endian target? Best case I'd like to run the tests on a big-endian Github Actions workflow, worst case locally on my machine.

@Oakchris1955 Oakchris1955 merged commit 2c0a3af into Oakchris1955:main Feb 18, 2026
3 checks passed
@Oakchris1955
Copy link
Copy Markdown
Owner

@lukaslihotzki I'd like to thank you for your contribution. If you find any other issues with simple-fatfs or anothet way to improve the project, don't hesitate to open an issue or a pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants