Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

emiltayl
Copy link
Contributor

@emiltayl emiltayl commented May 21, 2025

I've been busy the last few weeks, but now I am back with a bigger PR.

This PR contains a rewrite of middle::Type. I started to rewrite it because I found it hard to follow how the old Type worked. I realize that I might have just replaced something that I did not understand with something everybody else does not understand, but I have at least tried to separate out safe functionality (such as just describing libffi type signatures) from unsafe code to handle allocation of memory.

In addition, I have made some breaking changes. None of the changes should require major changes downstream (hopefully), but I am not the best judge over what breaking changes are acceptable and not. Please let me know if you have any input on changes that should be made or changes that I've made that should be undone.

Breaking changes

  • middle::Type is now an enum. It does not manage the memory of ffi_types for Cifs, which is handled by new FfiTypeArray and FfiType structs.
    • Cloning and dropping FfiTypes passes tests in middle::types with miri. The tests should probably be modified to use FfiTypeAray as well.
  • middle::Type does not have a void variant as ffi_type_void cannot be used for argument types in libffi. Instead, when creating a Cif, the result type is an Option<Type> where None represents a void return type.
    • () no longer implements CType. Instead, it implements CRetType, a new trait which is implemented for () and all types that implement CType.
  • Removed the RetType from the CType trait as it is no longer needed. It was previously needed, but become obsolete after Fixed out-of-bounds write in low::call #108 was merged.
  • There might be more subtle breaking changes that I have failed to noticed

Other changes

There are a few other changes I have made that I think it is worth to highlight to get eventual feedback.

Memory management

This crate previously used libc::malloc and libc::free to manage memory for ffi_type. This PR exchanges that for creating Boxes, Box::into_raw and Box::from_raw. With this PR, there is no longer any use of libc and it can be removed as a dependency.

Boxes come with their own set of problems, but this change should avoid problems with uniqueness by not creating Boxes except for allocating memory and from_raw when dropping them. This has the advantage that it allocates memory using the global allocator configured in projects using libffi-rs. In the future this should also make it fairly easy to allow the usage of custom (non-global) allocators if anyone should wish to do so.

I did consider using alloc::alloc to allocate memory, but was hesitant to do so as the documentation states that it "is expected to be deprecated".

Documentation of unsafe code

I have tried to add comments where it is useful to understand the memory management and why/when it is safe to do everything needed to manage memory for ffi_types. I am sure there are places where things could be documented better, and there are probably comments that are superfluous and only add clutter.

I will try to review the comments later, but currently I am blind on my own code.

Deprecation of Type functions

Previously, Types had to be constructed from functions such as Type::u8(). Since the new Type is an enum, this is no longer needed (for all types except structs). I chose to keep the old functions, but deprecate them as I find it better to refer directly to the Type enum variants when possible.

Unresolved issues

Does not currently compile with system feature flag on Mac due to missing long double symbols.

emiltayl and others added 9 commits May 11, 2025 10:54
`middle::Arg` accepts a reference and turns it into a pointer, throwing
away any lifetime information. This change adds a a marker to
`middle::Arg` with a lifetime so Rust is able to verify that the
provided argument lives long enough.
* Disable `complex` when compiling for Windows

Libffi does not include any code for complex types when `_WIN32` is
defined.

* Fix padding on structs in `libffi-sys`

* `ffi_raw` and `ffi_trampoline` does not have an explicit padding set
  by libffi.
* `ffi_closure` is padded to 8 bytes and has an extra padding field when
  compiling with MSVC for i686.

* Set the correct ABI for x86_64 MSVC

The default ABI when compiling for x86_64 with MSVC was set to GNUW64
instead of the expected WIN64. This caused an error when passing
`double`s to and from functions.

* Fixed warning about unused import on Windows

`std::path::{Path, PathBuf}` was imported in the build scripts of
libffi-sys, but not used on Windows. This moves the import to
`not_msvc.rs` to avoid the error message.

* Clean up and modify build script for msvc

* Use the full (relative) path for all files and folders to
  keep every file and folder path consistent.
* Use separate header files for x86, x86_64, and aarch64. See the bottom
  of the commit message for the commands used to generate the headers.
* Use compiler from `cc` crate instead of looking for `cl.exe` using
  `cc::windows_registry`. This will hopefully open up for
  cross-compiling for msvc targets in the future.
* Store the resulting assembly file from pre-processing in the target
  directory instead of inside `libffi-sys-rs/libffi/`.
* Define `FFI_STATIC_BUILD` when building for MSVC.
* Emit helpers for cargo for when to recompile libffi.

The headers were generated using Visual Studio 2022 and
msys2-x86_64-20250221.

The headers for x86_64 were generated using the following command:
`./configure --build x86_64-pc-msys --target x86_64-pc-msys --host x86_64-pc-msys --disable-dependency-tracking --with-pic --disable-shared --disable-docs CC="$(pwd)/msvcc.sh -m64" CXX="$(pwd)/msvcc.sh -m64" CPP="cl.exe -nologo -EP" CXXCPP="cl.exe -nologo -EP"`

The headers for i686 were generated using the following command:
`./configure --build i686-pc-msys --target i686-pc-msys --host i686-pc-msys --disable-dependency-tracking --with-pic --disable-shared --disable-docs CC="$(pwd)/msvcc.sh" CXX="$(pwd)/msvcc.sh" CPP="cl.exe -nologo -EP" CXXCPP="cl.exe -nologo -EP"`

The headers for aarch64 were generated using the following command:
`./configure --build x86_64-pc-msys --target aarch64-pc-msys --host aarch64-pc-msys --disable-dependency-tracking --with-pic --disable-shared --disable-docs CC="$(pwd)/msvcc.sh -m64" CXX="$(pwd)/msvcc.sh -m64" CPP="cl.exe -nologo -EP" CXXCPP="cl.exe -nologo -EP"`

* Select build process by `CARGO_CFG_TARGET_ENV`

`target_env` represent the host building the code for build scripts, and
does not necessarily reflect the target platform. This change uses
`CARGO_CFG_TARGET_ENV` to select the correct build process.

Currently, it is seemingly not possible to cross-compile libffi-sys for
MSVC, but it might be possible in the future.
Type tags are stored in the `type_` field of `ffi_type`s, which is a
`c_ushort`. `c_ushort` is an unsigned integer with at least 16 bits. On
most modern architectures, `c_ushort` is equivalent to `u16`.

This commit changes the types of the constants used for the `type_`
field to `u16` to avoid having to cast the constants when using them.
Removes duplicate definitions of the following symbols:

* libffi_sys::ffi_type_complex_longdouble
* libffi::middle::types::longdouble
* libffi::middle::types::complex_longdouble
* libffi::middle::Type::complex_longdouble
Added `ffi_type`s to `middle::types` that can be used when testing
`middle::Type` with miri.
This adds another test to `middle::types` that tests the behavior when
cloning and dropping `Type`s. The test can run with miri.
`longdouble` does not work with "system" feature on MacOS.
@arihant2math arihant2math requested a review from littledivy May 22, 2025 02:14
@emiltayl
Copy link
Contributor Author

Changing middle::Type from a struct to an enum should not have that much of an effect on crates using libffi as I understand it. It does not contain any fields or methods that are used by other crates to interact with the type.

I would say that the "most" breaking change is removing void as a Type. I like how it prevents setting argument types to void, which libffi does not support. For simple calls, this only requires putting return types inside a Some or setting it to None for void return types. For more advanced use cases this might require some larger changes, but I personally cannot see that this should be too big of a problem. While I think removing void provides the best external interface from this crate, I will leave it up to you to decide whether this is a breaking change that is acceptable or not.

Cifs in the middle and high modules had a `set_abi` function to set a
non-default ABI. This is not necessarily safe as the ABI is used for
initializing a cif in `ffi_prep_cif`. Changing the ABI after
initialization may cause problems when calling a function using the cif.

The `set_abi` functions were replaced with `_with_abi` variants for cif
constructors (`new` and `new_variadic`).
@arihant2math
Copy link
Collaborator

I'll have to take a closer look at the safety guarantees to see if that change is worth it.

@emiltayl emiltayl force-pushed the next-new-type branch 2 times, most recently from f6fa715 to fb8fac9 Compare June 8, 2025 19:13
This commit changes `middle::Type` from a smart pointer to `ffi_type` to
an enum that represents possible types that can be sent to, and returned
from libffi. Allocation of `ffi_type`s has been moved to a separate
`FfiType` struct. A new `FfiTypeArray` has also been created.

Functions that were previously used to create different `Type`s are
still present, but they are deprecated in favor of referring to `Type`
variants directly.

All allocations now happens using functionality from `alloc` instead of
using `libc::malloc` and `libc::free`. After this change, it is possible
to remove libc as a dependency.

`middle::Type` does not contain a `void` variant as it is not a valid
type for arguments in libffi. Instead, functions now accept an
`Option<Type>` for the return type, where `None` represents a void.

`high:CType` was slightly simplified as the `RetType` is no longer
needed after PR libffi-rs#108 was merged. For return types, a new `CRetType`
trait was added that is implemented for all types that implements
`CType` in addition to `()`.
@emiltayl
Copy link
Contributor Author

emiltayl commented Jun 9, 2025

I have thought some more about this, and I think something should change with types/cifs in middle for libffi-rs. As I have previously stated, ffi_type_void cannot be used for argument types in libffi. Yet, it seems that libffi does not necessarily return an error when a cif is created with a void argument. This sounds like undefined behavior (or at least undefined behavior-adjacent) to me.

I have three proposals for how this could be dealt with in middle.

  • The way it is done in this PR, by making it impossible to represent an "illegal" set of argument types.
  • Check the provided types when creating a cif in middle and return an error if a void is provided as an argument type.
  • Make any function creating a cif in middle unsafe and document that the caller needs to make sure that none of the argument types are void.

There might be other solutions that I have not thought of, but I think something should be changed to make the middle module as correct and safe as possible. I don't know if this PR is the best place to discuss this, or if it should be moved to a separate issue?

@arihant2math arihant2math force-pushed the next branch 2 times, most recently from 9989ec4 to ea037e5 Compare August 31, 2025 05:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants