Tapioca DSL Compiler #975
Tapioca DSL Compiler #975
Conversation
|
On second thought, and considering that Tapioca has a disclaimer about their API stability and no mechanism for API versioning as far as I can tell, perhaps it's better for the compiler to live in a separate gem for now. It could declare a minimum version on ffi to ensure it can generate good types, and it can declare a dependency on Tapioca to ensure that API versions match. I'll leave the second commit up for now as an illustration but I suggest the final PR would consist of just the first commit (afaaa53) unless anyone feels strongly another way. |
lib/tapioca/dsl/compilers/ffi.rb
Outdated
| constant.class_variables.each do |class_var| | ||
| candidate = constant.class_variable_get(class_var) | ||
| if candidate.is_a?(::FFI::Function) | ||
| method_name = class_var.to_s.gsub(/\A@*/, "") |
There was a problem hiding this comment.
Accessing internal variables of FFI is risky. I think this is a good reason to add this compiler directly into ffi.
On the other hand I wonder, if we shouldn't make a public API to query this information and for other two cases below. Then I would prefer to store the compiler in another gem.
There was a problem hiding this comment.
It feels pretty hacky to me, I think new method(s) to query what is needed is much better.
For instance this logic is unlikely to work on the TruffleRuby backend of ffi (IIRC no class variables are used and I consider them deprecated).
There was a problem hiding this comment.
Thanks for your feedback both. All good points. I've pushed a change that exposes attached functions, getters and setters along with metadata in the following format (up for discussion):
- For getters and setters, a map from symbol (
:fooor:foo=) to type class (e.g.FFI::Type::STRING) - For functions, a map from symbol to an array of the shape
[param_types, result_type]mimicking the arguments toattach_function, again using type classes. - For variadic functions, the same with symbol
:varargstacked on the end of theparam_types.
I'm a bit unhappy with param_types containing a mix of classes (FFI::Type::String) and the :varargs symbol. It makes me wonder if maybe the values should contain symbols instead (e.g. :string). However, we'd still have a mix when e.g. structure references are used so I'm not sure that would be an improvement. It would also force me to duplicate some of the type resolution code from FFI in the compiler.
Let me know what you think.
There was a problem hiding this comment.
PS I haven't tested the latest changes to the DSL compiler yet, I've added them only for illustration.
There was a problem hiding this comment.
It just occurred to me that since getters and setters are functions, we might as well roll them into the attached_functions data, wdyt?
Remove direct exposure of FunctionType. Also, remove Tapioca DSL compiler code paths handling older ffi versions.
…o be attached This allows to freeze the FFI::Function immediately, so that it is shareable by Ractor without the need to freeze the module explicit. To make it shareable the typedef hash used for variadic functions is duplicated and made frozen. This creates a small compatibility issue: Only typedefs defined above the variadic function can be used by that function. If a typedef is created after the definition of the variadic function, then this typedef can no longer be used as parameter to that variadic function. Also fix the retrieval of simple (non-struct) global variables per #attached_variables. Closes ffi#975
…o be attached This allows to freeze the FFI::Function immediately, so that it is shareable by Ractor without the need to freeze the module explicit. To make it shareable the typedef hash used for variadic functions is duplicated and made frozen. This creates a small compatibility issue: Only typedefs defined above the variadic function can be used by that function. If a typedef is created after the definition of the variadic function, then this typedef can no longer be used as parameter to that variadic function. Also fix the retrieval of simple (non-struct) global variables per #attached_variables. Closes ffi#975
|
Sorry, this fell through the cracks. I don't have my head in it at all at the moment but from a quick look it seems like it'll do the trick, thanks! |
This adds a Tapioca DSL compiler for FFI declarations, to generate RBI files with type information for Sorbet.
The first commit exposes some information that the compiler needs to be able to generate types. Without this information the types would be a lot less precise. The second commit adds the compiler itself.
The compiler is picked up automatically by Tapioca when this gem is installed in the project in which Tapioca is run.
I had originally planned to open a PR for only the first commit here and contribute the compiler to the Tapioca repo, but it occurred to me that here is a better place for it. One advantage of having it here is that it prevents people from using the compiler with an older FFI version, which would generate disappointing types.
I have a full test suite for the compiler sitting here but it was written for the Tapioca test harness, I'll have to adapt it to rspec. Before I do that, is there any interest in this at all? And if not, would you accept a PR with just the first commit?