Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[MNT]: Pybind11 transition plan #23846

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
13 tasks done
ianthomas23 opened this issue Sep 9, 2022 · 6 comments · Fixed by #28857
Closed
13 tasks done

[MNT]: Pybind11 transition plan #23846

ianthomas23 opened this issue Sep 9, 2022 · 6 comments · Fixed by #28857
Milestone

Comments

@ianthomas23
Copy link
Member

ianthomas23 commented Sep 9, 2022

We intend to switch the C/C++ extension build system to use pybind11 as discussed in this week's call. There is a demonstration in PR #23787 showing that the approach works and listing the pros and cons. It is too much work to do in a single PR so will probably be done one extension at a time. This issue is intended to track the overall progress by editing this post to tick the relevant check box and add the PR number for each extension completed.

C/C++ extensions:

Other work required:

@QuLogic
Copy link
Member

QuLogic commented Mar 30, 2023

On a related note, has anyone tried out nanobind? It's probably a bit too new for us, but curious if anyone had any opinions on it.

@matthewfeickert
Copy link
Contributor

matthewfeickert commented Mar 30, 2023

On a related note, has anyone tried out nanobind? It's probably a bit too new for us, but curious if anyone had any opinions on it.

I just today migrated a (small) research project to nanobin v1.0.0.

As this project is array heavy nanobin is super nice for this as it allows you to just use the nb::ndarray API and avoid all the buffer stuff in pybind11 (which honestly isn't that bad, but if you don't have to do it yourself that's great).

Other advantages that are nice about nanobind (YMMV though project to project):

More concretely, benchmarks show ~2-3× faster compile time, ~3× smaller binaries, and up to ~8× lower runtime overheads compared to pybind11.

nanobind can exchange n-dimensional arrays (henceforth “ndarrays”) with popular array programming frameworks including NumPy, PyTorch, TensorFlow, and JAX. It supports zero-copy exchange using two protocols:

nanobind knows how to talk to each framework and takes care of all the nitty-gritty details.

My first impressions are that it is great!

@QuLogic
Copy link
Member

QuLogic commented Oct 5, 2023

* [ ]  `macosx` (developer needs access to macOS)

I'm not sure about this one, as it doesn't use NumPy, and is written in Objective C, not C or C++.

@greglucas
Copy link
Contributor

  • macosx (developer needs access to macOS)

I'm not sure about this one, as it doesn't use NumPy, and is written in Objective C, not C or C++.

I agree pybind doesn't seem like the right fit. I have wondered if it'd be worthwhile to transition to use pyobjc at all so we could write the extension in Python code... https://pyobjc.readthedocs.io/en/latest/
Of course, that would mean an extra dependency on macos and needing to learn how to write that style of Python.

@QuLogic
Copy link
Member

QuLogic commented Sep 6, 2024

The last PR #28785 does ft2font. After that there will be some clean up to remove the NumPy wrappers.

@QuLogic
Copy link
Member

QuLogic commented Sep 24, 2024

The last two PRs: #28856 takes care of porting remaining common functionality to pybind11, and #28857 removes NumPy and the helpers.

@QuLogic QuLogic linked a pull request Sep 24, 2024 that will close this issue
4 tasks
@QuLogic QuLogic added this to the v3.10.0 milestone Oct 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants