Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Use pybind11 for C/C++ extensions #23787

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from
Closed

Use pybind11 for C/C++ extensions #23787

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8381ef9
Use pybind11 for qhull wrapper
ianthomas23 Aug 28, 2022
f2f597f
Use pybind11 for tri module
ianthomas23 Aug 31, 2022
cdfaf20
Add CI dependencies
ianthomas23 Aug 31, 2022
b15fd9c
Fix module wrapper
ianthomas23 Sep 1, 2022
293dffc
Avoid using -std=c++11 when compiling C files
ianthomas23 Sep 2, 2022
7da3539
Address review comments
ianthomas23 Sep 2, 2022
f8226d8
Try altering CI so pybind11 extensions build OK
ianthomas23 Sep 2, 2022
2f8cfec
Remove C++ flags when compiling C files in qhull extension
ianthomas23 Sep 4, 2022
846f4c7
More CI installation of pybind11
ianthomas23 Sep 4, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .appveyor.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ install:
- echo %PYTHON_VERSION% %TARGET_ARCH%
# Install dependencies from PyPI.
- python -m pip install --upgrade -r requirements/testing/all.txt %EXTRAREQS% %PINNEDVERS%
- python -m pip install "pybind11<2.10"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the pin?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't recall the precise reason, presumably I was experimenting when the build didn't work. But it doesn't matter now, since merging of #24102 we don't need to preinstall pybind11 for editable installs so changes like this can be reverted.

# Install optional dependencies from PyPI.
# Sphinx is needed to run sphinxext tests
- python -m pip install --upgrade sphinx
Expand Down
2 changes: 2 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ commands:
python -m pip install --upgrade --user pip
python -m pip install --upgrade --user wheel
python -m pip install --upgrade --user 'setuptools!=60.6.0'
python -m pip install pybind11

doc-deps-install:
parameters:
Expand All @@ -102,6 +103,7 @@ commands:
python -m pip install --user \
numpy<< parameters.numpy_version >> codecov coverage \
-r requirements/doc/doc-requirements.txt
python -m pip install pybind11

mpl-install:
steps:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ jobs:
# Install dependencies from PyPI.
python -m pip install --upgrade $PRE \
'contourpy>=1.0.1' cycler fonttools kiwisolver numpy packaging \
pillow pyparsing python-dateutil setuptools-scm \
pillow pybind11 pyparsing python-dateutil setuptools-scm \
-r requirements/testing/all.txt \
${{ matrix.extra-requirements }}

Expand Down
1 change: 1 addition & 0 deletions azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ stages:

- bash: |
python -m pip install --upgrade pip
python -m pip install pybind11
python -m pip install -r requirements/testing/all.txt -r requirements/testing/extra.txt ||
[[ "$PYTHON_VERSION" = 'Pre' ]]
displayName: 'Install dependencies with pip'
Expand Down
23 changes: 12 additions & 11 deletions lib/matplotlib/tests/test_triangulation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1167,43 +1167,43 @@ def test_internal_cpp_api():
# C++ Triangulation.
with pytest.raises(
TypeError,
match=r'function takes exactly 7 arguments \(0 given\)'):
match=r'__init__\(\): incompatible constructor arguments.'):
mpl._tri.Triangulation()

with pytest.raises(
ValueError, match=r'x and y must be 1D arrays of the same length'):
mpl._tri.Triangulation([], [1], [[]], None, None, None, False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if None is not accepted anymore this warrants a changelog entry

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the internal C++ Triangulation API (matplotllib._tri.Triangulation) that is used by the Python Triangulation class (matplotlib.tri.Triangulation). It is all private and nobody should be using it, so it doesn't need a changelog entry.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Python Triangulation class still accepts None for triangles, etc.

mpl._tri.Triangulation([], [1], [[]], (), (), (), False)

x = [0, 1, 1]
y = [0, 0, 1]
with pytest.raises(
ValueError,
match=r'triangles must be a 2D array of shape \(\?,3\)'):
mpl._tri.Triangulation(x, y, [[0, 1]], None, None, None, False)
mpl._tri.Triangulation(x, y, [[0, 1]], (), (), (), False)

tris = [[0, 1, 2]]
with pytest.raises(
ValueError,
match=r'mask must be a 1D array with the same length as the '
r'triangles array'):
mpl._tri.Triangulation(x, y, tris, [0, 1], None, None, False)
mpl._tri.Triangulation(x, y, tris, [0, 1], (), (), False)

with pytest.raises(
ValueError, match=r'edges must be a 2D array with shape \(\?,2\)'):
mpl._tri.Triangulation(x, y, tris, None, [[1]], None, False)
mpl._tri.Triangulation(x, y, tris, (), [[1]], (), False)

with pytest.raises(
ValueError,
match=r'neighbors must be a 2D array with the same shape as the '
r'triangles array'):
mpl._tri.Triangulation(x, y, tris, None, None, [[-1]], False)
mpl._tri.Triangulation(x, y, tris, (), (), [[-1]], False)

triang = mpl._tri.Triangulation(x, y, tris, None, None, None, False)
triang = mpl._tri.Triangulation(x, y, tris, (), (), (), False)

with pytest.raises(
ValueError,
match=r'z array must have same length as triangulation x and y '
r'array'):
match=r'z must be a 1D array with the same length as the '
r'triangulation x and y arrays'):
triang.calculate_plane_coefficients([])

with pytest.raises(
Expand All @@ -1215,7 +1215,7 @@ def test_internal_cpp_api():
# C++ TriContourGenerator.
with pytest.raises(
TypeError,
match=r'function takes exactly 2 arguments \(0 given\)'):
match=r'__init__\(\): incompatible constructor arguments.'):
mpl._tri.TriContourGenerator()

with pytest.raises(
Expand All @@ -1233,7 +1233,8 @@ def test_internal_cpp_api():

# C++ TrapezoidMapTriFinder.
with pytest.raises(
TypeError, match=r'function takes exactly 1 argument \(0 given\)'):
TypeError,
match=r'__init__\(\): incompatible constructor arguments.'):
mpl._tri.TrapezoidMapTriFinder()

trifinder = mpl._tri.TrapezoidMapTriFinder(triang)
Expand Down
8 changes: 6 additions & 2 deletions lib/matplotlib/tri/triangulation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,8 +120,12 @@ def get_cpp_triangulation(self):
from matplotlib import _tri
if self._cpp_triangulation is None:
self._cpp_triangulation = _tri.Triangulation(
self.x, self.y, self.triangles, self.mask, self._edges,
self._neighbors, not self.is_delaunay)
# For unset arrays use empty tuple which has size of zero.
self.x, self.y, self.triangles,
self.mask if self.mask is not None else (),
self._edges if self._edges is not None else (),
self._neighbors if self._neighbors is not None else (),
not self.is_delaunay)
return self._cpp_triangulation

def get_masked_triangles(self):
Expand Down
22 changes: 22 additions & 0 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,15 @@ def has_flag(self, flagname):
return True


# Wrapper for distutils.ccompiler.CCompiler._compile to remove C++-specific
# flags when compiling C files.
def compile_wrapper(compiler, obj, src, ext, cc_args, extra_postargs, pp_opts):
if src.lower().endswith(".c"):
extra_postargs = list(filter(lambda x: x[1:4] != "std",
extra_postargs))
compiler._compile_old(obj, src, ext, cc_args, extra_postargs, pp_opts)


class BuildExtraLibraries(setuptools.command.build_ext.build_ext):
def finalize_options(self):
self.distribution.ext_modules[:] = [
Expand Down Expand Up @@ -184,9 +193,20 @@ def build_extension(self, ext):
orig_build_temp = self.build_temp
self.build_temp = os.path.join(self.build_temp, ext.name)
try:
if ext.name == "matplotlib._qhull":
# For qhull extension some C++ flags must be removed before
# compiling C files.
from distutils.ccompiler import CCompiler
self.compiler._compile_old = self.compiler._compile
self.compiler._compile = compile_wrapper.__get__(
self.compiler, CCompiler)
super().build_extension(ext)
finally:
self.build_temp = orig_build_temp
if ext.name == "matplotlib._qhull" and hasattr(
self.compiler, "_compile_old"):
self.compiler._compile = self.compiler._compile_old
delattr(self.compiler, "_compile_old")


def update_matplotlibrc(path):
Expand Down Expand Up @@ -302,6 +322,8 @@ def make_release_tree(self, base_dir, files):
setup_requires=[
"certifi>=2020.06.20",
"numpy>=1.19",
"pybind11>=2.6",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a new build dependency, but is not a runtime dependency.

"setuptools>=42",
"setuptools_scm>=7",
],
install_requires=[
Expand Down
11 changes: 6 additions & 5 deletions setupext.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
import textwrap
import urllib.request

from pybind11.setup_helpers import Pybind11Extension
from setuptools import Distribution, Extension

_log = logging.getLogger(__name__)
Expand Down Expand Up @@ -432,10 +433,10 @@ def get_extensions(self):
add_libagg_flags_and_sources(ext)
yield ext
# qhull
ext = Extension(
ext = Pybind11Extension(
"matplotlib._qhull", ["src/_qhull_wrapper.cpp"],
cxx_std=11,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can specify the C++ standard to follow, so here we are explicitly restricting to C++11. This would probably be a configuration param at the top of the file to make it clear what C++ standard the project is currently working to.

define_macros=[("MPL_DEVNULL", os.devnull)])
add_numpy_flags(ext)
Qhull.add_flags(ext)
yield ext
# tkagg
Expand All @@ -452,12 +453,12 @@ def get_extensions(self):
add_libagg_flags(ext)
yield ext
# tri
ext = Extension(
ext = Pybind11Extension(
"matplotlib._tri", [
"src/tri/_tri.cpp",
"src/tri/_tri_wrapper.cpp",
])
add_numpy_flags(ext)
],
cxx_std=11)
yield ext
# ttconv
ext = Extension(
Expand Down
Loading