Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 4002a7d

Browse files
rgommersstefanv
rgommers
and
stefanv
committed
BLD: enable building NumPy with Meson
This enables building with NumPy on Linux and macOS. Windows support should be complete to, but is untested as of now and may need a few tweaks. This contains: - A set of `meson.build` files and related code generation script tweaks, header templates, etc. - One CI job on Linux - Basic docs on using Meson to build NumPy (not yet integrated in the html docs, it's too early for that - this is for early adopters right now). The build should be complete, with the major exception of SIMD support. The full test suite passes. See gh-22546 for the tracking issue with detailed notes on the plan for switching NumPy to Meson as its build system. Co-authored-by: Stefan van der Walt <[email protected]>
1 parent 632f573 commit 4002a7d

39 files changed

Lines changed: 2549 additions & 50 deletions

‎.github/workflows/linux_meson.yml‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
name: Test Meson build (Linux)
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
- maintenance/**
8+
pull_request:
9+
branches:
10+
- main
11+
- maintenance/**
12+
13+
defaults:
14+
run:
15+
shell: bash
16+
17+
env:
18+
PYTHON_VERSION: 3.11
19+
20+
concurrency:
21+
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
22+
cancel-in-progress: true
23+
24+
permissions:
25+
contents: read # to fetch code (actions/checkout)
26+
27+
jobs:
28+
meson_devpy:
29+
runs-on: ubuntu-latest
30+
steps:
31+
- uses: actions/checkout@v3
32+
with:
33+
submodules: recursive
34+
fetch-depth: 0
35+
- uses: actions/setup-python@v4
36+
with:
37+
python-version: ${{ env.PYTHON_VERSION }}
38+
- name: Install dependencies
39+
run: |
40+
pip install -r build_requirements.txt
41+
sudo apt-get install -y libopenblas-serial-dev
42+
- name: Build
43+
shell: 'script -q -e -c "bash --noprofile --norc -eo pipefail {0}"'
44+
env:
45+
TERM: xterm-256color
46+
run:
47+
./dev.py build -- --werror
48+
- name: Check build-internal dependencies
49+
run:
50+
ninja -C build -t missingdeps
51+
- name: Check installed test and stub files
52+
run:
53+
python tools/check_installed_files.py $(find ./build-install -path '*/site-packages/numpy')
54+
- name: Test
55+
shell: 'script -q -e -c "bash --noprofile --norc -eo pipefail {0}"'
56+
env:
57+
TERM: xterm-256color
58+
run: |
59+
pip install pytest hypothesis typing_extensions
60+
./dev.py test

‎.gitignore‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,8 +61,11 @@ GTAGS
6161

6262
# Python files #
6363
################
64-
# setup.py working directory
64+
# meson build/installation directories
6565
build
66+
build-install
67+
# meson python output
68+
.mesonpy-native-file.ini
6669
# sphinx build directory
6770
_build
6871
# setup.py dist directory

‎build_requirements.txt‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
meson-python>=0.10.0
2+
Cython>=0.29.30,<3.0
3+
wheel==0.37.0
4+
ninja
5+
git+https://github.com/scientific-python/devpy

‎building_with_meson.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# Building with Meson
2+
3+
_Note: this is for early adopters. It has been tested on Linux and macOS, and
4+
with Python 3.9-3.12. Windows will be tested soon. There is one CI job to keep
5+
the build stable. This may have rough edges, please open an issue if you run
6+
into a problem._
7+
8+
### Developer build
9+
10+
**Install build tools:** Use one of:
11+
12+
- `mamba create -f environment.yml
13+
- `pip install -r build_requirements.txt # also make sure you have pkg-config and the usual system dependencies for NumPy`
14+
15+
**Compile and install:** `./dev.py build`
16+
17+
This builds in the `build/` directory, and installs into the `build-install` directory.
18+
19+
Then run the test suite or a shell via `dev.py`:
20+
```
21+
./dev.py test
22+
./dev.py ipython
23+
```
24+
25+
Alternatively, to use the package, add it to your `PYTHONPATH`:
26+
```
27+
export PYTHONPATH=${PWD}/build/lib64/python3.10/site-packages # may vary
28+
pytest --pyargs numpy
29+
```
30+
31+
32+
### pip install
33+
34+
Note that `pip` will use the default build system, which is (as of now) still
35+
`numpy.distutils`. In order to switch that default to Meson, uncomment the
36+
`build-backend = "mesonpy"` line at the top of `pyproject.toml`.
37+
38+
After that is done, `pip install .` or `pip install --no-build-isolation .`
39+
will work as expected. As does building an sdist or wheel with `python -m build`.
40+
Note, however, that `pip install -e .` (in-place developer install) does not!
41+
Use `dev.py` instead (see above).
42+
43+
44+
45+
### Workaround for a hiccup on Fedora
46+
47+
- Fedora does not distribute `openblas.pc`. Install the following file in `~/lib/pkgconfig/openblas.pc`:
48+
49+
```
50+
prefix=/usr
51+
includedir=${prefix}/include
52+
libdir=${prefix}/lib64
53+
54+
Name: openblas
55+
Description: OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version
56+
Version: 0.3.19
57+
Cflags: -I${includedir}/openblas
58+
Libs: -L${libdir} -lopenblas
59+
```
60+
61+
Then build with:
62+
63+
```
64+
./dev.py build -- -Dpkg_config_path=${HOME}/lib/pkgconfig
65+
```

‎dev.py‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
#!/usr/bin/env python
2+
#
3+
# Example stub for running `python -m dev.py`
4+
#
5+
# Copy this into your project root.
6+
7+
import os
8+
import sys
9+
import runpy
10+
11+
sys.path.remove(os.path.abspath(os.path.dirname(sys.argv[0])))
12+
try:
13+
runpy.run_module("devpy", run_name="__main__")
14+
except ImportError:
15+
print("Cannot import devpy; please install it using")
16+
print()
17+
print(" pip install git+https://github.com/scientific-python/devpy")
18+
print()
19+
sys.exit(1)

‎doc/source/f2py/buildtools/meson.rst‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,7 @@ build system like ``meson``. We will acquire this by:
2828
Now, consider the following ``meson.build`` file for the ``fib`` and ``scalar``
2929
examples from :ref:`f2py-getting-started` section:
3030

31-
.. literalinclude:: ./../code/meson.build
32-
:language: meson
31+
.. literalinclude:: ../code/meson.build
3332

3433
At this point the build will complete, but the import will fail:
3534

@@ -93,8 +92,7 @@ for reasons discussed in :ref:`f2py-bldsys`.
9392
However, we can augment our workflow in a straightforward to take into account
9493
files for which the outputs are known when the build system is set up.
9594

96-
.. literalinclude:: ./../code/meson_upd.build
97-
:language: meson
95+
.. literalinclude:: ../code/meson_upd.build
9896

9997
This can be compiled and run as before.
10098

‎doc/source/f2py/code/meson.build‎

Lines changed: 18 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,35 @@
11
project('f2py_examples', 'c',
22
version : '0.1',
3-
default_options : ['warning_level=2'])
3+
license: 'BSD-3',
4+
meson_version: '>=0.64.0',
5+
default_options : ['warning_level=2'],
6+
)
47

58
add_languages('fortran')
69

710
py_mod = import('python')
8-
py3 = py_mod.find_installation('python3')
9-
py3_dep = py3.dependency()
10-
message(py3.path())
11-
message(py3.get_install_dir())
11+
py = py_mod.find_installation(pure: false)
12+
py_dep = py.dependency()
1213

13-
incdir_numpy = run_command(py3,
14+
incdir_numpy = run_command(py,
1415
['-c', 'import os; os.chdir(".."); import numpy; print(numpy.get_include())'],
1516
check : true
1617
).stdout().strip()
1718

18-
incdir_f2py = run_command(py3,
19+
incdir_f2py = run_command(py,
1920
['-c', 'import os; os.chdir(".."); import numpy.f2py; print(numpy.f2py.get_include())'],
2021
check : true
2122
).stdout().strip()
2223

2324
inc_np = include_directories(incdir_numpy, incdir_f2py)
2425

25-
py3.extension_module('fib2',
26-
'fib1.f',
27-
'fib2module.c',
28-
incdir_f2py+'/fortranobject.c',
29-
include_directories: inc_np,
30-
dependencies : py3_dep,
31-
install : true)
26+
py.extension_module('fib2',
27+
[
28+
'fib1.f',
29+
'fib2module.c', # note: this assumes f2py was manually run before!
30+
],
31+
incdir_f2py / 'fortranobject.c',
32+
include_directories: inc_np,
33+
dependencies : py_dep,
34+
install : true
35+
)

‎doc/source/f2py/code/meson_upd.build‎

Lines changed: 19 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,38 @@
11
project('f2py_examples', 'c',
22
version : '0.1',
3-
default_options : ['warning_level=2'])
3+
license: 'BSD-3',
4+
meson_version: '>=0.64.0',
5+
default_options : ['warning_level=2'],
6+
)
47

58
add_languages('fortran')
69

710
py_mod = import('python')
8-
py3 = py_mod.find_installation('python3')
9-
py3_dep = py3.dependency()
10-
message(py3.path())
11-
message(py3.get_install_dir())
11+
py = py_mod.find_installation(pure: false)
12+
py_dep = py.dependency()
1213

13-
incdir_numpy = run_command(py3,
14+
incdir_numpy = run_command(py,
1415
['-c', 'import os; os.chdir(".."); import numpy; print(numpy.get_include())'],
1516
check : true
1617
).stdout().strip()
1718

18-
incdir_f2py = run_command(py3,
19+
incdir_f2py = run_command(py,
1920
['-c', 'import os; os.chdir(".."); import numpy.f2py; print(numpy.f2py.get_include())'],
2021
check : true
2122
).stdout().strip()
2223

2324
fibby_source = custom_target('fibbymodule.c',
24-
input : ['fib1.f'], # .f so no F90 wrappers
25-
output : ['fibbymodule.c', 'fibby-f2pywrappers.f'],
26-
command : [ py3, '-m', 'numpy.f2py', '@INPUT@',
27-
'-m', 'fibby', '--lower'])
25+
input : ['fib1.f'], # .f so no F90 wrappers
26+
output : ['fibbymodule.c', 'fibby-f2pywrappers.f'],
27+
command : [py, '-m', 'numpy.f2py', '@INPUT@', '-m', 'fibby', '--lower']
28+
)
2829

2930
inc_np = include_directories(incdir_numpy, incdir_f2py)
3031

31-
py3.extension_module('fibby',
32-
'fib1.f',
33-
fibby_source,
34-
incdir_f2py+'/fortranobject.c',
35-
include_directories: inc_np,
36-
dependencies : py3_dep,
37-
install : true)
32+
py.extension_module('fibby',
33+
['fib1.f', fibby_source],
34+
incdir_f2py / 'fortranobject.c',
35+
include_directories: inc_np,
36+
dependencies : py_dep,
37+
install : true
38+
)

‎environment.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,10 @@ dependencies:
1313
- openblas
1414
- nomkl
1515
- setuptools=59.2.0
16+
- meson >= 0.64.0
17+
- ninja
18+
- pkg-config # note: not available on Windows, comment out this line
19+
- meson-python
1620
# For testing
1721
- pytest
1822
- pytest-cov

‎generate_version.py‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Note: This file has to live next to versioneer.py or it will not work
2+
import argparse
3+
import os
4+
5+
import versioneer
6+
7+
8+
def write_version_info(path):
9+
vinfo = versioneer.get_versions()
10+
full_version = vinfo['version']
11+
git_revision = vinfo['full-revisionid']
12+
13+
if os.environ.get("MESON_DIST_ROOT"):
14+
path = os.path.join(os.environ.get("MESON_DIST_ROOT"), path)
15+
16+
with open(path, "w") as f:
17+
f.write("def get_versions():\n")
18+
f.write(" return {\n")
19+
f.write(f" 'full-revisionid': '{git_revision}',\n")
20+
f.write(f" 'version': '{full_version}'\n")
21+
f.write("}")
22+
23+
24+
def main():
25+
parser = argparse.ArgumentParser()
26+
parser.add_argument(
27+
"-o", "--outfile", type=str, help="Path to write version info to"
28+
)
29+
args = parser.parse_args()
30+
31+
if not args.outfile.endswith(".py"):
32+
raise ValueError(
33+
f"Output file must be a Python file. "
34+
f"Got: {args.outfile} as filename instead"
35+
)
36+
37+
write_version_info(args.outfile)
38+
39+
40+
main()

0 commit comments

Comments
 (0)