Thanks to visit codestin.com
Credit goes to github.com

Skip to content

DOC: f2py rewrite with meson details #25483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Dec 24, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 190 additions & 0 deletions doc/source/f2py/buildtools/distutils-to-meson.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
.. _f2py-meson-distutils:


1 Migrating to ``meson``
------------------------

As per the timeline laid out in :ref:`distutils-status-migration`,
``distutils`` has ceased to be the default build backend for ``f2py``. This page
collects common workflows in both formats.

.. note::

This is a ****living**** document, `pull requests <https://numpy.org/doc/stable/dev/howto-docs.html>`_ are very welcome!

1.1 Baseline
~~~~~~~~~~~~

We will start out with a slightly modern variation of the classic Fibonnaci
series generator.

.. code:: fortran

! fib.f90
subroutine fib(a, n)
use iso_c_binding
integer(c_int), intent(in) :: n
integer(c_int), intent(out) :: a(n)
do i = 1, n
if (i .eq. 1) then
a(i) = 0.0d0
elseif (i .eq. 2) then
a(i) = 1.0d0
else
a(i) = a(i - 1) + a(i - 2)
end if
end do
end

This will not win any awards, but can be a reasonable starting point.

1.2 Compilation options
~~~~~~~~~~~~~~~~~~~~~~~

1.2.1 Basic Usage
^^^^^^^^^^^^^^^^^

This is unchanged:

.. code:: bash

python -m numpy.f2py -c fib.f90 -m fib
❯ python -c "import fib; print(fib.fib(30))"
[ 0 1 1 2 3 5 8 13 21 34
55 89 144 233 377 610 987 1597 2584 4181
6765 10946 17711 28657 46368 75025 121393 196418 317811 514229]

1.2.2 Specify the backend
^^^^^^^^^^^^^^^^^^^^^^^^^

.. tab-set::

.. tab-item:: Distutils
:sync: distutils

.. code-block:: bash

python -m numpy.f2py -c fib.f90 -m fib --backend distutils

This is the default for Python versions before 3.12.

.. tab-item:: Meson
:sync: meson

.. code-block:: bash

python -m numpy.f2py -c fib.f90 -m fib --backend meson

This is the only option for Python versions after 3.12.

1.2.3 Pass a compiler name
^^^^^^^^^^^^^^^^^^^^^^^^^^

.. tab-set::

.. tab-item:: Distutils
:sync: distutils

.. code-block:: bash

python -m numpy.f2py -c fib.f90 -m fib --backend distutils --fcompiler=gfortran

.. tab-item:: Meson
:sync: meson

.. code-block:: bash

FC="gfortran" python -m numpy.f2py -c fib.f90 -m fib --backend meson

Native files can also be used.

Similarly, ``CC`` can be used in both cases to set the ``C`` compiler. Since the
environment variables are generally pretty common across both, so a small
sample is included below.

.. table::

+------------------------------------+-------------------------------+
| **Name** | **What** |
+------------------------------------+-------------------------------+
| FC | Fortran compiler |
+------------------------------------+-------------------------------+
| CC | C compiler |
+------------------------------------+-------------------------------+
| CFLAGS | C compiler options |
+------------------------------------+-------------------------------+
| FFLAGS | Fortran compiler options |
+------------------------------------+-------------------------------+
| LDFLAGS | Linker options |
+------------------------------------+-------------------------------+
| LD\ :sub:`LIBRARY`\ \ :sub:`PATH`\ | Library file locations (Unix) |
+------------------------------------+-------------------------------+
| LIBS | Libraries to link against |
+------------------------------------+-------------------------------+
| PATH | Search path for executables |
+------------------------------------+-------------------------------+
| LDFLAGS | Linker flags |
+------------------------------------+-------------------------------+
| CXX | C++ compiler |
+------------------------------------+-------------------------------+
| CXXFLAGS | C++ compiler options |
+------------------------------------+-------------------------------+


.. note::

For Windows, these may not work very reliably, so `native files <https://mesonbuild.com/Native-environments.html>`_ are likely the
best bet, or by direct `1.3 Customizing builds`_.

1.2.4 Dependencies
^^^^^^^^^^^^^^^^^^

Here, ``meson`` can actually be used to set dependencies more robustly.

.. tab-set::

.. tab-item:: Distutils
:sync: distutils

.. code-block:: bash

python -m numpy.f2py -c fib.f90 -m fib --backend distutils -llapack

Note that this approach in practice is error prone.

.. tab-item:: Meson
:sync: meson

.. code-block:: bash

python -m numpy.f2py -c fib.f90 -m fib --backend meson --dep lapack

This maps to ``dependency("lapack")`` and so can be used for a wide variety
of dependencies. They can be `customized further <https://mesonbuild.com/Dependencies.html>`_
to use CMake or other systems to resolve dependencies.

1.3 Customizing builds
~~~~~~~~~~~~~~~~~~~~~~

.. tab-set::

.. tab-item:: Distutils
:sync: distutils

.. code-block:: bash

python -m numpy.f2py -c fib.f90 -m fib --backend distutils --build-dir blah

This can be technically integrated with other codes, see :ref:`f2py-distutils`.

.. tab-item:: Meson
:sync: meson

.. code-block:: bash

python -m numpy.f2py -c fib.f90 -m fib --backend meson --build-dir blah

The resulting build can be customized via the
`Meson Build How-To Guide <https://mesonbuild.com/howtox.html>`_.
In fact, the resulting set of files can even be commited directly and used
as a meson subproject in a separate codebase.
8 changes: 7 additions & 1 deletion doc/source/f2py/buildtools/distutils.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@
Using via `numpy.distutils`
=============================

.. legacy::

``distutils`` has been removed in favor of ``meson`` see
:ref:`distutils-status-migration`.


.. currentmodule:: numpy.distutils.core

:mod:`numpy.distutils` is part of NumPy, and extends the standard Python
Expand Down Expand Up @@ -75,4 +81,4 @@ Extensions to ``distutils``

.. code-block:: bash

python -m numpy.f2py -c --help-fcompiler
python -m numpy.f2py -c --backend distutils --help-fcompiler
32 changes: 21 additions & 11 deletions doc/source/f2py/buildtools/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,21 @@ F2PY and Build Systems
In this section we will cover the various popular build systems and their usage
with ``f2py``.

.. versionchanged:: NumPy 1.26.x

The default build system for ``f2py`` has traditionally been through the
enhanced ``numpy.distutils`` module. This module is based on ``distutils``
which was removed in ``Python 3.12.0`` in **October 2023**. Like the rest of
NumPy and SciPy, ``f2py`` uses ``meson`` now, see
:ref:`distutils-status-migration` for some more details.

All changes to ``f2py`` are tested on SciPy, so their `CI configuration`_ is
always supported.


.. note::
**As of November 2021**

The default build system for ``F2PY`` has traditionally been through the
enhanced ``numpy.distutils`` module. This module is based on ``distutils`` which
will be removed in ``Python 3.12.0`` in **October 2023**; ``setuptools`` does not
have support for Fortran or ``F2PY`` and it is unclear if it will be supported
in the future. Alternative methods are thus increasingly more important.
See :ref:`f2py-meson-distutils` for migration information.


Basic Concepts
Expand Down Expand Up @@ -78,12 +85,12 @@ Signature files
their contents; which shifts the burden of checking for generated files onto
the build system.

.. note::
.. versionchanged:: NumPy ``1.22.4``

From NumPy ``1.22.4`` onwards, ``f2py`` will deterministically generate
wrapper files based on the input file Fortran standard (F77 or greater).
``--skip-empty-wrappers`` can be passed to ``f2py`` to restore the previous
behaviour of only generating wrappers when needed by the input .
``f2py`` will deterministically generate wrapper files based on the input
file Fortran standard (F77 or greater). ``--skip-empty-wrappers`` can be
passed to ``f2py`` to restore the previous behaviour of only generating
wrappers when needed by the input .


In theory keeping the above requirements in hand, any build system can be
Expand All @@ -104,5 +111,8 @@ Build Systems
meson
cmake
skbuild
distutils-to-meson

.. _`issue 20385`: https://github.com/numpy/numpy/issues/20385

.. _`CI configuration`: https://docs.scipy.org/doc/scipy/dev/toolchain.html#official-builds
19 changes: 12 additions & 7 deletions doc/source/f2py/buildtools/meson.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,25 @@
Using via ``meson``
===================

.. note::

Much of this document is now obsoleted, one can run ``f2py`` with
``--build-dir`` to get a skeleton ``meson`` project with basic dependencies
setup.

.. versionchanged:: 1.26.x

The default build system for ``f2py`` is now ``meson``, see
:ref:`distutils-status-migration` for some more details..

The key advantage gained by leveraging ``meson`` over the techniques described
in :ref:`f2py-distutils` is that this feeds into existing systems and larger
projects with ease. ``meson`` has a rather pythonic syntax which makes it more
comfortable and amenable to extension for ``python`` users.

.. note::

Meson needs to be at-least ``0.46.0`` in order to resolve the ``python`` include directories.


Fibonacci Walkthrough (F77)
Fibonacci walkthrough (F77)
===========================


We will need the generated ``C`` wrapper before we can use a general purpose
build system like ``meson``. We will acquire this by:

Expand Down
13 changes: 8 additions & 5 deletions doc/source/f2py/f2py-examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ F2PY examples
Below are some examples of F2PY usage. This list is not comprehensive, but can
be used as a starting point when wrapping your own code.

.. note::

The best place to look for examples is the `NumPy issue tracker`_, or the test
cases for ``f2py``

F2PY walkthrough: a basic extension module
------------------------------------------

Expand All @@ -31,11 +36,6 @@ Python just like any other extension module.
Creating a compiled extension module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. note::

This usage depends heavily on ``numpy.distutils``, see :ref:`f2py-bldsys`
for more details.

You can also get f2py to both compile :file:`add.f` along with the produced
extension module leaving only a shared-library extension file that can
be imported from Python::
Expand Down Expand Up @@ -243,3 +243,6 @@ Read more
* `F2py example: Interactive System for Ice sheet Simulation <http://websrv.cs.umt.edu/isis/index.php/F2py_example>`_
* `"Interfacing With Other Languages" section on the SciPy Cookbook.
<https://scipy-cookbook.readthedocs.io/items/idx_interfacing_with_other_languages.html>`_

.. _`NumPy issue tracker`: https://github.com/numpy/numpy/issues?q=is%3Aissue+label%3A%22component%3A+numpy.f2py%22+is%3Aclosed
.. _`test cases`: https://github.com/numpy/numpy/tree/main/doc/source/f2py/code
27 changes: 19 additions & 8 deletions doc/source/f2py/f2py.getting-started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,25 +22,34 @@ following steps:
* F2PY compiles all sources and builds an extension module containing
the wrappers.

* In building the extension modules, F2PY uses ``numpy_distutils`` which
supports a number of Fortran 77/90/95 compilers, including Gnu, Intel, Sun
Fortran, SGI MIPSpro, Absoft, NAG, Compaq etc. For different build systems,
see :ref:`f2py-bldsys`.
* In building the extension modules, F2PY uses ``meson`` and used to use
``numpy.distutils`` For different build systems, see :ref:`f2py-bldsys`.


.. note::

See :ref:`f2py-meson-distutils` for migration information.


* Depending on your operating system, you may need to install the Python
development headers (which provide the file ``Python.h``) separately. In
Linux Debian-based distributions this package should be called ``python3-dev``,
in Fedora-based distributions it is ``python3-devel``. For macOS, depending
how Python was installed, your mileage may vary. In Windows, the headers are
typically installed already.
typically installed already, see :ref:`f2py-windows`.

.. note::

F2PY supports all the operating systems SciPy is tested on so their
`system dependencies panel`_ is a good reference.

Depending on the situation, these steps can be carried out in a single composite
command or step-by-step; in which case some steps can be omitted or combined
with others.

Below, we describe three typical approaches of using F2PY. These can be read in
order of increasing effort, but also cater to different access levels depending
on whether the Fortran code can be freely modified.
Below, we describe three typical approaches of using F2PY with Fortran 77. These
can be read in order of increasing effort, but also cater to different access
levels depending on whether the Fortran code can be freely modified.

The following example Fortran 77 code will be used for
illustration, save it as ``fib1.f``:
Expand Down Expand Up @@ -298,3 +307,5 @@ the previous case::

>>> print(fib3.fib(8))
[ 0. 1. 1. 2. 3. 5. 8. 13.]

.. _`system dependencies panel`: http://scipy.github.io/devdocs/building/index.html#system-level-dependencies
Loading