Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BUG: f2py in numpy 2.0.1 cannot compile code with explicit openmp procedure calls [Intel compilers] #27163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AgilentGCMS opened this issue Aug 9, 2024 · 11 comments
Labels
00 - Bug 29 - Intel/Anaconda component: numpy.f2py Meson Items related to the introduction of Meson as the new build system for NumPy

Comments

@AgilentGCMS
Copy link

AgilentGCMS commented Aug 9, 2024

Describe the issue:

I have compiled python 3.12.4 and numpy 2.0.1 with intel oneapi 2024.1 compilers on a compute cluster for multiple people to use. I have some f2py code that explicitly uses openmp library calls such as omp_set_num_threads. With previous versions of python, including 3.11.6, I could compile this with

f2py --compiler=intelem --fcompiler=intelem -c mwe_f2py.pyf --opt='-qopenmp' -liomp5 mwe_f2py.F90

which would produce a perfectly working python module mwe_f2py.cpython-311-x86_64-linux-gnu.so. However, since everything has moved to the new-fangled meson now, this is no longer possible on python 3.12.4. So instead if I do

CFLAGS='-qopenmp' FC=ifx CC=icx CXX=icpx f2py -c mwe_f2py.pyf --f77flags='-qopenmp' --f90flags='-qopenmp' -liomp5 mwe_f2py.F90

it raises the error undefined reference to 'omp_set_num_threads'. This should not happen because I am supplying -qopenmp as a compiler option.

Reproduce the code example:

Put the following in mwe_f2py.F90

module precon

implicit none

public :: xc_to_x

contains

subroutine xc_to_x(n_state, G_state, num_threads, x)

   use omp_lib

   integer, intent(in)             :: n_state, num_threads
   double precision, intent(in)    :: G_state(n_state)
   double precision, intent(out)   :: x(n_state)
   integer                         :: i

   x = 0.0

   call omp_set_num_threads(num_threads)

   !$omp parallel do schedule(guided)
   do i = 1, n_state
      x(i) = G_state(i) * G_state(n_state-i+1)
   end do
   !$omp end parallel do

end subroutine xc_to_x

end module precon

and the following in mwe_f2py.pyf,

python module mwe_f2py
    interface

    module precon

        subroutine xc_to_x(n_state, G_state, num_threads, x)
            integer,optional,intent(in),check(len(G_state)==n_state),depend(G_state) :: n_state=len(G_state)
            double precision,dimension(n_state),intent(in) :: G_state
            integer,intent(in) :: num_threads
            double precision,dimension(n_state),intent(out) :: x
        end subroutine xc_to_x

    end module precon

    end interface
end python module mwe_f2py

Error message:

Cannot use distutils backend with Python>=3.12, using meson backend instead.
Ignoring -m untitled.
mwe_f2py.pyf defines mwe_f2py to be the modulename.
Reading fortran codes...
	Reading file 'mwe_f2py.pyf' (format:free)
Post-processing...
	Block: mwe_f2py
			Block: precon
				Block: xc_to_x
Applying post-processing hooks...
  character_backward_compatibility_hook
Post-processing (stage 2)...
	Block: mwe_f2py
		Block: unknown_interface
			Block: precon
				Block: xc_to_x
Building modules...
    Building module "mwe_f2py"...
		Constructing F90 module support for "precon"...
            Constructing wrapper function "precon.xc_to_x"...
              x = xc_to_x(G_state,num_threads,[n_state])
    Wrote C/API module "mwe_f2py" to file "./mwe_f2pymodule.c"
    Fortran 90 wrappers are saved to "./mwe_f2py-f2pywrappers2.f90"
The Meson build system
Version: 1.5.1
Source dir: /tmp/tmp1gfz5yx5
Build dir: /tmp/tmp1gfz5yx5/bbdir
Build type: native build
Project name: mwe_f2py
Project version: 0.1
Fortran compiler for the host machine: ifx (intel-llvm 2024.1.0 "ifx (IFX) 2024.1.0 20240308")
Fortran linker for the host machine: ifx ld.bfd 2.35.2-24
C compiler for the host machine: icx (intel-llvm 11.3.1 "Intel(R) oneAPI DPC++/C++ Compiler 2024.1.0 (2024.1.0.20240308)")
C linker for the host machine: icx ld.bfd 2.35.2-24
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program /work2/noaa/co2/sbasu/packages/python/3.12.4/bin/python found: YES (/work2/noaa/co2/sbasu/packages/python/3.12.4/bin/python)
Found pkg-config: YES (/usr/bin/pkg-config) 1.7.3
Run-time dependency python found: YES 3.12
Library quadmath found: YES
Build targets in project: 1

Found ninja-1.11.1.git.kitware.jobserver-1 at /work2/noaa/co2/sbasu/packages/python/3.12.4/bin/ninja
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /work2/noaa/co2/sbasu/packages/python/3.12.4/bin/ninja -C /tmp/tmp1gfz5yx5/bbdir
ninja: Entering directory `/tmp/tmp1gfz5yx5/bbdir'
[6/6] Linking target mwe_f2py.cpython-312-x86_64-linux-gnu.so
FAILED: mwe_f2py.cpython-312-x86_64-linux-gnu.so
icx  -o mwe_f2py.cpython-312-x86_64-linux-gnu.so mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/mwe_f2py.F90.o mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/mwe_f2pymodule.c.o mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/mwe_f2py-f2pywrappers2.f90.o mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/e3838bb052daf315fa49226b51f93a7020d9e0a6_.._.._f2py_src_fortranobject.c.o -Wl,--as-needed -Wl,--allow-shlib-undefined -Wl,-O1 -shared -fPIC -qopenmp -Wl,--start-group -lquadmath -liomp5 -lifcore -limf -Wl,--end-group
/usr/bin/ld: mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/mwe_f2py.F90.o: in function `precon_mp_xc_to_x_':
ifxKzK7kY.i90:(.text+0x3e): undefined reference to `omp_set_num_threads'
/usr/bin/ld: mwe_f2py.cpython-312-x86_64-linux-gnu.so: hidden symbol `omp_set_num_threads' isn't defined
/usr/bin/ld: final link failed: bad value
icx: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/work2/noaa/co2/sbasu/packages/python/3.12.4/bin/f2py", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/work2/noaa/co2/sbasu/packages/python/3.12.4/lib/python3.12/site-packages/numpy/f2py/f2py2e.py", line 769, in main
    run_compile()
  File "/work2/noaa/co2/sbasu/packages/python/3.12.4/lib/python3.12/site-packages/numpy/f2py/f2py2e.py", line 741, in run_compile
    builder.compile()
  File "/work2/noaa/co2/sbasu/packages/python/3.12.4/lib/python3.12/site-packages/numpy/f2py/_backends/_meson.py", line 193, in compile
    self.run_meson(self.build_dir)
  File "/work2/noaa/co2/sbasu/packages/python/3.12.4/lib/python3.12/site-packages/numpy/f2py/_backends/_meson.py", line 188, in run_meson
    self._run_subprocess_command(compile_command, build_dir)
  File "/work2/noaa/co2/sbasu/packages/python/3.12.4/lib/python3.12/site-packages/numpy/f2py/_backends/_meson.py", line 182, in _run_subprocess_command
    subprocess.run(command, cwd=cwd, check=True)
  File "/work2/noaa/co2/sbasu/packages/python/3.12.4/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['meson', 'compile', '-C', 'bbdir']' returned non-zero exit status 1.

Python and NumPy Versions:

2.0.1
3.12.4 (main, Jul 29 2024, 10:14:44) [Clang 18.0.0 (icx 2024.1.0.20240308)]

Runtime Environment:

[{'numpy_version': '2.0.1',
'python': '3.12.4 (main, Jul 29 2024, 10:14:44) [Clang 18.0.0 (icx '
'2024.1.0.20240308)]',
'uname': uname_result(system='Linux', node='hercules-login-3.hpc.msstate.edu', release='5.14.0-162.23.1.el9_1.x86_64', version='#1 SMP PREEMPT_DYNAMIC Tue Apr 11 19:09:37 UTC 2023', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2',
'AVX512F',
'AVX512CD',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL'],
'not_found': ['AVX512_KNL']}},
{'filepath': '/apps/spack-managed/gcc-11.3.1/intel-oneapi-compilers-2024.1.0-76aou5yu2oqfcz72vq7lp77bp3rqcoht/compiler/2024.1/lib/libiomp5.so',
'internal_api': 'openmp',
'num_threads': 160,
'prefix': 'libiomp',
'user_api': 'openmp',
'version': None}]

Context for the issue:

No response

@AgilentGCMS AgilentGCMS changed the title BUG: f2py cannot compile code with explicit openmp procedure calls BUG: f2py in numpy 2.0.1 cannot compile code with explicit openmp procedure calls Aug 10, 2024
@AgilentGCMS
Copy link
Author

I went back to python 3.11.6 and after modifying numpy/distutils/intelccompiler.py and numpy/distutils/fcompiler/intel.py to use icx and ifx to match oneapi, I was able to compile my f2py code successfully into python modules. I will keep this open since I haven't seen any documentation online as to how one would do this for python 3.12.x with meson build.

@HaoZeke
Copy link
Member

HaoZeke commented Aug 18, 2024

OTOH, shouldn't this just be done by editing the meson.build for using fortran_args on the target?

@HaoZeke
Copy link
Member

HaoZeke commented Aug 18, 2024

I know there's still not a lot of clarity on the way the scope of F2PY has changed with the shift to meson, F2PY is meant to focus purely on generating the wrappers, the build system is meant to be customized with the meson.build as a starting point, and as such, these issues are typically upstream / deprecated forms (but very useful to have nonetheless :) )

@HaoZeke
Copy link
Member

HaoZeke commented Aug 18, 2024

This should be working by default since #26703 in any case, could you also share the generated meson.build ?

@AgilentGCMS
Copy link
Author

CFLAGS='-qopenmp' FC=ifx CC=icx CXX=icpx f2py -c mwe_f2py.pyf --f77flags='-qopenmp' --f90flags='-qopenmp' -liomp5 mwe_f2py.F90

Where can I find this meson.build that is being generated? I don't see it in the folder where my F90 and pyf files are. One of the joys of the meson system, I have discovered, is that it very helpfully deletes all intermediate files.

@HaoZeke
Copy link
Member

HaoZeke commented Aug 20, 2024

CFLAGS='-qopenmp' FC=ifx CC=icx CXX=icpx f2py -c mwe_f2py.pyf --f77flags='-qopenmp' --f90flags='-qopenmp' -liomp5 mwe_f2py.F90

Where can I find this meson.build that is being generated? I don't see it in the folder where my F90 and pyf files are. One of the joys of the meson system, I have discovered, is that it very helpfully deletes all intermediate files.

This was true before as well though, passing --build-dir whatever is required to keep all the generated files (including the whatever/meson.build file).

@AgilentGCMS
Copy link
Author

CFLAGS='-qopenmp' FC=ifx CC=icx CXX=icpx f2py -c mwe_f2py.pyf --f77flags='-qopenmp' --f90flags='-qopenmp' -liomp5 mwe_f2py.F90

Where can I find this meson.build that is being generated? I don't see it in the folder where my F90 and pyf files are. One of the joys of the meson system, I have discovered, is that it very helpfully deletes all intermediate files.

This was true before as well though, passing --build-dir whatever is required to keep all the generated files (including the whatever/meson.build file).

Indeed, for f2py that was always the case :-) My complaint was a carryover from trying to use meson instead of distutils for compiling (other) python modules. Anyway, here's what is inside meson.build:

project('mwe_f2py',
        ['c', 'fortran'],
        version : '0.1',
        meson_version: '>= 1.1.0',
        default_options : [
                            'warning_level=1',
                            'buildtype=release'
                          ])
fc = meson.get_compiler('fortran')

py = import('python').find_installation('''/work2/noaa/co2/sbasu/packages/python/3.12.4/bin/python''', pure: false)
py_dep = py.dependency()

incdir_numpy = run_command(py,
  ['-c', 'import os; os.chdir(".."); import numpy; print(numpy.get_include())'],
  check : true
).stdout().strip()

incdir_f2py = run_command(py,
    ['-c', 'import os; os.chdir(".."); import numpy.f2py; print(numpy.f2py.get_include())'],
    check : true
).stdout().strip()

inc_np = include_directories(incdir_numpy)
np_dep = declare_dependency(include_directories: inc_np)

incdir_f2py = incdir_numpy / '..' / '..' / 'f2py' / 'src'
inc_f2py = include_directories(incdir_f2py)
fortranobject_c = incdir_f2py / 'fortranobject.c'

inc_np = include_directories(incdir_numpy, incdir_f2py)
# gh-25000
quadmath_dep = fc.find_library('quadmath', required: false)

iomp5 = declare_dependency(link_args : ['-liomp5'])


py.extension_module('mwe_f2py',
                     [
                     '''mwe_f2py.F90''',
                     '''mwe_f2pymodule.c''',
                     '''mwe_f2py-f2pywrappers2.f90''',
                     fortranobject_c
                     ],
                     include_directories: [
                     inc_np,

                     ],
                     dependencies : [
                     py_dep,
                     quadmath_dep,

                     iomp5,

                     ],
                     fortran_args: ['-qopenmp'],
                     install : true)

If you look at the original error, it's the C compiler invocation that is failing.

@HaoZeke
Copy link
Member

HaoZeke commented Aug 21, 2024 via email

@AgilentGCMS
Copy link
Author

OTOH, does adding c_args: ['-qopenmp'] fix this? Might be worth having a way to pass c_args from the command line.

Since meson.build is auto-generated, I do not know how to add c_args: ['-qopenmp']. Can you show me?

However, for OpenMP in particular it might be best to just pass --dep openmp which should solve this (or make it an upstream error [1] since it'll add dependency('openmp')... [1] https://mesonbuild.com/Dependencies.html#openmp

You mean like this?

CFLAGS='-qopenmp' FC=ifx CC=icx CXX=icpx f2py --build-dir mwe_build -c mwe_f2py.pyf --f77flags='-qopenmp' --f90flags='-qopenmp' --dep openmp -liomp5 mwe_f2py.F90

This did not work, it still barfed with the error

FAILED: mwe_f2py.cpython-312-x86_64-linux-gnu.so
icx  -o mwe_f2py.cpython-312-x86_64-linux-gnu.so mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/mwe_f2py.F90.o mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/mwe_f2pymodule.c.o mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/mwe_f2py-f2pywrappers2.f90.o mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/e3838bb052daf315fa49226b51f93a7020d9e0a6_.._.._f2py_src_fortranobject.c.o -Wl,--as-needed -Wl,--allow-shlib-undefined -Wl,-O1 -shared -fPIC -qopenmp -Wl,--start-group -lquadmath -fopenmp -liomp5 -lifcore -limf -Wl,--end-group
icx: warning: Use of '-qopenmp' recommended over '-fopenmp' [-Wrecommended-option]
/usr/bin/ld: mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/mwe_f2py.F90.o: in function `precon_mp_xc_to_x_':
ifxwWBjCB.i90:(.text+0x3e): undefined reference to `omp_set_num_threads'
/usr/bin/ld: mwe_f2py.cpython-312-x86_64-linux-gnu.so: hidden symbol `omp_set_num_threads' isn't defined
/usr/bin/ld: final link failed: bad value
icx: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

@HaoZeke
Copy link
Member

HaoZeke commented Aug 21, 2024

OTOH, does adding c_args: ['-qopenmp'] fix this? Might be worth having a way to pass c_args from the command line.

Since meson.build is auto-generated, I do not know how to add c_args: ['-qopenmp']. Can you show me?

Ah sorry, so you can always go into the build directory (the one passed to --build-dir) and then modify the meson.build and run meson compile -C bbdir in that folder. In fact, the recommended way to keep F2PY wrappers as part of a larger build system is now to use the auto-generated meson.build as a starting point to hack into a working form.

However, for OpenMP in particular it might be best to just pass --dep openmp which should solve this (or make it an upstream error [1] since it'll add dependency('openmp')... [1] https://mesonbuild.com/Dependencies.html#openmp

You mean like this?

CFLAGS='-qopenmp' FC=ifx CC=icx CXX=icpx f2py --build-dir mwe_build -c mwe_f2py.pyf --f77flags='-qopenmp' --f90flags='-qopenmp' --dep openmp -liomp5 mwe_f2py.F90

This did not work, it still barfed with the error

FAILED: mwe_f2py.cpython-312-x86_64-linux-gnu.so
icx  -o mwe_f2py.cpython-312-x86_64-linux-gnu.so mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/mwe_f2py.F90.o mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/mwe_f2pymodule.c.o mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/mwe_f2py-f2pywrappers2.f90.o mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/e3838bb052daf315fa49226b51f93a7020d9e0a6_.._.._f2py_src_fortranobject.c.o -Wl,--as-needed -Wl,--allow-shlib-undefined -Wl,-O1 -shared -fPIC -qopenmp -Wl,--start-group -lquadmath -fopenmp -liomp5 -lifcore -limf -Wl,--end-group
icx: warning: Use of '-qopenmp' recommended over '-fopenmp' [-Wrecommended-option]
/usr/bin/ld: mwe_f2py.cpython-312-x86_64-linux-gnu.so.p/mwe_f2py.F90.o: in function `precon_mp_xc_to_x_':
ifxwWBjCB.i90:(.text+0x3e): undefined reference to `omp_set_num_threads'
/usr/bin/ld: mwe_f2py.cpython-312-x86_64-linux-gnu.so: hidden symbol `omp_set_num_threads' isn't defined
/usr/bin/ld: final link failed: bad value
icx: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Hmm, odd. The flags seemed to be passed, but evidently meson's dependency openmp doesn't seem to play nicely with Intel. For reference the MWE works for me with gfortran / gcc / openmp and simply using:

f2py -c mwe.pyf mwe.f90 --dep openmp
python -c "import mwe_f2py"

Does FC=ifx CC=icx f2py -c mwe.pyf mwe.f90 --dep openmp not work? Ah there's an upstream issue mesonbuild/meson#13516

However as a work around, adding link_args manually in the meson.build and then compiling the extension module as noted above should work, I think. Sometime this weekend I'll pull up the intel compilers and try to open bug reports / fixes upstream at meson.

@HaoZeke HaoZeke changed the title BUG: f2py in numpy 2.0.1 cannot compile code with explicit openmp procedure calls BUG: f2py in numpy 2.0.1 cannot compile code with explicit openmp procedure calls [Intel compilers] Aug 21, 2024
@HaoZeke HaoZeke added 29 - Intel/Anaconda Meson Items related to the introduction of Meson as the new build system for NumPy labels Aug 21, 2024
@sujona
Copy link

sujona commented Aug 27, 2024

Maybe try the meson-specific --dep flag, like

f2py <your args> --dep openmp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
00 - Bug 29 - Intel/Anaconda component: numpy.f2py Meson Items related to the introduction of Meson as the new build system for NumPy
Projects
None yet
Development

No branches or pull requests

4 participants