Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BUG: f2py does not wrap modules that are used by other modules #26156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ftherrien opened this issue Mar 28, 2024 · 4 comments
Closed

BUG: f2py does not wrap modules that are used by other modules #26156

ftherrien opened this issue Mar 28, 2024 · 4 comments
Labels
00 - Bug component: numpy.f2py

Comments

@ftherrien
Copy link

ftherrien commented Mar 28, 2024 •
edited
Loading

Describe the issue:

If a module is used by another module it is no longer wrapped and cannot be used in python. This was not the behavior before 1.26.3. In my example below, I want func and ufunc to be available in the test python module, but they are not. Notice that the compile output below mentions Skipping utils since it is in 'use'...

I believe that this PR: #25217 is causing the issue. It appears that any module that uses another module is skipped. It was trying to solve the issue described here: #19161, but there are cases like mine were using another module has nothing do with "common blocks".

Reproduce the code example:

test.f90:

module utils

  implicit none

contains

    function ufunc(A)

    double precision, intent(in) :: &
         A

    double precision :: &
         ufunc
    
  end function ufunc

end module utils


module test

  use utils

  implicit none

contains

    function func(A)

    double precision, intent(in) :: &
         A

    double precision :: &
         func
    
  end function func

end module test

Compile:

$ python3 -m numpy.f2py -c --f90exec=gfortran --fcompiler=gnu95 -m test test.f90
/home/felixtherrien/p2penv/lib/python3.10/site-packages/numpy/f2py/f2py2e.py:719: VisibleDeprecationWarning: distutils has been deprecated since NumPy 1.26.xUse the Meson backend instead, or generate wrapperswithout -c and use a custom build script
  builder = build_backend(
running build
running config_cc
INFO: unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
INFO: unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
INFO: build_src
INFO: building extension "test" sources
INFO: f2py options: ['only:', 'func', 'ufunc', 'skip:', '', ':']
INFO: f2py:> /tmp/tmp035u7whr/src.linux-x86_64-3.10/testmodule.c
creating /tmp/tmp035u7whr/src.linux-x86_64-3.10
Reading fortran codes...
	Reading file 'test.f90' (format:free)
Post-processing...
	Block: test
			Block: utils
				Block: ufunc
			Block: test
				Block: func
Applying post-processing hooks...
  character_backward_compatibility_hook
Post-processing (stage 2)...
	Block: test
		Block: unknown_interface
			Block: utils
				Block: ufunc
			Block: test
				Block: func
Building modules...
    Building module "test"...
		Constructing F90 module support for "utils"...
			Skipping utils since it is in 'use'... #<-------LOOK HERE
		Constructing F90 module support for "test"...
		Creating wrapper for Fortran function "func"("func")...
            Constructing wrapper function "test.func"...
              func = func(a)
    Wrote C/API module "test" to file "/tmp/tmp035u7whr/src.linux-x86_64-3.10/testmodule.c"
    Fortran 90 wrappers are saved to "/tmp/tmp035u7whr/src.linux-x86_64-3.10/test-f2pywrappers2.f90"
INFO:   adding '/tmp/tmp035u7whr/src.linux-x86_64-3.10/fortranobject.c' to sources.
INFO:   adding '/tmp/tmp035u7whr/src.linux-x86_64-3.10' to include_dirs.
copying /home/felixtherrien/p2penv/lib/python3.10/site-packages/numpy/f2py/src/fortranobject.c -> /tmp/tmp035u7whr/src.linux-x86_64-3.10
copying /home/felixtherrien/p2penv/lib/python3.10/site-packages/numpy/f2py/src/fortranobject.h -> /tmp/tmp035u7whr/src.linux-x86_64-3.10
INFO:   adding '/tmp/tmp035u7whr/src.linux-x86_64-3.10/test-f2pywrappers2.f90' to sources.
INFO: build_src: building npy-pkg config files
running build_ext
INFO: customize UnixCCompiler
INFO: customize UnixCCompiler using build_ext
INFO: customize Gnu95FCompiler
INFO: Found executable /usr/bin/gfortran
INFO: customize Gnu95FCompiler using build_ext
INFO: building 'test' extension
INFO: compiling C sources
INFO: C compiler: x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC

creating /tmp/tmp035u7whr/tmp
creating /tmp/tmp035u7whr/tmp/tmp035u7whr
creating /tmp/tmp035u7whr/tmp/tmp035u7whr/src.linux-x86_64-3.10
INFO: compile options: '-DNPY_DISABLE_OPTIMIZATION=1 -I/tmp/tmp035u7whr/src.linux-x86_64-3.10 -I/home/felixtherrien/p2penv/lib/python3.10/site-packages/numpy/core/include -I/home/felixtherrien/p2penv/include -I/usr/include/python3.10 -c'
INFO: x86_64-linux-gnu-gcc: /tmp/tmp035u7whr/src.linux-x86_64-3.10/testmodule.c
INFO: x86_64-linux-gnu-gcc: /tmp/tmp035u7whr/src.linux-x86_64-3.10/fortranobject.c
INFO: compiling Fortran 90 module sources
INFO: Fortran f77 compiler: /usr/bin/gfortran -Wall -g -ffixed-form -fno-second-underscore -fPIC -O3 -funroll-loops
Fortran f90 compiler: gfortran -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops
Fortran fix compiler: gfortran -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops
INFO: compile options: '-I/tmp/tmp035u7whr/src.linux-x86_64-3.10 -I/home/felixtherrien/p2penv/lib/python3.10/site-packages/numpy/core/include -I/home/felixtherrien/p2penv/include -I/usr/include/python3.10 -c'
extra options: '-J/tmp/tmp035u7whr/ -I/tmp/tmp035u7whr/'
INFO: gfortran:f90: test.f90
test.f90:7:20:

    7 |     function ufunc(A)
      |                    1
Warning: Unused dummy argument ‘a’ at (1) [-Wunused-dummy-argument]
test.f90:7:4:

    7 |     function ufunc(A)
      |    1
Warning: Return value of function ‘ufunc’ at (1) not set [-Wreturn-type]
test.f90:27:19:

   27 |     function func(A)
      |                   1
Warning: Unused dummy argument ‘a’ at (1) [-Wunused-dummy-argument]
test.f90:27:4:

   27 |     function func(A)
      |    1
Warning: Return value of function ‘func’ at (1) not set [-Wreturn-type]
INFO: compiling Fortran sources
INFO: Fortran f77 compiler: /usr/bin/gfortran -Wall -g -ffixed-form -fno-second-underscore -fPIC -O3 -funroll-loops
Fortran f90 compiler: gfortran -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops
Fortran fix compiler: gfortran -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops
INFO: compile options: '-I/tmp/tmp035u7whr/src.linux-x86_64-3.10 -I/home/felixtherrien/p2penv/lib/python3.10/site-packages/numpy/core/include -I/home/felixtherrien/p2penv/include -I/usr/include/python3.10 -c'
extra options: '-J/tmp/tmp035u7whr/ -I/tmp/tmp035u7whr/'
INFO: gfortran:f90: /tmp/tmp035u7whr/src.linux-x86_64-3.10/test-f2pywrappers2.f90
INFO: /usr/bin/gfortran -Wall -g -Wall -g -shared /tmp/tmp035u7whr/tmp/tmp035u7whr/src.linux-x86_64-3.10/testmodule.o /tmp/tmp035u7whr/tmp/tmp035u7whr/src.linux-x86_64-3.10/fortranobject.o /tmp/tmp035u7whr/test.o /tmp/tmp035u7whr/tmp/tmp035u7whr/src.linux-x86_64-3.10/test-f2pywrappers2.o -L/usr/lib/gcc/x86_64-linux-gnu/11 -L/usr/lib/gcc/x86_64-linux-gnu/11 -lgfortran -o ./test.cpython-310-x86_64-linux-gnu.so
Removing build directory /tmp/tmp035u7whr

Running

$ python -c "import test; print(test.__doc__)"
This module 'test' is auto-generated with f2py (version:1.26.4).
Functions:
Fortran 90/95 modules:
  test --- func().

Python and NumPy Versions:

Numpy 1.26.4
Python 3.10.12

Tested that the behavior is not the same in Numpy 1.26.2

Runtime Environment:

No response

Context for the issue:

This was reported to me here: ftherrien/p2ptrans#34 and makes some of my functions unavailable in python without any error message at compile time.
@ftherrien
Copy link
Author

Maybe it is relevant to tag @charris and @HaoZeke

@HaoZeke
Copy link
Member

HaoZeke commented Apr 9, 2024

Thanks for the report and reproducer @ftherrien

I can't seem to reproduce this on the latest main.

>>> import test
>>> [x for x in dir(test) if '__' not in x]
['_test_error', 'test', 'utils']
>>> [x for x in dir(test.test) if '__' not in x]
['func']
>>> [x for x in dir(test.utils) if '__' not in x]
['ufunc']

From:

❯ python -c "import numpy as np; print(np.__version__)"
2.1.0.dev0+git20240409.2d2ff25
python3 -m numpy.f2py -c --f90exec=gfortran --fcompiler=gnu95 -m test test.f90 --backend meson

I believe this was fixed in #25361

Note that there is a slight misunderstanding in the way the reproducer is run. The ufunc and func will be made available from the respective modules under the "pythonic" toplevel test. That is the wrapped functions are test.test.func and test.utils.ufunc..

Feel free to reopen this if anything isn't clear..

@HaoZeke HaoZeke closed this as completed Apr 9, 2024
@ftherrien
Copy link
Author

ftherrien commented Apr 9, 2024 •
edited
Loading

@HaoZeke using the latest main does fix it for me. Thanks!

For reference for the moment this fix is only available in versions 2+ which are currently pre-released: v2.1.0.dev0 v2.0.0rc1 v2.0.0b1 and can be installed through pip install --pre numpy

@HaoZeke
Copy link
Member

HaoZeke commented Apr 9, 2024

@HaoZeke using the latest main does fix it for me. Thanks!

For reference for the moment this fix is only available in versions 2+ which are currently pre-released: v2.1.0.dev0 v2.0.0rc1 v2.0.0b1

Thanks for the quick verification! It is a bit unfortunate, but there were so many changes to f2py that were a major pain to backport... So many of the fixes were only present in 2.x onwards where the entire directory was replaced essentially.

@bilderbuchi bilderbuchi mentioned this issue Jun 13, 2024
Closed
@HaoZeke HaoZeke mentioned this issue Nov 4, 2024
Merged
@charris charris mentioned this issue Nov 17, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug component: numpy.f2py
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@rgommers @HaoZeke @ftherrien