Thanks to visit codestin.com
Credit goes to github.com

Skip to content

f2py doesn't finishing compiling #19673

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
kconnour opened this issue Aug 15, 2021 · 5 comments
Closed

f2py doesn't finishing compiling #19673

kconnour opened this issue Aug 15, 2021 · 5 comments
Labels
component: numpy.distutils component: numpy.f2py

Comments

@kconnour
Copy link

I have some FORTRAN code that I want to combine into a .so file and import with Python. I wrote a script that does that as part of setup.py. This script works on some computers but not others. When it fails, f2py appears to go through the entire compilation process and prints a command-line command, but it seems this command is never executed. If I manually execute this command, the compilation succeeds.

The build works on my Ubuntu 20.10 and macOS 10.15.7 computers but fails on macOS 11, but I'm not convinced this is an OS issue. In all cases I'm using gfortran from gcc 10.2.0.

Reproducing code example:

I can't give the full code without my fortran modules but this is the key f2py line.

from numpy import f2py
with open('path/to/DISORT.f') as disort:
        f2py.compile(disort.read(), modulename='disort', extra_args=mod_paths)

Error message:

None. Regardless of whether or not the build succeeds, f2py outputs something like the following near the end of its output:

/usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle /var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/src.macosx-11.3-x86_64-3.9/disortmodule.o /var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/src.macosx-11.3-x86_64-3.9/fortranobject.o /var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmpg2n7bma6.o /var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/disort4.0.99/BDREF.o /var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/disort4.0.99/DISOBRDF.o /var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/disort4.0.99/ERRPACK.o /var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/disort4.0.99/LAPACK.o /var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/disort4.0.99/LINPAK.o /var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/disort4.0.99/RDI1MACH.o /var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/var/folders/_2/5thyvq3j25n1_wx318h5lqf00000h8/T/tmp4g4jlqyl/src.macosx-11.3-x86_64-3.9/disort-f2pywrappers.o -L/usr/local/lib/gcc/x86_64-apple-darwin19/10.2.0 -L/usr/local/lib/gcc/x86_64-apple-darwin19/10.2.0/../../.. -L/usr/local/lib/gcc/x86_64-apple-darwin19/10.2.0/../../.. -lgfortran -o ./disort.cpython-39-darwin.so

On some computers, f2py won't have generated a .so file. However, in every instance that this happens, if I simply copy the above command and paste it into Terminal and execute it, it does create the .so file. When it does create the .so file (without my manual intervention) I do also get a line like the following:

Removing build directory /var/folders/2w/9zwyqhw11218bmh13tp__lj0008m_6/T/tmp6_95cpvx

NumPy/Python version information:

1.20.2 3.9.5 (default, May 7 2021, 21:28:16)
[Clang 12.0.5 (clang-1205.0.22.9)]
@melissawm
Copy link
Member

Hello, @kconnour - can you give a little more detail? In which systems does the compilation fail?

The message about removing the build directory is expected.

@kconnour
Copy link
Author

Yes, I'm happy to provide as much detail as you'd like. Don't hesitate to ask.

If it's helpful, this is my repo where the issue is happening (not that I expect anyone to look at it but it'd provide full context if the above code snippet is insufficient).

I encounter this error when attempting to run my code on Github's CI using macos-11 and macos-10.15 runners, and a remote cluster I have access to. I'm not completely sure what other info you're after besides the OS info from the previous message, but if I run system_profiler SPHardwareDataType from Terminal on the cluster I get:

Hardware:

    Hardware Overview:

      Model Name: Mac Pro
      Model Identifier: MacPro7,1
      Enclosure: Tower
      Processor Name: 28-Core Intel Xeon W
      Processor Speed: 2.5 GHz
      Number of Processors: 1
      Total Number of Cores: 28
      L2 Cache (per Core): 1 MB
      L3 Cache: 38.5 MB
      Hyper-Threading Technology: Enabled
      Memory: 192 GB
      System Firmware Version: 1554.120.19.0.0 (iBridge: 18.16.14663.0.0,0)

Does this help?

@HaoZeke
Copy link
Member

HaoZeke commented Apr 28, 2022

@kconnour, does this happen when using the regular CLI? e.g. f2py -m distort disort.f?

@kconnour
Copy link
Author

Yes, it gives the same error; however, I think a collaborator may have found the problem. If I run the aforementioned code directly from a Bash script, I get this message:

fatal error: Python.h: No such file or directory
    7 | #include <Python.h>
      |          ^~~~~~~~~~
compilation terminated.

The error completely goes away if python3.9-dev is installed, so the error may not be with f2py but with my system configuration. If I do need dev tools installed to use f2py, it would be nice if that was included with the documentation (or perhaps I just missed that detail, which is entirely possible).

@HaoZeke
Copy link
Member

HaoZeke commented Sep 5, 2023

Yes, it gives the same error; however, I think a collaborator may have found the problem. If I run the aforementioned code directly from a Bash script, I get this message:

fatal error: Python.h: No such file or directory
    7 | #include <Python.h>
      |          ^~~~~~~~~~
compilation terminated.

The error completely goes away if python3.9-dev is installed, so the error may not be with f2py but with my system configuration. If I do need dev tools installed to use f2py, it would be nice if that was included with the documentation (or perhaps I just missed that detail, which is entirely possible).

Development headers should be installed for working with compiled extensions :) also now deprecated by #24532.

@HaoZeke HaoZeke closed this as completed Sep 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
component: numpy.distutils component: numpy.f2py
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@melissawm @HaoZeke @kconnour