I think it's time to throw away this code, it's way too hacky. We should be able to define F2PY_THREAD_LOCAL_DECL in the same way as NPY_TLS instead. That looks much cleaner:

Hmm, not so easy unfortunately, that requires compile-time checks:

That's more robust, but we can't require all f2py users to implement that (unconditionally at least).

@tylerjereddy is the only difference between success and failure using GCC 14 vs. 13 on your machine?

@rgommers no, I was able to confirm that both of those GCC versions fail identically due to missing threads.h (via f2py), on OSX 14.7, Python 3.12.3, NumPy 2.1.3. So both of these incantations fail identically (with git clean -xfd in between):

• CC=gcc-13 CXX=g++-13 python dev.py build -j 16 (Homebrew GCC 13.3.0)
• CC=gcc-14 CXX=g++-14 python dev.py build -j 16 (Homebrew GCC 14.2.0_1)

I often use a fixed GNU FC version, but also just confirmed this also has the same behavior:
CC=gcc-13 CXX=g++-13 FC=gfortran-13 python dev.py build -j 16

With conda-forge compilers I currently use, I get this with both GCC 12.3.0 on Linux and Clang 16.0.6 on macOS 14.7:

/* #undef HAVE_THREAD_LOCAL */
#define HAVE__THREAD_LOCAL 1
#define HAVE__THREAD 1
/* #undef HAVE___DECLSPEC_THREAD_ */

Gating things only on OS version is clearly not right.

Agreed, I don't replicate the problem with AppleClang (I only noticed it because MacPorts folks were complaining about GNU toolchain) so full suppression of the header include on OS 14.x seems a bit aggressive.

I don't know how much work it would be to get the guard "just right" (investigate clang version and GNU version ranges?).

I don't know how much work it would be to get the guard "just right" (investigate clang version and GNU version ranges?).

It's impossible to get it "just right" without build-time compiler checks AFAICT. I think what we should do (similar to what I suggested as workaround in #19437 (comment)) is to document how to do the compiler checks in the package using f2py and then adding the correct F2PY_THREAD_LOCAL_DECL define to the build flags.

Does anyone see a cleaner solution?

threads.h is also missing on earlier macOS, and the source erroneously assumes it is present due to C version.

[2/508] Compiling C object scipy/linalg/_interpolative.cpython-311-darwin.so.p/meson-generated__interpolativemodule.c.o
FAILED: scipy/linalg/_interpolative.cpython-311-darwin.so.p/meson-generated__interpolativemodule.c.o 
/opt/local/bin/gcc-mp-14 -Iscipy/linalg/_interpolative.cpython-311-darwin.so.p -Iscipy/linalg -I../scipy/linalg -I../../../../../../../../Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/numpy/core/include -I../../../../../../../../Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/numpy/f2py/src -I/opt/local/include/openblas -I/opt/local/Library/Frameworks/Python.framework/Versions/3.11/include/python3.11 -fvisibility=hidden -fdiagnostics-color=always -DNDEBUG -Wall -Winvalid-pch -std=c17 -O3 -Wno-unused-but-set-variable -Wno-unused-function -Wno-conversion -Wno-misleading-indentation -isysroot/ -MD -MQ scipy/linalg/_interpolative.cpython-311-darwin.so.p/meson-generated__interpolativemodule.c.o -MF scipy/linalg/_interpolative.cpython-311-darwin.so.p/meson-generated__interpolativemodule.c.o.d -o scipy/linalg/_interpolative.cpython-311-darwin.so.p/meson-generated__interpolativemodule.c.o -c scipy/linalg/_interpolative.cpython-311-darwin.so.p/_interpolativemodule.c
scipy/linalg/_interpolative.cpython-311-darwin.so.p/_interpolativemodule.c:96:10: fatal error: threads.h: No such file or directory
   96 | #include <threads.h>
      |          ^~~~~~~~~~~
compilation terminated.
[3/508] Compiling Fortran object scipy/linalg/_interpolative.cpython-311-darwin.so.p/meson-generated__interpolative-f2pywrappers.f.o
[4/508] Compiling C object scipy/linalg/_flapack.cpython-311-darwin.so.p/meson-generated_..__flapackmodule.c.o
FAILED: scipy/linalg/_flapack.cpython-311-darwin.so.p/meson-generated_..__flapackmodule.c.o 
/opt/local/bin/gcc-mp-14 -Iscipy/linalg/_flapack.cpython-311-darwin.so.p -Iscipy/linalg -I../scipy/linalg -I../../../../../../../../Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/numpy/core/include -I../../../../../../../../Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/numpy/f2py/src -I/opt/local/include/openblas -I/opt/local/Library/Frameworks/Python.framework/Versions/3.11/include/python3.11 -fvisibility=hidden -fdiagnostics-color=always -DNDEBUG -Wall -Winvalid-pch -std=c17 -O3 -Wno-unused-but-set-variable -Wno-unused-function -Wno-conversion -Wno-misleading-indentation -isysroot/ -Wno-empty-body -MD -MQ scipy/linalg/_flapack.cpython-311-darwin.so.p/meson-generated_..__flapackmodule.c.o -MF scipy/linalg/_flapack.cpython-311-darwin.so.p/meson-generated_..__flapackmodule.c.o.d -o scipy/linalg/_flapack.cpython-311-darwin.so.p/meson-generated_..__flapackmodule.c.o -c scipy/linalg/_flapackmodule.c
scipy/linalg/_flapackmodule.c:90:10: fatal error: threads.h: No such file or directory
   90 | #include <threads.h>
      |          ^~~~~~~~~~~
compilation terminated.

threads.h is also missing on earlier macOS, and the source erroneously assumes it is present due to C version.

Yes, this is correct indeed, and the bug is that __STDC_NO_THREADS__ isn't set despite no threads.h. This is the compilers job I believe (e.g., MSVC takes care of defining it), so it's the GCC build in use that has the bug.

I think this was probably always broken, but no one noticed because they weren't yet compiling with -std=c11 or higher (plus using GCC on macOS is rare).

Looks like the C standards committee saw the error of its ways - in C23 thread_local is removed from threads.h and becomes a C keyword instead: https://en.cppreference.com/w/c/thread/thread_local.

threads.h is also missing on earlier macOS, and the source erroneously assumes it is present due to C version.

Yes, this is correct indeed, and the bug is that __STDC_NO_THREADS__ isn't set despite no threads.h. This is the compilers job I believe (e.g., MSVC takes care of defining it), so it's the GCC build in use that has the bug.

Discussion with gcc upstream: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53769

I think this was probably always broken, but no one noticed because they weren't yet compiling with -std=c11 or higher (plus using GCC on macOS is rare).

I think this is just untypical to use this header. GCC supports emulated TLS on systems where it is not supported otherwise (and recent macOS support TLS in the OS).

GCC supports emulated TLS on systems where it is not supported otherwise (and recent macOS support TLS in the OS).

You mean GCC and/or macOS recognize thread_local without any header include? Or it expects you to use _Thread_local?

_Thread_local

I think gcc normally uses this one, yes.

Re emutls, I do not know internals of the implementation, but gcc have some docs on it.

Is there any real value in using "real" gcc to build things on macOS? As soon as your code uses macOS headers, you hit the infamous "blocks" feature, not implemented in gcc. (e.g. SageMath stopped using gcc on macOS ages ago).

@dimpase Blocks are only used in ObjC, which is a minuscule part of open-source software. (Besides, gcc upstream plans to support blocks somewhat soon, though I am not aware of intended timeline.)

Also it is good for code quality. GCC is one of the two major compilers with its own C++ runtime. Quality code should work with it on whatever system, at least as long as it is C/C++/Fortran.

No, macOS headers have blocks in plain C as well. Anything related to cryptography on macOS will need to use them; e.g. you can't even build a fully functioning Python using gcc only (you'd hit problems with urllib, IIRC).

I certainly agree with the code quality point.

you can't even build a fully functioning Python

I do not know how is “fully” defined, but I have certainly built Python with gcc on macOS, and it was functioning.

Anything related to cryptography on macOS will need to use them

You mean their cryptography? That is unsurprising, since it uses ObjC (used at least, and perhaps nothing much changed). But there are plenty of open-source cryptography solutions not relying on Apple ObjC extensions.

I do not know how is “fully” defined, but I have certainly built Python with gcc on macOS, and it was functioning.

how about installing packages from PyPI ? (That is, pip etc. should be built with "real" gcc too).

I do not know how is “fully” defined, but I have certainly built Python with gcc on macOS, and it was functioning.

how about installing packages from PyPI ? (That is, pip etc. should be built with "real" gcc too).

I use MacPorts, I do not think anything fails with Python as such or C/C++ code in Python ports. Rust needs LLVM, so that won’t work with gcc, and there a few select packages which heavily depend on Apple SDK (and perhaps Apple ObjC syntax).

BTW, modern gcc supports libc++, so it can be used with Apple C++ native runtime, if one wishes to.

modern gcc supports libc++

in theory at least, clang supports using libstdc++, so one can use any of the two compilers with any of the two C++ standard libraries.

modern gcc supports libc++

in theory at least, clang supports using libstdc++, so one can use any of the two compilers with any of the two C++ standard libraries.

In fact Apple clang did, I think up to 10.8 libstdc++ was the default runtime (despite appearance of libc++ in 10.7). Never heard anyone using modern clang with libstdc++ though, at least on macOS.

Never heard anyone using modern clang with libstdc++ though, at least on macOS.

Isn't this the default on Linux, to use libstdc++ with clang ? Cf. e.g. https://wiki.gentoo.org/wiki/Clang#Configuration

One way or another, libstdc++ on a modern macOS will make it near-impossible to use OS libraries. So this has a purely academic value, at most.

There is certainly value in building with GCC on macOS, it should work more robustly. For SciPy we rely on Gfortran - no alternative, since Apple doesn't ship a Fortran compiler.

I'll have a go at fixing this up now in both NumPy and SciPy.

we rely on Gfortran

sure, that's what everyone does, there is no reliable open-source alternative, as far as I know.

Attempted fix in gh-27729. @barracuda156 or @tylerjereddy it'd be useful if one of you could confirm that this indeed closes the issue.

@rgommers For some reason I do not get an error with numpy without patching this, maybe MacPorts uses the older numpy which does not have the issue? It is at 1.26.4 now.

For scipy the fix is needed though.

@barracuda156 that makes sense; f2py isn't used by numpy itself at build time, since there is no Fortran code in numpy. My question was indeed if it fixes the scipy issue, which you confirmed (and @tylerjereddy did as well, on the PR), so all good here.

@rgommers Wait, I did not yet test scipy exactly with your patch (though I think it should work).
Do I need to rebuild numpy with this patch first and then build scipy, or just apply the patch to a corresponding spot in scipy without rebuilding numpy?

Do I need to rebuild numpy with this patch first and then build scipy

Yes, this indeed. The scipy 1.14.1 build will be fixed when building against a numpy with this patch included.

or just apply the patch to a corresponding spot in scipy without rebuilding numpy?

This isn't an option, since the patch only changes the f2py-generated C code that is used by scipy to build a number of SciPy's extension modules.

Turns out that numpy 1.26.4 had the code which did not cause issues to begin with:

cppmacros["F2PY_THREAD_LOCAL_DECL"] = """
#ifndef F2PY_THREAD_LOCAL_DECL
#if defined(_MSC_VER)
#define F2PY_THREAD_LOCAL_DECL __declspec(thread)
#elif defined(NPY_OS_MINGW)
#define F2PY_THREAD_LOCAL_DECL __thread
#elif defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 201112L)
#define F2PY_THREAD_LOCAL_DECL _Thread_local
#elif defined(__GNUC__) \\
      && (__GNUC__ > 4 || (__GNUC__ == 4 && (__GNUC_MINOR__ >= 4)))
#define F2PY_THREAD_LOCAL_DECL __thread
#endif
#endif
"""

This should just pick _Thread_local with gcc, which works.

The patch does not apply onto 1.26.4, and numpy/_core/meson.build does not even exist yet.

I could try to build a suitably newer version of numpy, but that may require a bit of time to sort it out, rebase patches, possibly update some dependencies.

@ rgommers Or will it suffice for the testing purpose just to edit manually the script which numpy installs?
This file has the related code: /opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/numpy/f2py/cfuncs.py (and it has <threads.h> and the rest).

Instead of passing __STDC_NO_THREADS__ to scipy build I can edit cfuncs.py and try building scipy

@barracuda156 yes manually editing cfuncs.py will work - that seems much easier indeed.

Running the build now, will update in a while.

@rgommers Yes, it works as expected. Thank you!

Great - and thanks for all your help!

@dimpase Blocks are only used in ObjC, which is a minuscule part of open-source software. (Besides, gcc upstream plans to support blocks somewhat soon, though I am not aware of intended timeline.)

Also it is good for code quality. GCC is one of the two major compilers with its own C++ runtime. Quality code should work with it on whatever system, at least as long as it is C/C++/Fortran.

PS. I don't know what makes you think that blocks are only used in ObjC (in macOS or in general).
Please see https://en.wikipedia.org/wiki/Blocks_(C_language_extension)
In @sagemath we switched to clang on macOS many years ago, due to gcc being unable to compile C/C++ code using Apple's macOS headers in a nontrivial way.