use static linking to link to OpenBLAS and GFortran

Pretty sure this is a lot harder than it first appears. Last time I checked we would need a custom build of the gcc suite from source so that libgfortran could be compiled with --with-pic. Pretty much no versions of gcc/gfortran that are pre-built on linux distros/CI will have this available. The Julia team made the same observation.

Maybe some of the stuff @pv appears to have done recently for 64-bit int support with linear algebra backends could be helpful: #15012

Ah, I see you've already reference that PR above.

Are there build recipes for the wheels?

https://github.com/MacPython/numpy-wheels

The OpenBLAS binaries are selected after being pre-built in:

https://github.com/MacPython/openblas-libs

I think some improvements/expansions to the workflow are coming, though I don't know all the details yet..

The Julia team made the same observation.

It looks like their issue was pretty much related to libquadmath and the fact that there was no available -static-libquadmath in GCC...

On the other hand NumPy does not link to libquadmath:

$ NP_LIBS_DIR=$(python3 -c "import numpy, os; print(os.path.abspath(os.path.join(os.path.dirname(numpy.__file__), '.libs')))")
$ NP_LIBS_DIR ldd $NP_LIBS_DIR/*
/home/jan/.local/lib/python3.5/site-packages/numpy/.libs/libgfortran-ed201abd.so.3.0.0:
	linux-vdso.so.1 =>  (0x00007ffcb13f3000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fdc71dc0000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fdc719f6000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fdc723c3000)
/home/jan/.local/lib/python3.5/site-packages/numpy/.libs/libopenblasp-r0-39a31c03.2.18.so:
	linux-vdso.so.1 =>  (0x00007ffee47aa000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f10120f6000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1011ed9000)
	libgfortran-ed201abd.so.3.0.0 => /home/jan/.local/lib/python3.5/site-packages/numpy/.libs/libgfortran-ed201abd.so.3.0.0 (0x00007f1011bdf000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1011815000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f1014a2a000)

Not so convincing indication that -static-libgfortran works:

$ echo 'program hello; print *, "Hello World!"; end program' | gfortran -x f95 -ffree-form -static-libgfortran -
$ ldd a.out 
	linux-vdso.so.1 (0x00007ffe68ff7000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff9dc943000)
	libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007ff9dc703000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff9dc365000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff9dbf74000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ff9dcd91000)

Compare to the case when you compile without -static-libgfortran...

libopenblas.a provided by the OpenBLAS build system is in my experience PIC. (I compiled here libopenblas.a and used it further down the stack where the final product was Matlab MEX, which is SO.)

So optimistically, one just uses FC=gfortran -static-libgfortran in OpenBLAS and NumPy (if any Fortran code in NumPy) and links it all with libopenblas.a. But there are probably caveats.

We might be talking about different things---I think @carlkl suggested we shouldn't statically link OpenBLAS to NumPy for Windows wheels, but that it would be beneficial to statically link the gcc runtime into the dynamic OpenBLAS lib we distribute with wheels. Maybe you could convince them otherwise, but they know the Windows stuff pretty well :)

FWIW, I did that static link of gcc runtime into OpenBLAS in #13191, but there wasn't much interest. Then again, doing that consistently across all platforms would have been tricky, so I can't blame the team for that!

I think the libquadmath special requirement is for statically linking gcc runtime into shared library OpenBLAS---you're saying that is not needed in this case because libopenblas.a is much easier to generate with gcc runtime baked in than the shared library equivalent?

One other question---have you thought about the various PEPs related to wheels standards? I think some of them go through a pretty substantial design effort to specify how libs get provided, so that might be another consideration. Nathaniel probably knows a lot about that.

Ok, one further thought---what about staying open to "hot-swapping" of linear algebra backends for NumPy? I think @stefanv reminded me of that a few months ago when I got too excited about static linking or something :)

I can only comment on CPython for Windows, but not Mac or Linux:

It is better to prefer static linking of all GCC runtime libraries for building binary extensions for several reasons:

• There is not so much size overhead to the GCC compiled pyd binaries beyond one's belief.
• To avoid problems with using a wrong variant of libstdc++-6.dll that could have been loaded beforehand into process space from another python package. Unfortunately there are libstdc++-6.dll with different ABI's, exception handling and threading variants all around with the same name.

Concerning OpenBLAS and libquadmath:

OpenBLAS should be linked without quadmath dependency to avoid possible problems with licensing
(LGPL without exeptions).

Build NumPy with statically linked OpenBLAS and test with Matlab

The main trick is extra_link_args = -l:libopenblas.a in [openblas] section of site.cfg which ensures static linking.

Tested with

• Ubuntu 18.04.3 LTS
• Python 3.6.9
• gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
• Matlab R2019b Update 1 (9.7.0.1216025) 64-bit (glnxa64)

The script only operates in the current dir and changes PYTHONPATH, so it should not break your installations.

#!/bin/bash
set -e


OPENBLAS_VERSION=0.3.7
NUMPY_VERSION=1.17.4
PREFIX="${PWD}/local"
BUILDDIR="${PWD}/build"
PYTHON="python3"
PYTHON_VERSION=$(${PYTHON} -c"import sys, sysconfig; sys.stdout.write(sysconfig.get_python_version())")
export PYTHONPATH="${PREFIX}/lib/python${PYTHON_VERSION}/site-packages:${PYTHONPATH}"
export NPY_NUM_BUILD_JOBS=$(nproc)
MATLAB="matlab"


# Create build dir
mkdir -p "${BUILDDIR}"

# Download and extract libopenblas.a
cd "${BUILDDIR}"
wget -c https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com/openblas-v${OPENBLAS_VERSION}-manylinux1_x86_64.tar.gz
tar -C ${PWD} --wildcards --strip-components=2 -xzf openblas-v${OPENBLAS_VERSION}-manylinux1_x86_64.tar.gz usr/local/lib/libopenblas*.a
LIBDIR="${PWD}/lib"

# Download and extract NumPy
cd "${BUILDDIR}"
wget -c https://github.com/numpy/numpy/releases/download/v${NUMPY_VERSION}/numpy-${NUMPY_VERSION}.tar.gz
tar -xzf numpy-${NUMPY_VERSION}.tar.gz

# Build Numpy
cd "${BUILDDIR}/numpy-${NUMPY_VERSION}"
cat - > site.cfg <<EOF
[openblas]
libraries = openblas
library_dirs = ${LIBDIR}
extra_link_args = -l:libopenblas.a
EOF
${PYTHON} -m pip install -vv --log=setup.log --prefix="${PREFIX}" .
export PYTHONPATH

# Test NumPy
cd /tmp
${PYTHON} -c 'import numpy; numpy.test()'

# Test Matlab
# NB: We test Numpy's OpenBLAS by np.linalg.inv() and Matlab's MKL by polyfit()
cd /tmp
${MATLAB} -nojvm -batch 'disp(py.numpy.linalg.inv(magic(3))); disp(polyfit(1:10, sin(1:10), 2))'

echo "NumPy for Matlab installed successfully."
echo "Install by 'export PYTHONPATH=${PYTHONPATH}'"

Linking NumPy dynamically to Matlab's MKL

This would be possibly a more convenient way. MKL shipped with Matlab is shipped with 64-bit-ints interface but without 64_ suffixes in symbols. Hence a modification of #15012, which did the most of the work, seems to be needed to build with 64-bit ints but without the suffix. I will have a look.

Sounds like this can be achieved now with the last PR linked – allowing to use 64 bit BLAS without suffix (although I won't claim to follow exactly why you need to pick up the Matlab MKL for use with NumPy exactly).

Going to close, if that is wrong, please just do a note/reopen.