Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Q: Prerelease wheels no longer have OpenBLAS #15580

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
larsoner opened this issue Feb 17, 2020 · 11 comments
Closed

Q: Prerelease wheels no longer have OpenBLAS #15580

larsoner opened this issue Feb 17, 2020 · 11 comments
Labels
00 - Bug 14 - Release
Milestone
1.19.0 release

Comments

@larsoner
Copy link
Contributor

larsoner commented Feb 17, 2020 •
edited
Loading

As of a day or two ago, some builds that use:

pip install -f "https://7933911d6844c6c53a7d-47bd50c35cd79bd838daf386af554a83.ssl.cf2.rackcdn.com" --pre numpy

seem to have gotten a lot slower. Digging into it, two days ago (e.g., on 1.19.0.dev0+af0dfce) we saw openblas listed as the linked library (i.e., had something other than NOT AVAILABLE listed under it in np.show_config()), now on 1.19.0.dev0+2eedaa4 we get:

np.show_config() 
$ python -c "import numpy; numpy.show_config()"
blas_mkl_info:
  NOT AVAILABLE
blis_info:
  NOT AVAILABLE
openblas_info:
  NOT AVAILABLE
atlas_3_10_blas_threads_info:
  NOT AVAILABLE
atlas_3_10_blas_info:
  NOT AVAILABLE
atlas_blas_threads_info:
  NOT AVAILABLE
atlas_blas_info:
  NOT AVAILABLE
accelerate_info:
  NOT AVAILABLE
blas_info:
  NOT AVAILABLE
blas_src_info:
  NOT AVAILABLE
blas_opt_info:
  NOT AVAILABLE
lapack_mkl_info:
  NOT AVAILABLE
openblas_lapack_info:
  NOT AVAILABLE
openblas_clapack_info:
  NOT AVAILABLE
flame_info:
  NOT AVAILABLE
atlas_3_10_threads_info:
  NOT AVAILABLE
atlas_3_10_info:
  NOT AVAILABLE
atlas_threads_info:
  NOT AVAILABLE
atlas_info:
  NOT AVAILABLE
lapack_info:
  NOT AVAILABLE
lapack_src_info:
  NOT AVAILABLE
lapack_opt_info:
  NOT AVAILABLE
numpy_linalg_lapack_lite:
    language = c
    define_macros = [('HAVE_BLAS_ILP64', None), ('BLAS_SYMBOL_SUFFIX', '64_')]

Is this intended, or a bug when migrating to manylinux2010 builds?

Apologies if this should live in https://github.com/MacPython/numpy-wheels instead, or if I'm missing something obvious.
@larsoner larsoner mentioned this issue Feb 17, 2020
Merged
@charris charris added this to the 1.19.0 release milestone Feb 17, 2020
@charris
Copy link
Member

charris commented Feb 17, 2020

It is a bug. Using manylinux2010 for the pre-wheels is an attempt to discover such problems early.

@charris
Copy link
Member

charris commented Feb 17, 2020 •
edited
Loading

We probably need to compile OpenBLAS libraries for manylinux2010, they may even already exist. @mattip @tylerjereddy Thoughts?

EDIT: Are such libraries distinguishable by name?

@mattip
Copy link
Member

mattip commented Feb 17, 2020

I don't see why this should happen.

Our different CI systems use different methods to find and download the OpenBLAS libs. This repo's CI uses tools/openblas_support.py, which has the manylinux1 part of the name hard-coded into the file name, but that is not relevant here.

numpy-wheels uses the gfortran_utils subrepo function get_gf_lib_for_suf, which gets the file name from get_distutils_platform which is also hard-coded to manylinux1.

@charris can you point me to the log where that build happened? @larsoner what platform and OS are you using?

@mattip
Copy link
Member

mattip commented Feb 17, 2020

I found the build log.

It seems the libfortran needed is not found when building wheels, here is the linked log lin:

gcc/x86_64-redhat-linux/8/ld: warning: libgfortran.so.3, needed \
      by /usr/local/lib/libopenblas.so, not found

@mattip
Copy link
Member

mattip commented Feb 17, 2020 •
edited
Loading

to prevent this in the future, the numpy-wheels build should have a check like the one in tools/.travis-test.sh

$PYTHON numpy/tools/openblas_support.py --check_version $OPENBLAS_VERSION

@charris
Copy link
Member

charris commented Feb 17, 2020

@mattip Nice find. I suppose we could install the libraries but ideally they would be part of the docker image.

@mattip
Copy link
Member

mattip commented Feb 17, 2020

Working on it as part of MacPython/numpy-wheels#73, but now I am getting a failure on the 32-bit build not passing the latest test_umath_accuracy.py PR.

This was referenced Feb 19, 2020
Merged
Merged
Merged
@seberg
Copy link
Member

seberg commented May 14, 2020

@mattip is this fixed and/or the milestone not important?

@charris
Copy link
Member

charris commented May 14, 2020

Yes, it has been fixed in the wheels repository, I checked a couple of days ago. Will do so again when the first 1.19 release is made.

@charris charris closed this as completed May 14, 2020
@mattip
Copy link
Member

mattip commented May 14, 2020

The wheel building (on linux and mac) now checks that the expected OpenBLAS is used. Unfortunately I still haven't found a good, simple way to check windows.

@charris
Copy link
Member

charris commented May 14, 2020

A quick way to check is the size, wheels -- now that they are stripped -- should run 10 - 14 MB compressed. OpenBLAS is the single largest part of the content.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug 14 - Release
Projects
None yet
Milestone
1.19.0 release
Development

No branches or pull requests
4 participants
@seberg @charris @mattip @larsoner