I think I'm fine with both options. The one that is implemented in this PR has the advantage of being already implemented ;)

I would be curious as to what other people think.

Anyway I am really glad that we can finally get rid of the embedded cblas source code 🎆

BTW, I merged #13084 so you can rebase this on top of master.

get_blas_info seems to be failing on one Travis instance :(

I also just noticed the following slowdown:

========================== slowest 20 test durations ===========================
25.10s call     metrics/tests/test_pairwise.py::test_pairwise_distances_data_derived_params[Y is not X-pairwise_distances_chunked-seuclidean-2]
25.00s call     metrics/tests/test_pairwise.py::test_pairwise_distances_data_derived_params[Y is not X-pairwise_distances_chunked-mahalanobis-2]
24.57s call     metrics/tests/test_pairwise.py::test_pairwise_distances_data_derived_params[Y is X-pairwise_distances_chunked-mahalanobis-2]
24.41s call     metrics/tests/test_pairwise.py::test_pairwise_distances_data_derived_params[Y is X-pairwise_distances_chunked-seuclidean-2]

I am pretty sure that those tests were running much faster before. However it was already there in some builds on travis that dates back to more than 24h ago, that is prior to merging #13084.

BTW could you please run a quick benchmark to check that the new TRON solver code that uses function pointers to call BLAS is not significantly slower than calling the embedded ATLAS cblas routines directly.

For instance, you can try with LinearSVC or LogisticRegression on a make_classification problem large enough to last a couple of seconds.

I also just noticed the following slowdown:

========================== slowest 20 test durations ===========================
25.10s call metrics/tests/test_pairwise.py::test_pairwise_distances_data_derived_params[Y is not X-pairwise_distances_chunked-seuclidean-2]
25.00s call metrics/tests/test_pairwise.py::test_pairwise_distances_data_derived_params[Y is not X-pairwise_distances_chunked-mahalanobis-2]
24.57s call metrics/tests/test_pairwise.py::test_pairwise_distances_data_derived_params[Y is X-pairwise_distances_chunked-mahalanobis-2]
24.41s call metrics/tests/test_pairwise.py::test_pairwise_distances_data_derived_params[Y is X-pairwise_distances_chunked-seuclidean-2]

I don't see how it would be related to cblas, since for those metrics we directly call scipy cdist/pdist.
However, there's definitely something going on. First, the slowdown only appear on the distrib=ubuntu job on travis. Then, it only affects pairwise_distances_chunked (and not pairwise_distances) when n_jobs=2. I think this requires a dedicated issue (opened #13208).

I am pretty sure that those tests were running much faster before. However it was already there in some builds on travis that dates back to more than 24h ago, that is prior to merging #13084.

I just looked at travis on master and it's there since the test was added.

quick question:
should _cython_blas_helpers.h go into utils, utils/src or svm/src/liblinear (where it's only used) ?

I just looked at travis on master and it's there since the test was added.

Alright. I thought I had run the full test suite on scikit-learn master locally yesterday and I did not recall those this is why I was misled. This should be ignored for this PR. Those tests should be sped up but this is unrelated.

should _cython_blas_helpers.h go into utils, utils/src or svm/src/liblinear (where it's only used) ?

I would vote for svm/src/liblinear for now. If we ever need to reuse it somewhere else we will move it to utils/src while making the BLAS covered API exposed in this .h file more exhaustive (that is to cover the full BLAS API which is not needed now).

Weird failure in circle ci:

generating gallery for auto_examples/linear_model... [ 93%] plot_lasso_model_selection.py
generating gallery for auto_examples/linear_model... [ 96%] plot_sparse_logistic_regression_20newsgroups.py
generating gallery for auto_examples/linear_model... [100%] plot_sgd_early_stopping.py
Killed
Makefile:43: recipe for target 'html' failed
make: *** [html] Error 137
Exited with code 2

probably unrelated but annoying. I cannot find any other issue that references this failure.

Here are the result of the small benchmark you propose.

from sklearn.datasets import make_classification                                                                                    
from sklearn.linear_model import LogisticRegression                                                                                 

X, y = make_classification(n_samples=100000, n_features=100, n_informative=50, n_classes=5)                                         
lr = LogisticRegression(solver='liblinear', random_state=0, multi_class='auto')                                                     
%timeit lr.fit(X,y)     

First, on a machine with no blas installed, forcing the use of bundled cblas, on master:
23.5 s ± 110 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
and on this branch, hence using blas shipped with scipy (pip installed):
22.6 s ± 133 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
So clearly no regression here :)

Then, on a machine with mkl installed, on master:
23.4 s ± 246 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
and on this branch (scipy uses the installed mkl):
24 s ± 216 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
There might be a small overhead in using scipy cython bindings instead of direct blas call but it seems quite negligible.

I switched to [MRG] but there's still the circle ci failure which I don't understand. Does not seem to appear on master...

green \o/

Merged! Thank you very much for this work @jeremiedbb!

Hum, circle CI has failed with the same error on master: https://circleci.com/gh/scikit-learn/scikit-learn/47352

So this is probably caused by the change in this PR unfortunately. Maybe this is is causing some extra memory usage when running the linear model examples?

I don't think so. The same failure appeared on master earlier today:
https://circleci.com/gh/scikit-learn/scikit-learn/47309

Alright then.

Thanks so much for tackling this @jeremiedbb ! 😄