Thanks to visit codestin.com
Credit goes to github.com

Skip to content

MNT Use conda-forge instead of the main channel in CI #21337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Oct 18, 2021
Merged

MNT Use conda-forge instead of the main channel in CI #21337

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
809ecfa
testing channels
adrinjalali Oct 15, 2021
f427b21
move to matplotlib 2.2.3 since 2.2.2 is not available on conda-forge
adrinjalali Oct 15, 2021
1880d0b
change matplotlib min version to 2.2.3 in docs
adrinjalali Oct 15, 2021
b79a2c0
testing channels
adrinjalali Oct 15, 2021
9f92c32
move to matplotlib 2.2.3 since 2.2.2 is not available on conda-forge
adrinjalali Oct 15, 2021
2bfec4e
change matplotlib min version to 2.2.3 in docs
adrinjalali Oct 15, 2021
757541d
Merge branch 'channel-test' of github.com:adrinjalali/scikit-learn in…
adrinjalali Oct 15, 2021
139fde4
test scipy 1.2.0
adrinjalali Oct 15, 2021
ec4598c
Revert "test scipy 1.2.0"
adrinjalali Oct 15, 2021
8cf732c
Merge branch 'main' into channel-test
ogrisel Oct 16, 2021
c60c083
Use conda-forge on all Azure builds explicitly
ogrisel Oct 16, 2021
931acd2
MAINT display more runtime info before launching tests
ogrisel Oct 16, 2021
fd6f0d8
Introspect python runtime from outside of the scikit-learn source folder
ogrisel Oct 16, 2021
9f98d33
Use a slightly more recent version of threadpoolctl to debug segfault…
ogrisel Oct 16, 2021
88958da
Merge remote-tracking branch 'origin/main' into pr/adrinjalali/21337
glemaitre Oct 18, 2021
68b1cd8
MNT switch back to main channel for segfault
glemaitre Oct 18, 2021
c740e35
Apply suggestions from code review
glemaitre Oct 18, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
.. |SciPyMinVersion| replace:: 1.1.0
.. |JoblibMinVersion| replace:: 0.11
.. |ThreadpoolctlMinVersion| replace:: 2.0.0
.. |MatplotlibMinVersion| replace:: 2.2.2
.. |MatplotlibMinVersion| replace:: 2.2.3
.. |Scikit-ImageMinVersion| replace:: 0.14.5
.. |PandasMinVersion| replace:: 0.25.0
.. |SeabornMinVersion| replace:: 0.9.0
Expand Down
21 changes: 13 additions & 8 deletions azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,9 @@ jobs:
)
)
matrix:
pylatest_conda_mkl:
pylatest_conda_forge_mkl:
DISTRIB: 'conda'
CONDA_CHANNEL: 'conda-forge'
PYTHON_VERSION: '*'
BLAS: 'mkl'
COVERAGE: 'false'
Expand All @@ -116,8 +117,9 @@ jobs:
not(contains(dependencies['git_commit']['outputs']['commit.message'], '[ci skip]'))
)
matrix:
pylatest_conda_mkl:
pylatest_conda_forge_mkl:
DISTRIB: 'conda'
CONDA_CHANNEL: 'conda-forge'
PYTHON_VERSION: '*'
BLAS: 'mkl'
COVERAGE: 'true'
Expand All @@ -135,8 +137,9 @@ jobs:
ne(variables['Build.Reason'], 'Schedule')
)
matrix:
py37_conda:
py37_conda_forge_openblas_ubuntu_1804:
DISTRIB: 'conda'
CONDA_CHANNEL: 'conda-forge'
PYTHON_VERSION: '3.7'
BLAS: 'openblas'
COVERAGE: 'false'
Expand Down Expand Up @@ -164,14 +167,15 @@ jobs:
THREADPOOLCTL_VERSION: 'min'
COVERAGE: 'false'
# Linux + Python 3.7 build with OpenBLAS and without SITE_JOBLIB
py37_conda_openblas:
py37_conda_defaults_openblas:
DISTRIB: 'conda'
CONDA_CHANNEL: 'defaults' # Anaconda main channel
Copy link
Member

@ogrisel ogrisel Oct 18, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to use the anaconda main channel to test against scipy='min' (that is 1.1.0 currently) because the version shipped in conda-forge has a segfault in cdist.

It's probably not worth wasting time debugging or reporting this segfault because this is a really old version of scipy that we probably won't support anymore once we bump the dependency on Python 3.8 for instance.

PYTHON_VERSION: '3.7'
BLAS: 'openblas'
NUMPY_VERSION: 'min'
SCIPY_VERSION: 'min'
MATPLOTLIB_VERSION: 'min'
THREADPOOLCTL_VERSION: '2.0.0'
THREADPOOLCTL_VERSION: '2.2.0'
# Linux environment to test the latest available dependencies and MKL.
# It runs tests requiring lightgbm, pandas and PyAMG.
pylatest_pip_openblas_pandas:
Expand Down Expand Up @@ -206,7 +210,7 @@ jobs:
# disable pytest xdist due to unknown bug with 32-bit container
PYTEST_XDIST_VERSION: 'none'
PYTEST_VERSION: 'min'
THREADPOOLCTL_VERSION: 'min'
THREADPOOLCTL_VERSION: '2.2.0'

- template: build_tools/azure/posix.yml
parameters:
Expand Down Expand Up @@ -242,13 +246,14 @@ jobs:
ne(variables['Build.Reason'], 'Schedule')
)
matrix:
py37_conda_mkl:
py37_conda_forge_mkl:
DISTRIB: 'conda'
CONDA_CHANNEL: 'conda-forge'
PYTHON_VERSION: '3.7'
CHECK_WARNINGS: 'true'
PYTHON_ARCH: '64'
PYTEST_VERSION: '*'
COVERAGE: 'true'
DISTRIB: 'conda'
py37_pip_openblas_32bit:
PYTHON_VERSION: '3.7'
PYTHON_ARCH: '32'
3 changes: 2 additions & 1 deletion build_tools/azure/install.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ make_conda() {
if [[ "$DISTRIB" == *"mamba"* ]]; then
mamba create -n $VIRTUALENV --yes $TO_INSTALL
else
conda config --show
conda create -n $VIRTUALENV --yes $TO_INSTALL
fi
source activate $VIRTUALENV
Expand All @@ -38,7 +39,7 @@ source build_tools/shared.sh
if [[ "$DISTRIB" == "conda" || "$DISTRIB" == *"mamba"* ]]; then

if [[ "$CONDA_CHANNEL" != "" ]]; then
TO_INSTALL="-c $CONDA_CHANNEL"
TO_INSTALL="--override-channels -c $CONDA_CHANNEL"
else
TO_INSTALL=""
fi
Expand Down
30 changes: 14 additions & 16 deletions build_tools/azure/test_script.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,20 @@ if [[ "$BUILD_WITH_ICC" == "true" ]]; then
source /opt/intel/oneapi/setvars.sh
fi

python --version
python -c "import numpy; print('numpy %s' % numpy.__version__)"
python -c "import scipy; print('scipy %s' % scipy.__version__)"
python -c "\
try:
import pandas
print('pandas %s' % pandas.__version__)
except ImportError:
print('pandas not installed')
"
python -c "import multiprocessing as mp; print('%d CPUs' % mp.cpu_count())"
pip list
mkdir -p $TEST_DIR
cp setup.cfg $TEST_DIR
cd $TEST_DIR

python -c "import sklearn; sklearn.show_versions()"
python -m threadpoolctl -i sklearn

if ! command -v conda &> /dev/null
then
pip list
else
# conda list provides more info than pip list (when available)
conda list
fi

TEST_CMD="python -m pytest --showlocals --durations=20 --junitxml=$JUNITXML"

Expand All @@ -46,10 +48,6 @@ if [[ "$PYTEST_XDIST_VERSION" != "none" ]]; then
TEST_CMD="$TEST_CMD -n2"
fi

mkdir -p $TEST_DIR
cp setup.cfg $TEST_DIR
cd $TEST_DIR

set -x
$TEST_CMD --pyargs sklearn
set +x
2 changes: 1 addition & 1 deletion doc/whats_new/v1.1.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Minimal dependencies
--------------------

Version 1.1.0 of scikit-learn requires python 3.7+, numpy 1.14.6+ and
scipy 1.1.0+. Optional minimal dependency is matplotlib 2.2.2+.
scipy 1.1.0+. Optional minimal dependency is matplotlib 2.2.3+.

Put the changes in their relevant module.

Expand Down
2 changes: 1 addition & 1 deletion sklearn/_min_dependencies.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
"joblib": (JOBLIB_MIN_VERSION, "install"),
"threadpoolctl": (THREADPOOLCTL_MIN_VERSION, "install"),
"cython": (CYTHON_MIN_VERSION, "build"),
"matplotlib": ("2.2.2", "benchmark, docs, examples, tests"),
"matplotlib": ("2.2.3", "benchmark, docs, examples, tests"),
"scikit-image": ("0.14.5", "docs, examples, tests"),
"pandas": ("0.25.0", "benchmark, docs, examples, tests"),
"seaborn": ("0.9.0", "docs, examples"),
Expand Down