Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit dce919f

Browse files
jjerphanjjerphan
committed
Merge branch 'main' into pairwise-distances-argkmin-plug
2 parents 3448b01 + 49043fc commit dce919f

28 files changed

+349
-61
lines changed

‎build_tools/azure/install.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,10 +23,10 @@ make_conda() {
2323
}
2424

2525
setup_ccache() {
26-
echo "Setting up ccache"
26+
echo "Setting up ccache with CCACHE_DIR=${CCACHE_DIR}"
2727
mkdir /tmp/ccache/
2828
which ccache
29-
for name in gcc g++ cc c++ x86_64-linux-gnu-gcc x86_64-linux-gnu-c++; do
29+
for name in gcc g++ cc c++ clang clang++ i686-linux-gnu-gcc i686-linux-gnu-c++ x86_64-linux-gnu-gcc x86_64-linux-gnu-c++ x86_64-apple-darwin13.4.0-clang x86_64-apple-darwin13.4.0-clang++; do
3030
ln -s $(which ccache) "/tmp/ccache/${name}"
3131
done
3232
export PATH="/tmp/ccache/:${PATH}"

‎build_tools/azure/posix-docker.yml

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,18 +36,31 @@ jobs:
3636
DISTRIB: ''
3737
DOCKER_CONTAINER: ''
3838
SHOW_SHORT_SUMMARY: 'false'
39+
CCACHE_DIR: $(Pipeline.Workspace)/ccache
40+
CCACHE_COMPRESS: '1'
3941
strategy:
4042
matrix:
4143
${{ insert }}: ${{ parameters.matrix }}
4244

4345
steps:
46+
- task: Cache@2
47+
inputs:
48+
key: '"ccache-v1" | "$(Agent.JobName)" | "$(Build.BuildNumber)"'
49+
restoreKeys: |
50+
"ccache-v1" | "$(Agent.JobName)"
51+
path: $(CCACHE_DIR)
52+
displayName: ccache
53+
continueOnError: true
54+
- script: >
55+
mkdir -p $CCACHE_DIR
4456
# Container is detached and sleeping, allowing steps to run commands
4557
# in the container. The TEST_DIR is mapped allowing the host to access
4658
# the JUNITXML file
4759
- script: >
4860
docker container run --rm
4961
--volume $TEST_DIR:/temp_dir
5062
--volume $PWD:/io
63+
--volume $CCACHE_DIR:/ccache
5164
-w /io
5265
--detach
5366
--name skcontainer
@@ -71,6 +84,8 @@ jobs:
7184
-e SKLEARN_SKIP_NETWORK_TESTS=$SKLEARN_SKIP_NETWORK_TESTS
7285
-e BLAS=$BLAS
7386
-e CPU_COUNT=$CPU_COUNT
87+
-e CCACHE_DIR=/ccache
88+
-e CCACHE_COMPRESS=$CCACHE_COMPRESS
7489
$DOCKER_CONTAINER
7590
sleep 1000000
7691
displayName: 'Start container'

‎build_tools/azure/posix.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,9 @@ jobs:
5050
condition: startsWith(variables['DISTRIB'], 'conda')
5151
- task: Cache@2
5252
inputs:
53-
key: '"$(Agent.JobName)"'
53+
key: '"ccache-v1" | "$(Agent.JobName)" | "$(Build.BuildNumber)"'
54+
restoreKeys: |
55+
"ccache-v1" | "$(Agent.JobName)"
5456
path: $(CCACHE_DIR)
5557
displayName: ccache
5658
continueOnError: true

‎doc/glossary.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1604,6 +1604,7 @@ functions or non-estimator constructors.
16041604
number of different distinct random seeds. Popular integer
16051605
random seeds are 0 and `42
16061606
<https://en.wikipedia.org/wiki/Answer_to_the_Ultimate_Question_of_Life%2C_the_Universe%2C_and_Everything>`_.
1607+
Integer values must be in the range `[0, 2**32 - 1]`.
16071608

16081609
A :class:`numpy.random.RandomState` instance
16091610
Use the provided random state, only affecting other users

‎doc/whats_new/v1.1.rst

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -273,6 +273,13 @@ Changelog
273273
F-statistic).
274274
:pr:`17819` by :user:`Juan Carlos Alfaro Jiménez <alfaro96>`.
275275

276+
:mod:`sklearn.gaussian_process`
277+
...............................
278+
279+
- |Fix| :class:`gaussian_process.GaussianProcessClassifier` raises
280+
a more informative error if `CompoundKernel` is passed via `kernel`.
281+
:pr:`22223` by :user:`MarcoM <marcozzxx810>`.
282+
276283
:mod:`sklearn.impute`
277284
.....................
278285

@@ -322,6 +329,9 @@ Changelog
322329
- |Enhancement| :class:`linear_model.QuantileRegressor` support sparse input
323330
for the highs based solvers.
324331
:pr:`21086` by :user:`Venkatachalam Natchiappan <venkyyuvy>`.
332+
In addition, those solvers now use the CSC matrix right from the
333+
beginning which speeds up fitting.
334+
:pr:`22206` by :user:`Christian Lorentzen <lorentzenchr>`.
325335

326336
- |Enhancement| Rename parameter `base_estimator` to `estimator` in
327337
:class:`linear_model.RANSACRegressor` to improve readability and consistency.
@@ -334,6 +344,11 @@ Changelog
334344
:pr:`21481` by :user:`Guillaume Lemaitre <glemaitre>` and
335345
:user:`Andrés Babino <ababino>`.
336346

347+
- |Enhancement| :func:`linear_model.ElasticNet` and
348+
and other linear model classes using coordinate descent show error
349+
messages when non-finite parameter weights are produced. :pr:`22148`
350+
by :user:`Christian Ritter <chritter>` and :user:`Norbert Preining <norbusan>`.
351+
337352
- |Fix| :class:`linear_model.ElasticNetCV` now produces correct
338353
warning when `l1_ratio=0`.
339354
:pr:`21724` by :user:`Yar Khine Phyo <yarkhinephyo>`.
@@ -359,6 +374,11 @@ Changelog
359374
A deprecation cycle was introduced.
360375
:pr:`21576` by :user:`Paul-Emile Dugnat <pedugnat>`.
361376

377+
- |API| The `"wminkowski"` metric of :class:`sklearn.metrics.DistanceMetric` is deprecated
378+
and will be removed in version 1.3. Instead the existing `"minkowski"` metric now takes
379+
in an optional `w` parameter for weights. This deprecation aims at remaining consistent
380+
with SciPy 1.8 convention. :pr:`21873` by :user:`Yar Khine Phyo <yarkhinephyo>`
381+
362382
- |Fix| :func:`metrics.silhouette_score` now supports integer input for precomputed
363383
distances. :pr:`22108` by `Thomas Fan`_.
364384

@@ -382,6 +402,11 @@ Changelog
382402
splits failed. Similarly raise an error during grid-search when the fits for
383403
all the models and all the splits failed. :pr:`21026` by :user:`Loïc Estève <lesteve>`.
384404

405+
- |Enhancement| it is now possible to pass `scoring="matthews_corrcoef"` to all
406+
model selection tools with a `scoring` argument to use the Matthews
407+
correlation coefficient (MCC). :pr:`22203` by :user:`Olivier Grisel
408+
<ogrisel>`.
409+
385410
- |Fix| :class:`model_selection.GridSearchCV`,
386411
:class:`model_selection.HalvingGridSearchCV`
387412
now validate input parameters in `fit` instead of `__init__`.
@@ -412,6 +437,14 @@ Changelog
412437
instead of `__init__`. :pr:`21430` by :user:`Desislava Vasileva <DessyVV>` and
413438
:user:`Lucy Jimenez <LucyJimenez>`.
414439

440+
:mod:`sklearn.neural_network`
441+
.............................
442+
443+
- |Enhancement| :func:`neural_network.MLPClassifier` and
444+
:func:`neural_network.MLPRegressor` show error
445+
messages when optimizers produce non-finite parameter weights. :pr:`22150`
446+
by :user:`Christian Ritter <chritter>` and :user:`Norbert Preining <norbusan>`.
447+
415448
:mod:`sklearn.pipeline`
416449
.......................
417450

‎examples/applications/plot_model_complexity_influence.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,8 @@ def _count_nonzero_coefficients(estimator):
165165
"alpha": 0.001,
166166
"loss": "modified_huber",
167167
"fit_intercept": True,
168-
"tol": 1e-3,
168+
"tol": 1e-1,
169+
"n_iter_no_change": 2,
169170
},
170171
"changing_param": "l1_ratio",
171172
"changing_param_values": [0.25, 0.5, 0.75, 0.9],

‎sklearn/cluster/_optics.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -673,13 +673,13 @@ def cluster_optics_xi(
673673
Parameters
674674
----------
675675
reachability : ndarray of shape (n_samples,)
676-
Reachability distances calculated by OPTICS (`reachability_`)
676+
Reachability distances calculated by OPTICS (`reachability_`).
677677
678678
predecessor : ndarray of shape (n_samples,)
679679
Predecessors calculated by OPTICS.
680680
681681
ordering : ndarray of shape (n_samples,)
682-
OPTICS ordered point indices (`ordering_`)
682+
OPTICS ordered point indices (`ordering_`).
683683
684684
min_samples : int > 1 or float between 0 and 1
685685
The same as the min_samples given to OPTICS. Up and down steep regions

‎sklearn/cluster/tests/test_hierarchical.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -410,6 +410,8 @@ def test_vector_scikit_single_vs_scipy_single(seed):
410410
assess_same_labelling(cut, cut_scipy)
411411

412412

413+
# TODO: Remove filterwarnings in 1.3 when wminkowski is removed
414+
@pytest.mark.filterwarnings("ignore:WMinkowskiDistance:FutureWarning:sklearn")
413415
@pytest.mark.parametrize("metric_param_grid", METRICS_DEFAULT_PARAMS)
414416
def test_mst_linkage_core_memory_mapped(metric_param_grid):
415417
"""The MST-LINKAGE-CORE algorithm must work on mem-mapped dataset.

‎sklearn/discriminant_analysis.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -631,8 +631,10 @@ def transform(self, X):
631631
632632
Returns
633633
-------
634-
X_new : ndarray of shape (n_samples, n_components)
635-
Transformed data.
634+
X_new : ndarray of shape (n_samples, n_components) or \
635+
(n_samples, min(rank, n_components))
636+
Transformed data. In the case of the 'svd' solver, the shape
637+
is (n_samples, min(rank, n_components)).
636638
"""
637639
if self.solver == "lsqr":
638640
raise NotImplementedError(

‎sklearn/ensemble/_voting.py

Lines changed: 27 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -157,8 +157,8 @@ class VotingClassifier(ClassifierMixin, _BaseVoting):
157157
estimators : list of (str, estimator) tuples
158158
Invoking the ``fit`` method on the ``VotingClassifier`` will fit clones
159159
of those original estimators that will be stored in the class attribute
160-
``self.estimators_``. An estimator can be set to ``'drop'``
161-
using ``set_params``.
160+
``self.estimators_``. An estimator can be set to ``'drop'`` using
161+
:meth:`set_params`.
162162
163163
.. versionchanged:: 0.21
164164
``'drop'`` is accepted. Using None was deprecated in 0.22 and
@@ -254,6 +254,18 @@ class VotingClassifier(ClassifierMixin, _BaseVoting):
254254
>>> eclf2 = eclf2.fit(X, y)
255255
>>> print(eclf2.predict(X))
256256
[1 1 1 2 2 2]
257+
258+
To drop an estimator, :meth:`set_params` can be used to remove it. Here we
259+
dropped one of the estimators, resulting in 2 fitted estimators:
260+
261+
>>> eclf2 = eclf2.set_params(lr='drop')
262+
>>> eclf2 = eclf2.fit(X, y)
263+
>>> len(eclf2.estimators_)
264+
2
265+
266+
Setting `flatten_transform=True` with `voting='soft'` flattens output shape of
267+
`transform`:
268+
257269
>>> eclf3 = VotingClassifier(estimators=[
258270
... ('lr', clf1), ('rf', clf2), ('gnb', clf3)],
259271
... voting='soft', weights=[2,1,1],
@@ -434,7 +446,7 @@ class VotingRegressor(RegressorMixin, _BaseVoting):
434446
Invoking the ``fit`` method on the ``VotingRegressor`` will fit clones
435447
of those original estimators that will be stored in the class attribute
436448
``self.estimators_``. An estimator can be set to ``'drop'`` using
437-
``set_params``.
449+
:meth:`set_params`.
438450
439451
.. versionchanged:: 0.21
440452
``'drop'`` is accepted. Using None was deprecated in 0.22 and
@@ -488,13 +500,23 @@ class VotingRegressor(RegressorMixin, _BaseVoting):
488500
>>> from sklearn.linear_model import LinearRegression
489501
>>> from sklearn.ensemble import RandomForestRegressor
490502
>>> from sklearn.ensemble import VotingRegressor
503+
>>> from sklearn.neighbors import KNeighborsRegressor
491504
>>> r1 = LinearRegression()
492505
>>> r2 = RandomForestRegressor(n_estimators=10, random_state=1)
506+
>>> r3 = KNeighborsRegressor()
493507
>>> X = np.array([[1, 1], [2, 4], [3, 9], [4, 16], [5, 25], [6, 36]])
494508
>>> y = np.array([2, 6, 12, 20, 30, 42])
495-
>>> er = VotingRegressor([('lr', r1), ('rf', r2)])
509+
>>> er = VotingRegressor([('lr', r1), ('rf', r2), ('r3', r3)])
496510
>>> print(er.fit(X, y).predict(X))
497-
[ 3.3 5.7 11.8 19.7 28. 40.3]
511+
[ 6.8... 8.4... 12.5... 17.8... 26... 34...]
512+
513+
In the following example, we drop the `'lr'` estimator with
514+
:meth:`~VotingRegressor.set_params` and fit the remaining two estimators:
515+
516+
>>> er = er.set_params(lr='drop')
517+
>>> er = er.fit(X, y)
518+
>>> len(er.estimators_)
519+
2
498520
"""
499521

500522
def __init__(self, estimators, *, weights=None, n_jobs=None, verbose=False):

‎sklearn/gaussian_process/_gpc.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -503,7 +503,8 @@ class GaussianProcessClassifier(ClassifierMixin, BaseEstimator):
503503
kernel : kernel instance, default=None
504504
The kernel specifying the covariance function of the GP. If None is
505505
passed, the kernel "1.0 * RBF(1.0)" is used as default. Note that
506-
the kernel's hyperparameters are optimized during fitting.
506+
the kernel's hyperparameters are optimized during fitting. Also kernel
507+
cannot be a `CompoundKernel`.
507508
508509
optimizer : 'fmin_l_bfgs_b' or callable, default='fmin_l_bfgs_b'
509510
Can either be one of the internally supported optimizers for optimizing
@@ -673,6 +674,9 @@ def fit(self, X, y):
673674
self : object
674675
Returns an instance of self.
675676
"""
677+
if isinstance(self.kernel, CompoundKernel):
678+
raise ValueError("kernel cannot be a CompoundKernel")
679+
676680
if self.kernel is None or self.kernel.requires_vector_input:
677681
X, y = self._validate_data(
678682
X, y, multi_output=False, ensure_2d=True, dtype="numeric"

‎sklearn/gaussian_process/tests/test_gpc.py

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,12 @@
1111
import pytest
1212

1313
from sklearn.gaussian_process import GaussianProcessClassifier
14-
from sklearn.gaussian_process.kernels import RBF, ConstantKernel as C, WhiteKernel
14+
from sklearn.gaussian_process.kernels import (
15+
RBF,
16+
CompoundKernel,
17+
ConstantKernel as C,
18+
WhiteKernel,
19+
)
1520
from sklearn.gaussian_process.tests._mini_sequence_kernel import MiniSeqKernel
1621
from sklearn.exceptions import ConvergenceWarning
1722

@@ -260,3 +265,20 @@ def test_warning_bounds():
260265
"Increasing the bound and calling "
261266
"fit again may find a better value."
262267
)
268+
269+
270+
@pytest.mark.parametrize(
271+
"params, error_type, err_msg",
272+
[
273+
(
274+
{"kernel": CompoundKernel(0)},
275+
ValueError,
276+
"kernel cannot be a CompoundKernel",
277+
)
278+
],
279+
)
280+
def test_gpc_fit_error(params, error_type, err_msg):
281+
"""Check that expected error are raised during fit."""
282+
gpc = GaussianProcessClassifier(**params)
283+
with pytest.raises(error_type, match=err_msg):
284+
gpc.fit(X, y)

‎sklearn/linear_model/_coordinate_descent.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1075,6 +1075,14 @@ def fit(self, X, y, sample_weight=None, check_input=True):
10751075
# workaround since _set_intercept will cast self.coef_ into X.dtype
10761076
self.coef_ = np.asarray(self.coef_, dtype=X.dtype)
10771077

1078+
# check for finiteness of coefficients
1079+
if not all(np.isfinite(w).all() for w in [self.coef_, self.intercept_]):
1080+
raise ValueError(
1081+
"Coordinate descent iterations resulted in non-finite parameter"
1082+
" values. The input data may contain large values and need to"
1083+
" be preprocessed."
1084+
)
1085+
10781086
# return self for chaining fit and predict calls
10791087
return self
10801088

‎sklearn/linear_model/_quantile.py

Lines changed: 15 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ class QuantileRegressor(LinearModel, RegressorMixin, BaseEstimator):
4747
programming formulation. Note that the highs methods are recommended
4848
for usage with `scipy>=1.6.0` because they are the fastest ones.
4949
Solvers "highs-ds", "highs-ipm" and "highs" support
50-
sparse input data.
50+
sparse input data and, in fact, always convert to sparse csc.
5151
5252
solver_options : dict, default=None
5353
Additional parameters passed to :func:`scipy.optimize.linprog` as
@@ -193,6 +193,12 @@ def fit(self, X, y, sample_weight=None):
193193
f"with scipy>=1.6.0, got {sp_version}"
194194
)
195195

196+
if sparse.issparse(X) and self.solver not in ["highs", "highs-ds", "highs-ipm"]:
197+
raise ValueError(
198+
f"Solver {self.solver} does not support sparse X. "
199+
"Use solver 'highs' for example."
200+
)
201+
196202
if self.solver_options is not None and not isinstance(
197203
self.solver_options, dict
198204
):
@@ -214,14 +220,14 @@ def fit(self, X, y, sample_weight=None):
214220
# min_x c x
215221
# A_eq x = b_eq
216222
# 0 <= x
217-
# x = (s0, s, t0, t, u, v) = slack variables
223+
# x = (s0, s, t0, t, u, v) = slack variables >= 0
218224
# intercept = s0 - t0
219225
# coef = s - t
220-
# c = (alpha * 1_p, alpha * 1_p, quantile * 1_n, (1-quantile) * 1_n)
226+
# c = (0, alpha * 1_p, 0, alpha * 1_p, quantile * 1_n, (1-quantile) * 1_n)
221227
# residual = y - X@coef - intercept = u - v
222228
# A_eq = (1_n, X, -1_n, -X, diag(1_n), -diag(1_n))
223229
# b_eq = y
224-
# p = n_features + fit_intercept
230+
# p = n_features
225231
# n = n_samples
226232
# 1_n = vector of length n with entries equal one
227233
# see https://stats.stackexchange.com/questions/384909/
@@ -246,14 +252,11 @@ def fit(self, X, y, sample_weight=None):
246252
c[0] = 0
247253
c[n_params] = 0
248254

249-
if sparse.issparse(X):
250-
if self.solver not in ["highs-ds", "highs-ipm", "highs"]:
251-
raise ValueError(
252-
f"Solver {self.solver} does not support sparse X. "
253-
"Use solver 'highs' for example."
254-
)
255-
# Note that highs methods do convert to csc.
256-
# Therefore, we work with csc matrices as much as possible.
255+
if self.solver in ["highs", "highs-ds", "highs-ipm"]:
256+
# Note that highs methods always use a sparse CSC memory layout internally,
257+
# even for optimization problems parametrized using dense numpy arrays.
258+
# Therefore, we work with CSC matrices as early as possible to limit
259+
# unnecessary repeated memory copies.
257260
eye = sparse.eye(n_indices, dtype=X.dtype, format="csc")
258261
if self.fit_intercept:
259262
ones = sparse.csc_matrix(np.ones(shape=(n_indices, 1), dtype=X.dtype))

0 commit comments

Comments
 (0)