Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ENH allow shrunk_covariance to handle multiple matrices at once #25275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Dec 13, 2023
Merged

ENH allow shrunk_covariance to handle multiple matrices at once #25275

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
a1e27d1
update shrunk_covariance
qbarthelemy Jan 2, 2023
3ff8c2e
complete tests
qbarthelemy Jan 2, 2023
09985c1
correct n_features
qbarthelemy Jan 12, 2023
9f2c51f
complete changelog
qbarthelemy Jan 12, 2023
44782e8
change quote
qbarthelemy Jan 13, 2023
cd75f5a
allow nd in check array
qbarthelemy Jan 13, 2023
09f8f8a
Apply suggestions from code review
qbarthelemy Jan 17, 2023
ab8aacb
correct log and correct expand_dims
qbarthelemy Jan 17, 2023
22beec4
split tests and parametrize
qbarthelemy Jan 17, 2023
cf1a82c
use assert_allclose
qbarthelemy Jan 19, 2023
de0ba82
update whats new
qbarthelemy Aug 29, 2023
61e1d7e
correct whats new
qbarthelemy Sep 22, 2023
c7a3dbd
Merge remote-tracking branch 'origin/main' into pr/qbarthelemy/25275
glemaitre Dec 7, 2023
4d88273
Merge branch 'main' into covariance_shrunk
jeremiedbb Dec 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions doc/whats_new/v1.4.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -320,6 +320,13 @@ Changelog
with `np.int64` indices are not supported.
:pr:`27240` by :user:`Yao Xiao <Charlie-XIAO>`.

:mod:`sklearn.covariance`
.........................

- |Enhancement| Allow :func:`covariance.shrunk_covariance` to process
multiple covariance matrices at once by handling nd-arrays.
:pr:`25275` by :user:`Quentin Barthélemy <qbarthelemy>`.

- |API| |FIX| :class:`~compose.ColumnTransformer` now replaces `"passthrough"`
with a corresponding :class:`~preprocessing.FunctionTransformer` in the
fitted ``transformers_`` attribute.
Expand Down
19 changes: 10 additions & 9 deletions sklearn/covariance/_shrunk_covariance.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,23 +109,23 @@ def _oas(X, *, assume_centered=False):
prefer_skip_nested_validation=True,
)
def shrunk_covariance(emp_cov, shrinkage=0.1):
"""Calculate a covariance matrix shrunk on the diagonal.
"""Calculate covariance matrices shrunk on the diagonal.

Read more in the :ref:`User Guide <shrunk_covariance>`.

Parameters
----------
emp_cov : array-like of shape (n_features, n_features)
Covariance matrix to be shrunk.
emp_cov : array-like of shape (..., n_features, n_features)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
emp_cov : array-like of shape (..., n_features, n_features)
emp_cov : array-like of shape (n_features, n_features) or \
(n_matrices, n_features, n_features)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestions about docstring seem to restrict usage to 2D and 3D arrays, whereas function is now able to process any nd array.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the use case of n-D shrinkage? Since the covariance matrix is always a 2-D matrix, I don't really see when you will pass a 4-D array, for instance.

Copy link
Contributor Author

@qbarthelemy qbarthelemy Jan 17, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of NumPy and SciPy functions use (..., n, n) to indicate that they can process ndarrays,
like numpy.linalg.eig and scipy.linalg.expm for example.

Use cases belong to users. But, for an example with a 4D array, shape (k, m, n, n), one might want to shrunk k sets, each set containing of m covariance matrices. I have tested, and code is ok.

For me, description should not restrain actual usage. But, as you wish.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking more of restraining on purpose the usage that we have in scikit-learn.
But we can go this road and see what other reviewers think.

Covariance matrices to be shrunk, at least 2D ndarray.

shrinkage : float, default=0.1
Coefficient in the convex combination used for the computation
of the shrunk estimate. Range is [0, 1].

Returns
-------
shrunk_cov : ndarray of shape (n_features, n_features)
Shrunk covariance.
shrunk_cov : ndarray of shape (..., n_features, n_features)
Shrunk covariance matrices.

Notes
-----
Expand All @@ -135,12 +135,13 @@ def shrunk_covariance(emp_cov, shrinkage=0.1):

where `mu = trace(cov) / n_features`.
"""
emp_cov = check_array(emp_cov)
n_features = emp_cov.shape[0]
emp_cov = check_array(emp_cov, allow_nd=True)
n_features = emp_cov.shape[-1]

mu = np.trace(emp_cov) / n_features
shrunk_cov = (1.0 - shrinkage) * emp_cov
shrunk_cov.flat[:: n_features + 1] += shrinkage * mu
mu = np.trace(emp_cov, axis1=-2, axis2=-1) / n_features
mu = np.expand_dims(mu, axis=tuple(range(mu.ndim, emp_cov.ndim)))
shrunk_cov += shrinkage * mu * np.eye(n_features)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, we materialize the eye matrix. If we restrain to a 3-D matrix then, we can efficiently do the in place addition with flattening as previously done.


return shrunk_cov

Expand Down
18 changes: 18 additions & 0 deletions sklearn/covariance/tests/test_covariance.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,25 @@ def test_covariance():
assert_array_equal(cov.location_, np.zeros(X.shape[1]))


@pytest.mark.parametrize("n_matrices", [1, 3])
def test_shrunk_covariance_func(n_matrices):
"""Check `shrunk_covariance` function."""

n_features = 2
cov = np.ones((n_features, n_features))
cov_target = np.array([[1, 0.5], [0.5, 1]])

if n_matrices > 1:
cov = np.repeat(cov[np.newaxis, ...], n_matrices, axis=0)
cov_target = np.repeat(cov_target[np.newaxis, ...], n_matrices, axis=0)

cov_shrunk = shrunk_covariance(cov, 0.5)
assert_allclose(cov_shrunk, cov_target)


def test_shrunk_covariance():
"""Check consistency between `ShrunkCovariance` and `shrunk_covariance`."""

# Tests ShrunkCovariance module on a simple dataset.
# compare shrunk covariance obtained from data and from MLE estimate
cov = ShrunkCovariance(shrinkage=0.5)
Expand Down