Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ENH Gaussian mixture bypassing unnecessary initialization computing - #26021

Merged
OmarManzoor merged 21 commits into
scikit-learn:mainfrom
jiawei-zhang-a:GaussianMixture
Aug 10, 2023
Merged

ENH Gaussian mixture bypassing unnecessary initialization computing#26021
OmarManzoor merged 21 commits into
scikit-learn:mainfrom
jiawei-zhang-a:GaussianMixture

Conversation

@jiawei-zhang-a

@jiawei-zhang-a jiawei-zhang-a commented Mar 29, 2023

Copy link
Copy Markdown
Contributor

Reference Issues/PRs

Fixes #26015

What does this implement/fix? Explain your changes.

I add a private variable _init_weights_means_precisions_skipped in _base.py.
if a user is passing some initial values for the weights, means, and precision then there is no need to run the initialization (via K-means or random) to estimate the gaussian parameters.

These two steps are now skipped if _init_weights_means_precisions_skipped is True

Any other comments?

@thomasjpfan thomasjpfan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR @jiawei-zhang-a !

Please add an entry to the change log at doc/whats_new/v1.3.rst with tag |Efficiency|. Like the other entries there, please reference this pull request with :pr: and credit yourself (and other contributors if applicable) with :user:.

Comment thread sklearn/mixture/_base.py Outdated
@jiawei-zhang-a

Copy link
Copy Markdown
Contributor Author

Thank you Mr. Fan @thomasjpfan . I have removed the new state and a new changelog :)

@thomasjpfan thomasjpfan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR @jiawei-zhang-a ! We still need a test to make sure that the parameters are not estimated during initialization. I think a simple way is to monkeypatching:

def test_gaussian_mixture_all_init_does_not_estimate_gaussian_parameters(monkeypatch):
    """When all init are provided, the Gaussian parameters are not estimated.

    Non-regression test for gh26015.
    """

    mock = Mock(side_effect=_estimate_gaussian_parameters)
    monkeypatch.setattr(
        sklearn.mixture._gaussian_mixture, "_estimate_gaussian_parameters", mock
    )

    rng = np.random.RandomState(0)
    rand_data = RandomData(rng)

    gm = GaussianMixture(
        n_components=rand_data.n_components,
        weights_init=rand_data.weights,
        means_init=rand_data.means,
        precisions_init=rand_data.precisions["full"],
        random_state=rng,
    )
    gm.fit(rand_data.X["full"])
    # The initial gaussian parameters are not estimated. They are estimated for every
    # m_step.
    assert mock.call_count == gm.n_iter_

Mock is from Python's untitest.mock module.

On main, the test would fail where mock.call_count is gm.n_iter_+1 from the extra call during initialization.

Comment thread doc/whats_new/v1.3.rst
:user:`Jérémie du Boisberranger <jeremiedbb>`,
:user:`Guillaume Lemaitre <glemaitre>`.


Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For git blame purpose, can you revert this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure!

Comment thread sklearn/mixture/_gaussian_mixture.py
@jiawei-zhang-a

Copy link
Copy Markdown
Contributor Author

Dear Mr.Fan, Thank you so much for all the advice! I will do it following your words!

@thomasjpfan thomasjpfan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor comments, otherwise LGTM

from sklearn.utils._testing import assert_array_equal
from sklearn.utils._testing import ignore_warnings

from unittest.mock import Mock

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Can you move this import to line 9 below import warning? This way the "first party Python modules" are at the top of the file.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure! I will do that immediately

Comment thread sklearn/mixture/tests/test_gaussian_mixture.py Outdated
@thomasjpfan thomasjpfan added the Waiting for Second Reviewer First reviewer is done, need a second one! label Apr 28, 2023
@glemaitre
glemaitre self-requested a review June 29, 2023 12:43
@haiatn

haiatn commented Jul 29, 2023

Copy link
Copy Markdown
Contributor

Good job! Waiting for this to merge

@OmarManzoor OmarManzoor left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @jiawei-zhang-a. Could you kindly resolve the conflicts by merging main and have a look at these few comments?

Comment thread doc/whats_new/v1.3.rst Outdated
Comment thread sklearn/mixture/_gaussian_mixture.py Outdated
@OmarManzoor OmarManzoor removed the Waiting for Second Reviewer First reviewer is done, need a second one! label Aug 9, 2023
@github-actions

github-actions Bot commented Aug 9, 2023

Copy link
Copy Markdown

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

Generated for commit: 9ce05ce. Link to the linter CI: here

@jiawei-zhang-a

Copy link
Copy Markdown
Contributor Author

@OmarManzoor Thank you so much for your review! I have committed your suggestions and fix the conflict with main branch

@OmarManzoor OmarManzoor left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates. I added a few more comments otherwise this looks good now!

Comment thread doc/whats_new/v1.3.rst Outdated
Comment thread sklearn/mixture/tests/test_gaussian_mixture.py Outdated
Comment thread sklearn/mixture/tests/test_gaussian_mixture.py Outdated
Comment thread sklearn/mixture/tests/test_gaussian_mixture.py Outdated
@jiawei-zhang-a

Copy link
Copy Markdown
Contributor Author

Thank you so much! I

Comment thread sklearn/mixture/tests/test_gaussian_mixture.py Outdated
@jiawei-zhang-a

Copy link
Copy Markdown
Contributor Author

Sure! I will check that now

@OmarManzoor
OmarManzoor merged commit 1a78993 into scikit-learn:main Aug 10, 2023
TamaraAtanasoska pushed a commit to TamaraAtanasoska/scikit-learn that referenced this pull request Aug 21, 2023
@glemaitre
glemaitre removed their request for review September 18, 2023 10:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GaussianMixture is runnning too many computation when parameters are alredy provided

4 participants