[MRG+1] add a `transform_max_iter` to `SparseCoder`, `DictionaryLearning`, and `MiniBatchDictionaryLearning` #12682

adrinjalali · 2018-11-27T08:30:51Z

Fixes #12650

SparseCoder now passes max_iter to the underlying LassoLarse when algorithm='lasso_lars'

jnothman

Can/should we test this?

adrinjalali · 2018-11-28T15:44:04Z

The test is basically a copy/paste of the example which was giving the warning.

adrinjalali · 2018-11-28T16:10:37Z

I'd also need to check the docs for this, but I'd appreciate some feedback before I do that.

jnothman · 2018-12-03T01:55:03Z

doc/whats_new/v0.21.rst

+:mod:`slkearn.decomposition`
+............................
+
+- |Fix| :class:`decomposition.SparseCoder` now passes `max_iter` to the


This is now out of date. Please describe here what this change does, or at least update the PR description, as now it seems unclear, with the addition of method_max_iter and transform_max_iter

adrinjalali · 2018-12-03T16:49:59Z

This PR now touches quite a few public functions/classes:

sparse_encode() now passes the max_iter to the underlying LassoLarse when algorithm='lasso_lars' (BUG)
dict_learning() and dict_learning_online() now accept method_max_iter and pass it to sparse_encode (ENH)
SparseCoder, DictionaryLearning, and MiniBatchDictionaryLearning now take a transform_max_iter parameter and pass it to either dict_learning() or sparse_encode() (ENH)

I've update the whats_new entries reflecting these changes, but I have two questions:

those are three separate entries in the log, I'm not sure how to merge them, or if I should. Should I also include the models in the Changed Models section?
I've added a test which checks for the warning I was encountering in the example (ConvergenceWarning), but there's no new explicit test for the 5 other public functions/classes which have a change in this PR. I'm not sure how to best test them though!

jnothman · 2018-12-06T21:57:35Z

I don't think there's a problem with separate what's new sections, and yes, if behaviour changes without user intervention, changed models is relevant

…ax_iter

jnothman · 2018-12-17T10:14:58Z

I've not yet understood this fully, but I am not very familiar with the algorithms. Why is method_max_iter different from transform_max_iter?

adrinjalali · 2018-12-17T10:35:37Z

I mostly followed whatever naming convention was there in each method/class. Some call the algorithm a method, and prefix the parameters related to it with method_, and some call it transform_algorithm, and use transform_ prefix for the algorithm's parameters.

…ax_iter

jnothman

As regards the other tests, perhaps checking that changing parameter value can affect the result is an okay level of testing.

doc/whats_new/v0.21.rst

jnothman · 2019-01-14T07:46:58Z

sklearn/decomposition/tests/test_dict_learning.py

+    def ricker_function(resolution, center, width):
+        """Discrete sub-sampled Ricker (Mexican hat) wavelet"""
+        x = np.linspace(0, resolution - 1, resolution)
+        x = ((2 / ((np.sqrt(3 * width) * np.pi ** 1 / 4)))


I think you must need parentheses around the 1/4... ** takes precedence. Perhaps use 0.25 instead of 1/4. You can remove some parentheses by changing * to /.

I didn't write these. They're copy pasted from the failing example (plot_sparse_coding.py).

But I'll try to simplify both then.

jnothman · 2019-01-14T07:47:09Z

sklearn/decomposition/tests/test_dict_learning.py

+        x = np.linspace(0, resolution - 1, resolution)
+        x = ((2 / ((np.sqrt(3 * width) * np.pi ** 1 / 4)))
+             * (1 - ((x - center) ** 2 / width ** 2))
+             * np.exp((-(x - center) ** 2) / (2 * width ** 2)))


You have extraneous brackets here making things hard to read. ** takes precedence over *

…ax_iter

adrinjalali · 2019-03-04T12:29:21Z

I'm not sure where the issue is coming from, but now the examples/decomposition/plot_sparse_coding.py doesn't converge with lasso_cd method, no matter what the max_iter. Any easy answers?

…ax_iter

adrinjalali · 2019-03-06T09:26:16Z

Tests are green again :)

sklearn/decomposition/dict_learning.py

adrinjalali · 2019-03-15T09:14:02Z

@jnothman happy with this now maybe?

adrinjalali · 2019-04-12T09:36:35Z

@jnothman should we put this in for 0.21?

thomasjpfan

The simplest test for MiniBatchDictionaryLearning and DictionaryLearning would be to mock out sparse_encode, but I think it is not needed.

thomasjpfan · 2019-04-12T16:02:43Z

sklearn/decomposition/tests/test_dict_learning.py

@@ -56,6 +56,54 @@ def test_dict_learning_overcomplete():
    assert dico.components_.shape == (n_components, n_features)


+def test_max_iter():


Testing for convergence this way is great!

thomasjpfan · 2019-04-12T16:06:42Z

sklearn/decomposition/dict_learning.py

@@ -832,7 +848,8 @@ def dict_learning_online(X, n_components=2, alpha=1, n_iter=100,
            print('|', end=' ')
        code = sparse_encode(X, dictionary.T, algorithm=method, alpha=alpha,
                             n_jobs=n_jobs, check_input=False,
-                             positive=positive_code)
+                             positive=positive_code, max_iter=method_max_iter,
+                             verbose=verbose)


We may have avoided this because sparse_encode accepts a int for verbose and verbose in this context is a bool. (Although this will still work)

We could pass verbose > 0, but I guess this way it's fine.

thomasjpfan · 2019-04-12T16:07:22Z

sklearn/decomposition/dict_learning.py

@@ -569,7 +576,8 @@ def dict_learning(X, n_components, alpha, max_iter=100, tol=1e-8,

        # Update code
        code = sparse_encode(X, dictionary, algorithm=method, alpha=alpha,
-                             init=code, n_jobs=n_jobs, positive=positive_code)
+                             init=code, n_jobs=n_jobs, positive=positive_code,
+                             max_iter=method_max_iter, verbose=verbose)


We may have avoided this because sparse_encode accepts a int for verbose and verbose in this context is a bool. (Although this will still work)

adrinjalali · 2019-04-13T13:12:38Z

@thomasjpfan I'm not sure if in your reviews you want me to change anything :P

thomasjpfan · 2019-04-23T20:44:17Z

sklearn/decomposition/dict_learning.py

@@ -493,6 +494,12 @@ def dict_learning(X, n_components, alpha, max_iter=100, tol=1e-8,

        .. versionadded:: 0.20

+    method_max_iter : int, optional (default=1000)
+        It is passed to the underlying ``method`` as their ``max_iter``


"Maximum number of iterations to perform it..."

You mean to add this at the beginning of the sentence? I would become Maximum number of iterations to perform it is passed to the underlying ``method`` as their ``max_iter`` parameter.. I can't parse the sentence, confused!

Since this is passed to sparse_encode I was thinking of using the docstring there here:

method_max_iter : int, 1000 by default Maximum number of iterations to perform if `algorithm='lasso_cd'`.

This way a user would not need to figure it out by parsing the code to get to sparse_code and figure out how max_iter is used.

I think this looks better now.

sklearn/decomposition/dict_learning.py

…ax_iter

thomasjpfan · 2019-04-26T15:09:21Z

sklearn/decomposition/dict_learning.py

@@ -713,8 +714,7 @@ def dict_learning_online(X, n_components=2, alpha=1, n_iter=100,
        .. versionadded:: 0.20

    method_max_iter : int, optional (default=1000)
-        It is passed to the underlying ``method`` as their ``max_iter``
-        parameter.
+        Maximum number of iterations to perform in each ``sparse_encode`` step.


To be consistent with dict_learning:

Maximum number of iterations to perform.

The reason for the extra part at the end is that dict_learning_online has an n_iter parameter as well, which has Maximum number of iterations to perform. as the description. I needed to distinguish the two somehow.

With two iters, we may need to describe what they actually do. Since dict_learning_online only supports lassos, maybe:

n_iter: Number of mini-batch iterations to perform. method_max_iter: Maximum number of iterations to perform when solving the lasso problem.

adrinjalali · 2019-05-02T16:02:39Z

Oops, I suppose I also need to change the versions in docstrings and move the whats_new

…ax_iter

adrinjalali · 2019-07-08T14:49:36Z

ping @thomasjpfan :)

thomasjpfan · 2019-07-08T21:10:45Z

Thank you @adrinjalali !

…, and `MiniBatchDictionaryLearning` (scikit-learn#12682)

SparseCoder passes max_iter to LassoLarse

efe7a27

eamanu approved these changes Nov 27, 2018

View reviewed changes

adrinjalali changed the title ~~SparseCoder passes max_iter to LassoLarse~~ [MRG+1] SparseCoder passes max_iter to LassoLarse Nov 28, 2018

jnothman reviewed Nov 28, 2018

View reviewed changes

adrinjalali changed the title ~~[MRG+1] SparseCoder passes max_iter to LassoLarse~~ [WIP] SparseCoder passes max_iter to LassoLarse Nov 28, 2018

adrinjalali added 3 commits November 28, 2018 15:03

expose methods' max_iter as transform_max_iter

75df47a

add a test

43bfe37

remove extra var

7dce627

adrinjalali changed the title ~~[WIP] SparseCoder passes max_iter to LassoLarse~~ [MRG] SparseCoder passes max_iter to LassoLarse Nov 28, 2018

pass tests

5f6882a

adrinjalali changed the title ~~[MRG] SparseCoder passes max_iter to LassoLarse~~ [MRG] ENH SparseCoder passes max_iter to LassoLarse Nov 29, 2018

adrinjalali changed the title ~~[MRG] ENH SparseCoder passes max_iter to LassoLarse~~ ENH SparseCoder passes max_iter to LassoLarse Nov 29, 2018

jnothman changed the title ~~ENH SparseCoder passes max_iter to LassoLarse~~ ENH SparseCoder passes max_iter to LassoLars Dec 3, 2018

jnothman reviewed Dec 3, 2018

View reviewed changes

add method_max_iter to dic_learning as well

d9ba9ef

adrinjalali changed the title ~~ENH SparseCoder passes max_iter to LassoLars~~ ENH add a transform_max_iter to SparseCoder, DictionaryLearning, and MiniBatchDictionaryLearning Dec 3, 2018

include all changes in the whats_new

1d2bd30

adrinjalali added 2 commits December 7, 2018 11:38

fix/improve the whats_new entries

b7cebac

Merge remote-tracking branch 'upstream/master' into bug/sparsecoder/m…

e92ab75

…ax_iter

Merge remote-tracking branch 'upstream/master' into bug/sparsecoder/m…

fd87128

…ax_iter

jnothman reviewed Jan 14, 2019

View reviewed changes

adrinjalali added 3 commits January 16, 2019 16:06

Merge remote-tracking branch 'upstream/master' into bug/sparsecoder/m…

35e156b

…ax_iter

fix paranthesis issue

fcff14a

less ()

e6db55b

Merge remote-tracking branch 'upstream/master' into bug/sparsecoder/m…

d107e19

…ax_iter

adrinjalali mentioned this pull request Mar 5, 2019

MNT fix cd_fast.pyx warning issues #13397

Merged

Merge remote-tracking branch 'upstream/master' into bug/sparsecoder/m…

5bdca25

…ax_iter

agramfort reviewed Mar 6, 2019

View reviewed changes

sklearn/decomposition/dict_learning.py Outdated Show resolved Hide resolved

add parameter at the end

0dd6a2c

agramfort approved these changes Mar 6, 2019

View reviewed changes

thomasjpfan reviewed Apr 12, 2019

View reviewed changes

merge upstream/master

442e2d5

thomasjpfan reviewed Apr 23, 2019

View reviewed changes

adrinjalali added 4 commits April 25, 2019 10:41

Merge remote-tracking branch 'upstream/master' into bug/sparsecoder/m…

49d9128

…ax_iter

update docstring

2e722b7

empty commit

4f28dbd

unifying the docstrings for max_iter

3829ef4

thomasjpfan reviewed Apr 26, 2019

View reviewed changes

apply Thomas's comment

e4a4f2d

adrinjalali added 2 commits May 2, 2019 18:03

Merge remote-tracking branch 'upstream/master' into bug/sparsecoder/m…

f1588a3

…ax_iter

move to 0.22

0c49d18

thomasjpfan approved these changes May 6, 2019

View reviewed changes

adrinjalali added 2 commits June 27, 2019 11:17

merge master

c229c06

remove extra line

2bb0ea8

thomasjpfan merged commit 7b8cbc8 into scikit-learn:master Jul 8, 2019

koenvandevelde pushed a commit to koenvandevelde/scikit-learn that referenced this pull request Jul 12, 2019

ENH add a transform_max_iter to SparseCoder, DictionaryLearning…

f9a69ae

…, and `MiniBatchDictionaryLearning` (scikit-learn#12682)

		@@ -56,6 +56,54 @@ def test_dict_learning_overcomplete():
		assert dico.components_.shape == (n_components, n_features)


		def test_max_iter():

Uh oh!

[MRG+1] add a transform_max_iter to SparseCoder, DictionaryLearning, and MiniBatchDictionaryLearning #12682

[MRG+1] add a transform_max_iter to SparseCoder, DictionaryLearning, and MiniBatchDictionaryLearning #12682

Uh oh!

Conversation

adrinjalali commented Nov 27, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jnothman left a comment

Choose a reason for hiding this comment

Uh oh!

adrinjalali commented Nov 28, 2018

Uh oh!

adrinjalali commented Nov 28, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adrinjalali commented Dec 3, 2018

Uh oh!

jnothman commented Dec 6, 2018 via email

Uh oh!

jnothman commented Dec 17, 2018

Uh oh!

adrinjalali commented Dec 17, 2018

Uh oh!

jnothman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adrinjalali commented Mar 4, 2019

Uh oh!

adrinjalali commented Mar 6, 2019

Uh oh!

Uh oh!

adrinjalali commented Mar 15, 2019

Uh oh!

adrinjalali commented Apr 12, 2019

Uh oh!

thomasjpfan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adrinjalali commented Apr 13, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adrinjalali commented May 2, 2019

Uh oh!

adrinjalali commented Jul 8, 2019

Uh oh!

thomasjpfan commented Jul 8, 2019

Uh oh!

[MRG+1] add a `transform_max_iter` to `SparseCoder`, `DictionaryLearning`, and `MiniBatchDictionaryLearning` #12682

[MRG+1] add a `transform_max_iter` to `SparseCoder`, `DictionaryLearning`, and `MiniBatchDictionaryLearning` #12682

adrinjalali commented Nov 27, 2018 •

edited

Loading