Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[MRG] Fix warm_start behavior in multilayer perceptron #12605

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
samwaterbury wants to merge 6 commits into from
Closed

[MRG] Fix warm_start behavior in multilayer perceptron #12605

samwaterbury wants to merge 6 commits into from

Conversation

@samwaterbury
Copy link
Contributor

@samwaterbury samwaterbury commented Nov 17, 2018 •
edited
Loading

Addresses #12505 (continued)

This PR addresses the unexpected behavior of warm_start in the multilayer perceptron. To recap, the current behavior is that warm_start breaks after a single iteration. I took the first steps of changing this by:

  • Reworking the MLP classes to support the change
  • Adding an option warm_start='full' to represent the future behavior
  • Raising a FutureWarning for warm_start=True that it will be changed in 0.22

There are two FIXMEs in the file that indicate the very simple changes required to make the transition in 0.22 (just remove the warning and remove an if statement condition). I tested making those 0.22 changes and it works as intended. Lastly, I added 2 tests: one for the FutureWarning and one for the new warm_start='full' option.

Please let me know also if I got the futurewarning/deprecation stuff right.

@jnothman tagging you again

Edit: The test that's failing is because the FutureWarning I added is getting raised.
Edit 2: fixed tests

@samwaterbury samwaterbury changed the title Warmstart [MRG] Fix warm_start behavior in multilayer perceptron Nov 17, 2018
jnothman
jnothman reviewed Nov 18, 2018
View reviewed changes
Copy link
Member

@jnothman jnothman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did my arithmetic wrong. As per the contributor docs, we need to support the old form for two releases, so should break in 0.23, not 0.22.

This is looking pretty good!

@samwaterbury
Copy link
Contributor Author

samwaterbury commented Nov 18, 2018

I fixed those items. BTW can I get confirmation that this is correct:

.. deprecated:: 0.21
   ``warm_start=True`` will adopt the new behavior of 'warm_start=full'
   in 0.23. To continue using the old behavior, use the function
   ``partial_fit`` instead or set 'max_iter=1'.

since it will be deprecated as of the 0.21 release?

@jnothman
Copy link
Member

jnothman commented Nov 19, 2018

Yes, I think that noticed is right.

jnothman
jnothman reviewed Nov 19, 2018
View reviewed changes
Copy link
Member

@jnothman jnothman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also add an entry to the change log at doc/whats_new/v0.21.rst. Like the other entries there, please reference this pull request with :issue: and credit yourself (and other contributors if applicable) with :user:

@samwaterbury
Copy link
Contributor Author

samwaterbury commented Nov 19, 2018

Alright those changes should be in now. I listed it as an API change in whats_new.

eamanu
eamanu approved these changes Nov 20, 2018
View reviewed changes
Copy link
Contributor

@eamanu eamanu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@samwaterbury samwaterbury mentioned this pull request Jan 20, 2019
Merged
@samwaterbury
Copy link
Contributor Author

samwaterbury commented Jan 25, 2019 •
edited
Loading

The merge conflicts should be fixed now, might need to @ somebody again to get a second reviewer.

Edit: Taking the liberty of pinging @rth since you reviewed my other PR related to MLP.

@samwaterbury
Copy link
Contributor Author

samwaterbury commented Feb 22, 2019

@jnothman Any idea on getting a second reviewer for this?

@rth
Copy link
Member

rth commented Jul 18, 2019

Thanks for this PR!

So to avoid warnings, the user would need to switch from warm_start=True to warm_start='full' now then in 0.23 (or 0.24) back to warm_start=True once the behavior for True changes. That's a bit cumbersome.

Can't we just consider it a bug fix and change the behavior of warm_start=True now? That would technically be backward incompatible and we should certainly document it in what's new, but I don't see how it could be bad for the user. At worst the model will converge further and take a bit longer to run.

Not asking to make these changes in this PR. Let's get a second opinion first.

@samwaterbury If you could resolve merge conflicts that would be useful. Thanks!

@samwaterbury
Copy link
Contributor Author

samwaterbury commented Jul 19, 2019

Ok! Merge conflicts should be fixed. That force push was just a rebase.

I can go either way, this is a breaking change but it's also definitely a bug fix. Just let me know.

@rth rth added the Needs Decision Requires decision label Jul 25, 2019
@adrinjalali adrinjalali deleted the branch scikit-learn:master January 22, 2021 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jnothman jnothman jnothman approved these changes

+1 more reviewer

@eamanu eamanu eamanu approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

module:neural_network Needs Decision Requires decision

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@samwaterbury @jnothman @rth @eamanu @adrinjalali