Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[MRG+1] Add classes_ parameter to hyperparameter CV classes #8295

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

stephen-hoover
Copy link
Contributor

@stephen-hoover stephen-hoover commented Feb 4, 2017

Reference Issue

Closes #8290 .

What does this implement/fix? Explain your changes.

In BaseSearchCV (superclass of GridSearchCV and RandomizedSearchCV), add a classes_ parameter which surfaces the classes_ parameter of the best_estimator_. Other parts of the scikit-learn code (e.g. sklearn.model_selection.cross_val_predict) as well as users expect this property to be present on fitted classifiers.

Any other comments?

When making this PR, I realized that there had already been a nearly identical PR, #7661 . That PR added the classes_ property in the deprecated sklearn.grid_search module. This PR adds it to the sklearn.model_selection module. The only difference is that I've added a

self._check_is_fitted("classes_")

call, with the intent of raising an error which is clearer than an AttributeError referencing best_estimator_.

The What's New file claims that this change has already happened, so there doesn't seem to be anything to change there.

In ``BaseSearchCV`` (superclass of ``GridSearchCV`` and ``RandomizedSearchCV``), add a ``clases_`` parameter which surfaces the ``classes_`` parameter of the ``best_estimator_``. Other parts of the scikit-learn code (e.g. ``cross_val_predict``) as well as users expect this property to be present on fitted classifiers.
@stephen-hoover
Copy link
Contributor Author

I don't know why Travis claims to have failed. It looks like all of the tests succeeded.

@jnothman
Copy link
Member

jnothman commented Feb 6, 2017

Travis is broken atm: travis-ci/travis-ci#7264

@jnothman
Copy link
Member

jnothman commented Feb 6, 2017

Whoops that we failed to port across #7661 to model_selection!! Could you please port across the test too?

@jnothman
Copy link
Member

jnothman commented Feb 6, 2017

Thanks!

@@ -914,7 +915,7 @@ def test_cross_val_predict_sparse_prediction():
assert_array_almost_equal(preds_sparse, preds)


def test_cross_val_predict_with_method():
def run_cross_val_predict_with_method(est):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we often call helpers in tests check_*

grid_search = GridSearchCV(clf, {'foo_param': [1, 2, 3]})

grid_search.fit(X, y)
assert_array_equal(grid_search.classes_, np.unique(y))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a test to check that hasattr(gscv, 'classes_') is false given a regressor?

@stephen-hoover
Copy link
Contributor Author

Thanks, @jnothman ! I copied the test from #7661 and added a couple of other checks on the classes_ attribute. I also renamed the test helper function in test_validation.py to check_cross_val_predict_with_method.

Copy link
Member

@jnothman jnothman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks. And thanks for catching our omission.

@codecov
Copy link

codecov bot commented Feb 9, 2017

Codecov Report

Merging #8295 into master will increase coverage by <.01%.

@@            Coverage Diff             @@
##           master    #8295      +/-   ##
==========================================
+ Coverage   94.74%   94.74%   +<.01%     
==========================================
  Files         342      342              
  Lines       60711    60735      +24     
==========================================
+ Hits        57519    57543      +24     
  Misses       3192     3192
Impacted Files Coverage Δ
sklearn/model_selection/_search.py 97.88% <100%> (+0.02%)
sklearn/model_selection/tests/test_validation.py 98.23% <100%> (+0.01%)
sklearn/model_selection/tests/test_search.py 99.3% <100%> (+0.01%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3a0ea19...6b136a2. Read the comment docs.

@stephen-hoover
Copy link
Contributor Author

@jnothman , I merged master, since GitHub reported a conflict in _search.py. It wasn't really a conflict; you can see that the diff in that file is still the same. AppVeyor ran this time. What's the next step?

@jnothman jnothman changed the title [MRG] Add classes_ parameter to hyperparameter CV classes [MRG+1] Add classes_ parameter to hyperparameter CV classes Feb 9, 2017
@jnothman
Copy link
Member

jnothman commented Feb 9, 2017

Usually I'd wait for another reviewer, just to be sure I didn't miss anything.

Could you add a note in what's new to say this was a bug fix because we failed to implement the change properly for 0.18?

@stephen-hoover
Copy link
Contributor Author

Okay. The change in #7661 was for v0.19; it hasn't been in a release yet. The text in the existing What's New entry describes what this PR does, so I added the PR number rather than making a new entry.

@@ -68,7 +68,8 @@ Enhancements

- Added ``classes_`` attribute to :class:`model_selection.GridSearchCV`
that matches the ``classes_`` attribute of ``best_estimator_``. :issue:`7661`
by :user:`Alyssa Batula <abatula>` and :user:`Dylan Werner-Meier <unautre>`.
and :issue:`8295` by :user:`Alyssa Batula <abatula>`,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add the class grid_search.GridSearchCV, grid_search.RandomizedSearchCV and model_selection.RandomizedSearchCV to make this entry more exact.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@lesteve
Copy link
Member

lesteve commented Feb 10, 2017

LGTM, thanks a lot, merging!

@lesteve lesteve merged commit eee8be4 into scikit-learn:master Feb 10, 2017
@stephen-hoover
Copy link
Contributor Author

Thank you!

sergeyf pushed a commit to sergeyf/scikit-learn that referenced this pull request Feb 28, 2017
@Przemo10 Przemo10 mentioned this pull request Mar 17, 2017
Sundrique pushed a commit to Sundrique/scikit-learn that referenced this pull request Jun 14, 2017
NelleV pushed a commit to NelleV/scikit-learn that referenced this pull request Aug 11, 2017
paulha pushed a commit to paulha/scikit-learn that referenced this pull request Aug 19, 2017
maskani-moh pushed a commit to maskani-moh/scikit-learn that referenced this pull request Nov 15, 2017
lemonlaug pushed a commit to lemonlaug/scikit-learn that referenced this pull request Jan 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cross_val_predict with predict_proba errors when the estimator is GridSearchCV
3 participants