Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[MRG] Allow sample weights and other fit_params for RFE #7333

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 5 commits into from
Closed

[MRG] Allow sample weights and other fit_params for RFE #7333

wants to merge 5 commits into from

Conversation

g-rutter
Copy link

@g-rutter g-rutter commented Sep 2, 2016 •
edited
Loading

Reference Issue

Fixes #7308

What does this implement/fix? Explain your changes.

Adds support for passing sample weights into RFE.fit() and having them used by the estimator's fit method.

Adds a test for this with the iris dataset, where sample weights are used to give one class double its normal weight and compare that to doubling the samples in that class. The test passes if both these approaches produce the same feature ranking. This feature ranking is different from the one which arises when all samples have the same weight.

Any other comments?

I will look in to adding the same support for RFECV, and to RFE.score().

amueller
amueller reviewed Sep 12, 2016
View reviewed changes
sklearn/feature_selection/tests/test_rfe.py Outdated
rfe.fit(X, y, sample_weight=w)
ranking_1 = rfe.ranking_.copy()

# Case 2 - duplicate the features of one class
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duplicate the samples?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, I'll change this.

@amueller amueller changed the title [WIP] Allow sample weights and other fit_params for RFE [MRG] Allow sample weights and other fit_params for RFE Sep 12, 2016
@amueller
Copy link
Member

Looks good apart from comment about the test.

@g-rutter
Copy link
Author

I do consider this a work in progress though, due to the remaining additions mentioned at the end of the first post.

@amueller
Copy link
Member

score doesn't take sample_weights (for now)

Improve sample weights testing
5870b7a 
- Test that the weighted feature ranking is different from the original
  feature ranking
- Clearer comments and variable names
@g-rutter
Copy link
Author

I have updated the tests with respect to @amueller's comments, but won't work on RFECV until the interaction of this pull request with #1574 is clarified. In fact, these two pull requests seem to have overlapping purposes. However, my pull request offers full fit_params, whereas #1574 offers only sample weights, for RFE.

@miguellacerda
Copy link

I think you also need to add the **fit_params argument to the fit method called just after line 200 (near the end of _fit):

# Set final attributes
features = np.arange(n_features)[support_]
self.estimator_ = clone(self.estimator)
self.estimator_.fit(X[:, features], y, **fit_params)

@lschneidpro
Copy link

any updates ?

@amueller
Copy link
Member

This needs a merge with master

@thomasjpfan thomasjpfan mentioned this pull request Jul 21, 2020
Closed
@hakeemo
Copy link

hakeemo commented Sep 11, 2020

any updates ?

Base automatically changed from master to main January 22, 2021 10:49
@proinsias
Copy link
Contributor

@g-rutter – any updates on this?

@cerlymarco
Copy link

Everyone looking for RFE with Gradient Boosting, like LGBM or XGB, I suggest shap-hypetune... A python package for simultaneous Hyperparameters Tuning and Features Selection for Gradient Boosting Models.

It supports RFE (also with shap feature ranking) with every fitting parameters like in the standard algorithm API

@fbidu
Copy link
Contributor

fbidu commented Jun 26, 2021

Me and @ijpulidos were taking over this PR to finish it up against the current main branch

@fbidu fbidu mentioned this pull request Jun 26, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
help wanted module:feature_selection Stalled
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

RFE/RFECV doesn't work with sample weights
9 participants
@g-rutter @amueller @miguellacerda @lschneidpro @hakeemo @proinsias @cerlymarco @fbidu @cmarmo