Thanks to visit codestin.com
Credit goes to github.com

Skip to content

REVERT ENH add the parameter prefit in the FixedThresholdClassifier (#29067) #30172

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Nov 5, 2024

Conversation

adrinjalali
Copy link
Member

@adrinjalali adrinjalali commented Oct 29, 2024

Towards #29893

This reverts #29067, and instead adds a note that users can use FrozenEstimator.

This doesn't need a deprecation since we haven't released it yet.

cc @glemaitre @ogrisel

Copy link

github-actions bot commented Oct 29, 2024

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

Generated for commit: f622b4e. Link to the linter CI: here

@adrinjalali adrinjalali added this to the 1.6 milestone Oct 29, 2024
@glemaitre
Copy link
Member

Uhm can you make a push with [doc build]. I'm almost sure we have a documentation example with it.

Copy link
Member

@glemaitre glemaitre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything is green so LGTM apart from the last comment.

Copy link
Member

@thomasjpfan thomasjpfan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think it's obvious for someone to infer all the features that FrozenEstimator enables. (Including using it as a pre-fitted estimator in FixedThresholdClassifier)

Can we have a central place in the docs that describe some of the features FrozenEstimator enables?

@adrinjalali
Copy link
Member Author

Added an example here. If we merge this one first, I can add to the same example for CalibratedClassifierCV in the other PR.

from sklearn.model_selection import FixedThresholdClassifier

model_fixed_threshold = FixedThresholdClassifier(
estimator=model, threshold=tuned_model.best_threshold_, prefit=True
estimator=FrozenEstimator(model), threshold=tuned_model.best_threshold_
).fit(data_train, target_train)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit weird that .fit(data_train, target_train) is still needed in this case (just to set the internal estimator_ attribute).

Maybe FixedThresholdClassifier.predict could be changed to avoid calling check_is_fitted(self, "estimator_") and use:

estimator = getattr(self, "estimator_", self.estimator)
...

Then the call to _get_response_values_binary would naturally call the check_is_fitted of the wrapped estimator to make sure that we raise the excepted exception if the underlying estimator has not been fitted beforehand.

This would allow using it with a FrozenEstimator without the useless call to .fit.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's odd. It's expected. Fit doesn't just do fit in so many cases. So I would have kept it as is.

But my last commit now does this (it's slightly bigger than the change you suggest).

Note that this means technically, the user doesn't have to wrap the sub-estimator in a FrozenEstimator.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But my last commit now does this (it's slightly bigger than the change you suggest).

I don't see the commit you are referring to. Did you push it to an alternative branch?

Copy link
Member

@ogrisel ogrisel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let merge and iterate in further PRs if needed.

#
# Please refer to
# :ref:`sphx_glr_auto_examples_model_selection_plot_cost_sensitive_learning.py`
# to learn about cost-sensitive learning and decision threshold tuning.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ogrisel
Copy link
Member

ogrisel commented Oct 30, 2024

Actually, I cannot click the merge button:

image

@glemaitre
Copy link
Member

Me too. It looks like a GitHub bug. The branch is "frozen".

@adrinjalali
Copy link
Member Author

It was a github issue. Now it's working. I also added to the example the CalibratedClassifierCV since @thomasjpfan asked for it and the other PR is merged.

@glemaitre glemaitre merged commit 70aab36 into scikit-learn:main Nov 5, 2024
30 checks passed
@glemaitre
Copy link
Member

Now we can merge it apparently since @adrinjalali pushed some fixes since.

@adrinjalali adrinjalali deleted the fixedthreshold/cv branch November 5, 2024 19:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants