Deprecate use of probability=True in SVC and NuSVC
#32050
Conversation
There was a problem hiding this comment.
Thanks @snath-xoc for the PR. Here is a pass of feedback. Once addressed, could you please try to fix any issue reported by the CI is green before pinging reviewers again?
Feel free to ask for help if there are particular CI failures you don't know how to address.
sklearn/svm/tests/test_svm.py
Outdated
| # XXX: this test is thread-unsafe because it uses probability=True: | ||
| # https://github.com/scikit-learn/scikit-learn/issues/31885 | ||
| @pytest.mark.thread_unsafe | ||
| @pytest.mark.filterwarnings("ignore::FutureWarning") |
There was a problem hiding this comment.
I think we could change this test to remove probability=True along with all the test function annotations: it seems that the convergence problem tested in this test is unrelated to the choice of probability=True.
There was a problem hiding this comment.
Sounds good, shall I do the same for other tests as well e.g., test_svc_clone_with_callable_kernel in L1061?
Never mind, it is using predict_proba.
sklearn/svm/_classes.py
Outdated
| @@ -877,6 +882,16 @@ def __init__( | |||
| break_ties=False, | |||
| random_state=None, | |||
| ): | |||
| if probability != "deprecated": | |||
There was a problem hiding this comment.
This should be moved to the fit method. As per the scikit-learn estimator API, the constructor parameters should always be stored unchanged as attributes. Any validation logic should be deferred to the fit method.
Co-authored-by: Olivier Grisel <[email protected]>
| @@ -304,7 +304,7 @@ def fit(self, X, y): | |||
| { | |||
| "estimators": [ | |||
| ("lr", LogisticRegression()), | |||
| ("svm", SVC(max_iter=50_000)), | |||
| ("svm", SVC(probability=True, max_iter=50_000)), | |||
There was a problem hiding this comment.
Why this change? The purpose of the original test was to check that ValueError is raised with a base estimator that does not expose the predict_proba method.
So let's not add a deprecated option to a test that does not require it.
| FutureWarning, | ||
| ) | ||
| else: | ||
| self.probability = False |
There was a problem hiding this comment.
As per scikit-learn estimator API, fit should not change the value of a public attribute set in the constructor. Instead, define a local variable named probability and set it to self.probability when self.probability != "deprecated" and False otherwise.
There was a problem hiding this comment.
Then update the rest of the fit method to use the probability local variable instead of self.probability.
| @@ -143,15 +144,19 @@ def fit(self, X, y): | |||
| X_test_sparse = sparse_container(X_test) | |||
| # Trained on sparse format | |||
| sparse_classifier = BaggingClassifier( | |||
| estimator=CustomSVC(kernel="linear", decision_function_shape="ovr"), | |||
| estimator=CustomSVC( | |||
| probability=True, kernel="linear", decision_function_shape="ovr" | |||
There was a problem hiding this comment.
Why enable the deprecated attribute in a test that would not need predict_proba in the first place?
| random_state=1, | ||
| **params, | ||
| ).fit(X_train_sparse, y_train) | ||
| sparse_results = getattr(sparse_classifier, method)(X_test_sparse) | ||
|
|
||
| # Trained on dense format | ||
| dense_classifier = BaggingClassifier( | ||
| estimator=CustomSVC(kernel="linear", decision_function_shape="ovr"), | ||
| estimator=CustomSVC( | ||
| probability=True, kernel="linear", decision_function_shape="ovr" |
There was a problem hiding this comment.
Same comment here. This test does should not use the deprecated attribute. If the tests need a base estimator with predict_proba we should instead wrap it with CalibratedClassifierCV.
Reference Issues/PRs
Related to issue #31885
What does this implement/fix? Explain your changes.
this PR deprecates the use of probability=True in both SVC and NuSVC
Any other comments?
For now the default is set as
probability="deprecated"and later on it is set toprobability=False. This is as within the baselibsvm.fitfunction pyx code, probability is required to be a boolean integer. Perhaps in future we hard-code this as always being 0 (i.e., False), as we should not useprobability=True?FYI @ogrisel