Codestin Search App

glemaitre · 2023-08-03T13:34:42Z

closes #26817

Accept "multilabel-indicator" in _get_response_values.

…onse

…to make_scorer_list_response

github-actions · 2023-08-03T13:36:33Z

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: 8aeb8b4. Link to the linter CI: here}

glemaitre · 2023-08-03T13:41:12Z

pinging @adrinjalali @thomasjpfan @betatim

I think this is ready for a review with a much shorter diff only for the initial bug.

thomasjpfan

Thanks for the PR!

Co-authored-by: Thomas J. Fan <[email protected]>

glemaitre · 2023-08-11T22:13:27Z

                if pos_label == classes[0]:
                    y_pred *= -1
+            elif target_type == "multilabel-indicator" and isinstance(y_pred, list):
+                y_pred = np.vstack([p for p in y_pred]).T


In this case, this is the same story. However, I don't recall any estimator that would return a list currently.

adrinjalali

I feel like there are things in your head which are not expressed here, and they would help me review this PR better 😁

adrinjalali · 2023-08-24T12:08:02Z

@@ -72,15 +73,15 @@ def _get_response_values(
    if is_classifier(estimator):


do you think this whole block could be simplified / explained? I have a hard time following all these different code paths, and trying to figure out if they actually cover all possible input types, and realized this is kind of a symptom of how this is written.

I am not sure where to put the cursor here. Mainly, the issue is that we need a compressed format (n_samples,) or (n_samples, n_outputs) (i.e. at least our metrics need to). So we leverage pos_label to provide the correct array.

I created new function and try to better explain what we intend to do.

LOVE the new implementation.

jeremiedbb

I like the refactoring into smaller dedicated functions and their description. It makes the code a lot easier to follow imo. I just have a few nitpicks, otherwise LGTM.

Co-authored-by: Jérémie du Boisberranger <[email protected]>

…kit-learn#26568) Co-authored-by: Guillaume Lemaitre <[email protected]> Co-authored-by: Jérémie du Boisberranger <[email protected]>

Co-authored-by: jeremie du boisberranger <[email protected]>

Co-authored-by: Tim Head <[email protected]>

…ikit-learn#27022)

…kit-learn#26521) Co-authored-by: Jérémie du Boisberranger <[email protected]> Co-authored-by: Olivier Grisel <[email protected]>

…s_neighbors_graph (scikit-learn#27245) Co-authored-by: Jérémie du Boisberranger <[email protected]>

…n#27297)

adrinjalali

this is a really nice refactor now. LGTM other than the small points.

adrinjalali · 2023-09-11T14:16:46Z

+    y_pred : ndarray of shape (n_samples,), (n_samples, n_classes) or \
+            (n_samples, n_output)
+        Compressed predictions format as requested by the metrics.


kinda wondering why this is not our output in the fist place 😁

Basically, I think this is dependent on the estimator and does it naturally or not output values.

adrinjalali · 2023-09-11T14:17:57Z

+    if target_type == "binary" and y_pred.shape[1] < 2:
+        # We don't handle classifiers trained on a single class.
+        raise ValueError(
+            f"Got predict_proba of shape {y_pred.shape}, but need "
+            "classifier with two classes."
+        )


I can imagine this happening in cases where we do cross validation or grid search, I feel like we shouldn't be raising.

I recall that this code was actually here in previous iteration before the refactoring. Looking a bit more, I don't think that all estimators support this one in scikit-learn (e.g. LogisticRegression does not for instance).

In terms of predict_proba, I could potentially make a PR that handle this case: we need to returns the probability in regards of pos_label to be consistent with estimator.classes_.

In short, we should have: y_pred.ravel() if pos_label == estimator.classes_[0] else np.abs(1 - y_pred.ravel()).

adrinjalali · 2023-09-11T14:44:41Z

+    In the binary case, it should invert the sign of the score if the positive label
+    is not `classes[1]`. In the multi-label case, it should stack the predictions if


this inversion of the sign is making me very uncomfortable, will need some time to see if it makes things explode in other places. Can you help me here?

Actually, the sign inversion and selection selection of the columns is indeed part of bug fixes that we got since a couple of release now :).

You you change your pos_label then you flip the hyperplane. In terms of binary probability, it comes to take the other columns and therefore the 1 - y_proba. Both, are consistent.

adrinjalali · 2023-09-11T14:51:48Z

+        # returns an array of shape `(n_samples, n_outputs)`.
+        # We could remove this code in the future?


that means this code is never run? then why do we have it?

Because we wrote some tests for it and thus silently supporting it.
We would not have tests, I would have been inclined removing it.
Here, we could potentially break code. It might not be worth it.

Then I would simply remove it. It's not a documented behavior anyway.

adrinjalali · 2023-09-11T14:56:01Z

@@ -72,15 +73,15 @@ def _get_response_values(
    if is_classifier(estimator):


LOVE the new implementation.

adrinjalali

much simplified

…#27002) Co-authored-by: Thomas J. Fan <[email protected]> Co-authored-by: Jérémie du Boisberranger <[email protected]> Co-authored-by: Stefanie Senger <[email protected]> Co-authored-by: Vladimir Fokow <[email protected]> Co-authored-by: jeremie du boisberranger <[email protected]> Co-authored-by: Xuefeng Xu <[email protected]> Co-authored-by: Tim Head <[email protected]> Co-authored-by: Raphael <[email protected]> Co-authored-by: Olivier Grisel <[email protected]> Co-authored-by: Sayed Qaiser Ali <[email protected]> Co-authored-by: Loïc Estève <[email protected]> Co-authored-by: Xiao Yuan <[email protected]>

Co-authored-by: Thomas J. Fan <[email protected]> Co-authored-by: Jérémie du Boisberranger <[email protected]> Co-authored-by: Stefanie Senger <[email protected]> Co-authored-by: Vladimir Fokow <[email protected]> Co-authored-by: jeremie du boisberranger <[email protected]> Co-authored-by: Xuefeng Xu <[email protected]> Co-authored-by: Tim Head <[email protected]> Co-authored-by: Raphael <[email protected]> Co-authored-by: Olivier Grisel <[email protected]> Co-authored-by: Sayed Qaiser Ali <[email protected]> Co-authored-by: Loïc Estève <[email protected]> Co-authored-by: Xiao Yuan <[email protected]>

…#27002) Co-authored-by: Thomas J. Fan <[email protected]> Co-authored-by: Jérémie du Boisberranger <[email protected]> Co-authored-by: Stefanie Senger <[email protected]> Co-authored-by: Vladimir Fokow <[email protected]> Co-authored-by: jeremie du boisberranger <[email protected]> Co-authored-by: Xuefeng Xu <[email protected]> Co-authored-by: Tim Head <[email protected]> Co-authored-by: Raphael <[email protected]> Co-authored-by: Olivier Grisel <[email protected]> Co-authored-by: Sayed Qaiser Ali <[email protected]> Co-authored-by: Loïc Estève <[email protected]> Co-authored-by: Xiao Yuan <[email protected]>

glemaitre added 10 commits July 15, 2023 13:35

ENH add new response_method in make_scorer

63c32bd

add non-regression test

1584c5b

update validation param

1a5a247

more coverage

4cc61b9

TST add mulitlabel test

8f36235

address tim comments

f3f844e

Merge remote-tracking branch 'origin/main' into make_scorer_list_resp…

44ad195

…onse

iter

d842cdc

Merge remote-tracking branch 'glemaitre/make_scorer_list_response' in…

e8ec34a

…to make_scorer_list_response

FIX accept multilabel-indicator in _get_response_values

fa93ca9

github-actions Bot added module:metrics module:utils labels Aug 3, 2023

glemaitre mentioned this pull request Aug 3, 2023

ENH/DEPR add new response_method and deprecate needs_* in make_scorer #26840

Merged

update changelog

b182ed5

glemaitre added 2 commits August 3, 2023 15:38

clean

029e6d2

Merge remote-tracking branch 'origin/main' into is/26817

5fd80a0

glemaitre added this to the 1.3.1 milestone Aug 3, 2023

thomasjpfan reviewed Aug 7, 2023

View reviewed changes

Comment thread sklearn/utils/_response.py Outdated

Comment thread sklearn/utils/_response.py Outdated

Apply suggestions from code review

24875f8

Co-authored-by: Thomas J. Fan <[email protected]>

glemaitre commented Aug 11, 2023

View reviewed changes

Comment thread sklearn/utils/_response.py Outdated

glemaitre commented Aug 11, 2023

View reviewed changes

Update sklearn/utils/_response.py

b99ecc0

glemaitre commented Aug 11, 2023

View reviewed changes

Comment thread sklearn/utils/_response.py Outdated

glemaitre added 2 commits August 12, 2023 00:14

Update sklearn/utils/_response.py

001a28a

Merge remote-tracking branch 'origin/main' into is/26817

d698420

adrinjalali reviewed Aug 24, 2023

View reviewed changes

glemaitre added 2 commits August 29, 2023 18:10

Merge remote-tracking branch 'origin/main' into is/26817

5bd1c23

add what is in my head

dba46ce

glemaitre added 3 commits August 29, 2023 18:35

more info

3753ff7

iter

a4aee17

refactor more

36a54cc

jeremiedbb approved these changes Sep 1, 2023

View reviewed changes

Comment thread sklearn/metrics/tests/test_score_objects.py Outdated

Comment thread sklearn/utils/_response.py Outdated

Comment thread sklearn/utils/_response.py Outdated

Comment thread doc/whats_new/v1.3.rst Outdated

glemaitre and others added 12 commits September 5, 2023 15:09

Apply suggestions from code review

ff37f21

Co-authored-by: Jérémie du Boisberranger <[email protected]>

MAINT Parameter validation for sklearn.neighbors.neighbors_graph (sci…

bd15809

…kit-learn#26568) Co-authored-by: Guillaume Lemaitre <[email protected]> Co-authored-by: Jérémie du Boisberranger <[email protected]>

EXA fix typo in HDBSCAN example (scikit-learn#27158)

b347305

Co-authored-by: jeremie du boisberranger <[email protected]>

DOC add reference DOI in PowerTransformer (scikit-learn#27247)

4cd1970

FIX Bisecting kmeans when data has 1 feature (scikit-learn#27243)

1e6f1c1

Co-authored-by: Tim Head <[email protected]>

DOC Add dropdown to Module 6.1 Pipelines and composite estimators (sc…

d346392

…ikit-learn#27022)

FIX make sure the decision function of weak learner is symmetric (sci…

40c4596

…kit-learn#26521) Co-authored-by: Jérémie du Boisberranger <[email protected]> Co-authored-by: Olivier Grisel <[email protected]>

MAINT Param validation for dbscan (scikit-learn#27234)

18d57d7

MAINT Added automatic validation function for sklearn.neighbors.radiu…

4140423

…s_neighbors_graph (scikit-learn#27245) Co-authored-by: Jérémie du Boisberranger <[email protected]>

MNT Avoid np.lib.arraysetops which is private in numpy 2 (scikit-lear…

6947a18

…n#27297)

DOC Fix a broken link in reference (scikit-learn#27299)

dc9ed71

address jeremie comments

7f74966

thomasjpfan reviewed Sep 5, 2023

View reviewed changes

Comment thread sklearn/cluster/tests/test_bisect_k_means.py

Comment thread sklearn/utils/_response.py Outdated

glemaitre added 2 commits September 6, 2023 11:01

Merge remote-tracking branch 'origin/main' into is/26817

94d848d

address comment from thomas

0db8245

adrinjalali reviewed Sep 11, 2023

View reviewed changes

glemaitre added 3 commits September 11, 2023 17:08

Merge remote-tracking branch 'origin/main' into is/26817

41c4f49

Merge remote-tracking branch 'origin/main' into is/26817

5941e6c

remove unknow case

8aeb8b4

adrinjalali approved these changes Sep 17, 2023

View reviewed changes

adrinjalali merged commit 3c9495c into scikit-learn:main Sep 17, 2023

		@@ -72,15 +73,15 @@ def _get_response_values(
		if is_classifier(estimator):

		In the binary case, it should invert the sign of the score if the positive label
		is not `classes[1]`. In the multi-label case, it should stack the predictions if

		# returns an array of shape `(n_samples, n_outputs)`.
		# We could remove this code in the future?

Uh oh!

Uh oh!

Conversation

glemaitre commented Aug 3, 2023

Uh oh!

github-actions Bot commented Aug 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✔️ Linting Passed

Uh oh!

glemaitre commented Aug 3, 2023

Uh oh!

thomasjpfan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

adrinjalali left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeremiedbb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adrinjalali left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adrinjalali left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

github-actions Bot commented Aug 3, 2023 •

edited

Loading