ENH Add `drop_intermediate` parameter to `metrics.precision_recall_curve` #24668

dberenbaum · 2022-10-15T02:11:51Z

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Adds a drop_intermediate kwarg to metrics.precision_recall_curve similar to the one that already exists for metrics.roc_curve. This removes unnecessary points on the curve to reduce its size.

betatim · 2022-10-17T07:58:27Z

sklearn/metrics/_ranking.py

+        # with the same tps value have the same recall and thus x coordinate.
+        # They appear as a vertical line on the plot.
+        optimal_idxs = np.where(
+            np.r_[True, np.logical_or(np.diff(tps[:-1]), np.diff(tps[1:])), True]


For my education: why does taking the "second derivative" in roc_curve work, but here it doesn't?

More precisely, why does using the second derivative work for roc_curve? What we are looking for is two (or more) points where there is no change, so the first derivative seems like the natural thing to use :-/

Thinking about this some more, could we use np.r_[True, np.diff(tps, 2), True] instead?

For tps = [1, 2, 3, 3, 3, 5, 6] we'd get [1, 3, 3, 5, 6] (the two gets dropped because its is on the line between 1 and 3. For tps = [1,2.1,3,3,3,5,6] we get [1., 2.1, 3., 3., 5., 6.].

I guess for plotting purposes it is fine to remove the 2?! Is there a reason to have different behaviour regarding the removal of points in roc_curve and this (with np.logical_or(np.diff(tps[:-1]), np.diff(tps[1:])) the 2 is kept)?

The difference is that both axes of an ROC curve have constant denominators:

fpr = fps / fps[-1] (linearly correlated with fps)

tpr = tps / tps[-1] (linearly correlated with tps)

By contrast, precision has a non-constant denominator (note that recall = tpr):

precision = tps / (tps + fps) (not linearly correlated with either tps or fps)

If you extend your example by one to tps = [1, 2, 3, 3, 3, 5, 6, 7], then you will get:

tps = [1, 3, 3, 3, 5, 6, 7] fps = [0, 0, 1, 2, 2, 2, 2] tpr = [1/7, 3/7, 3/7, 3/7, 5/7, 6/7, 7/7] fpr = [0/2, 0/2, 1/2, 2/2, 2/2, 2/2, 2/2] precision = [1/1, 3/3, 3/4, 3/5, 5/7, 6/8, 7/9]

np.r_[True, np.logical_or(np.diff(tps[:-1]), np.diff(tps[1:])), True] results in [1, 3, 3, 5, 6, 7].

np.r_[True, np.diff(tps, 2), True] results in [1, 3, 3, 5, 7].

The second method incorrectly drops the 6, which is not actually on a line in the precision-recall curve:

Today I learnt! Thanks for taking the time to explain it

betatim

Looks good to me.

Does this need an entry in "what's new"?

dberenbaum · 2022-10-17T20:31:07Z

Looks good to me.

Does this need an entry in "what's new"?

Not sure if this question is to me? I'm not sure what justifies a "what's new" entry but happy to provide one if needed.

betatim · 2022-10-18T16:25:07Z

Not sure if this question is to me? I'm not sure what justifies a "what's new" entry but happy to provide one if needed.

It was aimed at someone "in the know", because I also don't know the inclusion criteria.

glemaitre

We will need an entry in the changelog doc/whats_new/v1.2.rst, in the appropriate section. We can consider this as an enhancement. We should mentioned that the parameter is added to both the precision_recall_curve and the associated display.

sklearn/metrics/_plot/precision_recall_curve.py

sklearn/metrics/_ranking.py

sklearn/metrics/_plot/precision_recall_curve.py

sklearn/metrics/tests/test_ranking.py

Co-authored-by: Guillaume Lemaitre <[email protected]>

…parameter to `metrics.precision_recall_curve`

dberenbaum · 2022-11-04T00:32:20Z

Thanks @glemaitre! The comments should be addressed.

glemaitre

I pushed 2 fixes for the CIs to pass.
However, I still think that we need to force drop_intermediate=False for the moment before making a deprecation cycle.

sklearn/metrics/_ranking.py

sklearn/metrics/_plot/precision_recall_curve.py

Co-authored-by: Guillaume Lemaitre <[email protected]>

dberenbaum · 2022-11-04T21:04:40Z

However, I still think that we need to force drop_intermediate=False for the moment before making a deprecation cycle.

Sorry, just an oversight that I missed fixing in the rest of the code. 🙏

Co-authored-by: Guillaume Lemaitre <[email protected]>

glemaitre

LGTM is on my side. Thanks @dberenbaum.

betatim

LGTM. Maybe we can get this merged in time for v1.2

dberenbaum · 2023-01-06T20:02:36Z

Hi @betatim, just checking if there's anything I can do to help get this merged?

glemaitre · 2023-01-09T10:38:41Z

Adding the Waiting for second reviewer flag.

betatim · 2023-01-11T12:51:05Z

Hi @betatim, just checking if there's anything I can do to help get this merged?

In general no, we need a second reviewer to come along and review this. Having the label should help with that.

However, since v1.2 has been released already the changelog entry needs to move from the v1.2 file to the v1.3 file. Sorry for that busy work.

jeremiedbb

Thanks for the PR @dberenbaum. I see that there are already 2 approvals, I just updated the target version to 1.3. Let's merge.

…scikit-learn#24668) Co-authored-by: Guillaume Lemaitre <[email protected]>

* MAINT Clean deprecated losses in (hist) gradient boosting for 1.3 (scikit-learn#25834) * MAINT Clean deprecation of normalize in calibration_curve for 1.3 (scikit-learn#25833) * BLD Clean command removes generated from cython templates (scikit-learn#25839) * PERF Implement `PairwiseDistancesReduction` backend for `KNeighbors.predict_proba` (scikit-learn#24076) Signed-off-by: Julien Jerphanion <[email protected]> Co-authored-by: Julien Jerphanion <[email protected]> Co-authored-by: Olivier Grisel <[email protected]> * MAINT Added Parameter Validation for datasets.make_circles (scikit-learn#25848) Co-authored-by: jeremiedbb <[email protected]> * MNT use a single job by default with sphinx build (scikit-learn#25836) * BLD Generate warning automatically for templated cython files (scikit-learn#25842) * MAINT parameter validation for sklearn.datasets.fetch_lfw_people (scikit-learn#25820) Co-authored-by: jeremiedbb <[email protected]> * MAINT Parameters validation for metrics.fbeta_score (scikit-learn#25841) * TST add global_random_seed fixture to sklearn/covariance/tests/test_robust_covariance.py (scikit-learn#25821) * MAINT Parameter validation for linear_model.orthogonal_mp (scikit-learn#25817) * TST activate common tests for TSNE (scikit-learn#25374) * CI Update lock files (scikit-learn#25849) * MAINT Added Parameter Validation for metrics.mean_gamma_deviance (scikit-learn#25853) * MAINT Parameters validation for feature_selection.mutual_info_regression (scikit-learn#25850) * MAINT parameter validation metrics.class_likelihood_ratios (scikit-learn#25863) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Ensure disjoint interval constraints (scikit-learn#25797) * MAINT Parameters validation for utils.gen_batches (scikit-learn#25864) * TST use global_random_seed in test_dict_vectorizer.py (scikit-learn#24533) * TST use global_random_seed in test_pls.py (scikit-learn#24526) Co-authored-by: jeremiedbb <[email protected]> * TST use global_random_seed in test_gpc.py (scikit-learn#24600) Co-authored-by: jeremiedbb <[email protected]> * DOC Fix overlapping plot axis in bench_sample_without_replacement.py (scikit-learn#25870) * MAINT Use contiguous memoryviews in _random.pyx (scikit-learn#25871) * MAINT parameter validation sklearn.datasets.fetch_lfw_pair (scikit-learn#25857) * MAINT Parameters validation for metrics.classification_report (scikit-learn#25868) * Empty commit * DOC fix docstring dtype parameter in OrdinalEncoder (scikit-learn#25877) * MAINT Clean up depreacted "log" loss of SGDClassifier for 1.3 (scikit-learn#25865) * ENH Adds TargetEncoder (scikit-learn#25334) Co-authored-by: Andreas Mueller <[email protected]> Co-authored-by: Olivier Grisel <[email protected]> Co-authored-by: Jovan Stojanovic <[email protected]> Co-authored-by: Guillaume Lemaitre <[email protected]> * CI make it possible to cancel running Azure jobs (scikit-learn#25876) * MAINT Clean-up deprecated if_delegate_has_method for 1.3 (scikit-learn#25879) * MAINT Parameter validation for tree.export_text (scikit-learn#25867) * DOC impact of `tol` for solvers in RidgeClassifier (scikit-learn#25530) * MAINT Parameters validation for metrics.hinge_loss (scikit-learn#25880) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for metrics.ndcg_score (scikit-learn#25885) * ENH KMeans initialization account for sample weights (scikit-learn#25752) Co-authored-by: jeremiedbb <[email protected]> Co-authored-by: Guillaume Lemaitre <[email protected]> Co-authored-by: Jérémie du Boisberranger <[email protected]> * TST use global_random_seed in sklearn/tests/test_dummy.py (scikit-learn#25884) * DOC improve calibration user guide (scikit-learn#25687) * ENH Support for sparse matrices added to `sklearn.metrics.silhouette_samples` (scikit-learn#24677) Co-authored-by: Sahil Gupta <[email protected]> Co-authored-by: Thomas J. Fan <[email protected]> Co-authored-by: Guillaume Lemaitre <[email protected]> * MAINT validate_params for plot_tree (scikit-learn#25882) Co-authored-by: Itay <[email protected]> * MAINT add missing space in error message in SVM (scikit-learn#25913) * FIX Adds requires_y tag to TargetEncoder (scikit-learn#25917) * MAINT Consistent cython types continued (scikit-learn#25810) * TST Speed-up common tests of DictionaryLearning (scikit-learn#25892) * TST Speed-up test_dbscan_optics_parity (scikit-learn#25893) * ENH add np.nan option for zero_division in precision/recall/f-score (scikit-learn#25531) Co-authored-by: Guillaume Lemaitre <[email protected]> * MAINT Parameters validation for datasets.make_low_rank_matrix (scikit-learn#25901) * MAINT Parameter validation for metrics.cluster.adjusted_mutual_info_score (scikit-learn#25898) Co-authored-by: Jérémie du Boisberranger <[email protected]> * TST Speed-up test_partial_dependence.test_output_shape (scikit-learn#25895) Co-authored-by: Thomas J. Fan <[email protected]> * MAINT Parameters validation for datasets.make_regression (scikit-learn#25899) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for metrics.mean_squared_log_error (scikit-learn#25924) * TST Use global_random_seed in tests/test_naive_bayes.py (scikit-learn#25890) * TST add global_random_seed fixture to sklearn/datasets/tests/test_covtype.py (scikit-learn#25904) Co-authored-by: Jérémie du Boisberranger <[email protected]> Co-authored-by: jeremiedbb <[email protected]> * MAINT Parameters validation for datasets.make_multilabel_classification (scikit-learn#25920) * Fixed feature mapping typo (scikit-learn#25934) * MAINT switch to newer codecov uploader (scikit-learn#25919) Co-authored-by: Loïc Estève <[email protected]> * TST Speed-up test suite when using pytest-xdist (scikit-learn#25918) * DOC update license year to 2023 (scikit-learn#25936) * FIX Remove spurious feature names warning in IsolationForest (scikit-learn#25931) * TST fix unstable test_newrand_set_seed (scikit-learn#25940) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Clean-up deprecated max_features="auto" in trees/forests/gb (scikit-learn#25941) * MAINT LogisticRegression informative error msg when penaly=elasticnet and l1_ratio is None (scikit-learn#25925) Co-authored-by: jeremiedbb <[email protected]> * MAINT Clean-up remaining SGDClassifier(loss="log") (scikit-learn#25938) * FIX Fixes pandas extension arrays in check_array (scikit-learn#25813) * FIX Fixes pandas extension arrays with objects in check_array (scikit-learn#25814) * CI Disable pytest-xdist in pylatest_pip_openblas_pandas build (scikit-learn#25943) * MAINT remove deprecated call to resources.content (scikit-learn#25951) * DOC note on calibration impact on ranking (scikit-learn#25900) * Remove loguniform fix, use scipy.stats instead (scikit-learn#24665) Co-authored-by: Olivier Grisel <[email protected]> * MAINT Fix broken links in cluster.dbscan module (scikit-learn#25958) * DOC Fix lars Xy shape (scikit-learn#25952) * ENH Add drop_intermediate parameter to metrics.precision_recall_curve (scikit-learn#24668) Co-authored-by: Guillaume Lemaitre <[email protected]> * FIX improve error message when computing NDCG with a single document (scikit-learn#25672) Co-authored-by: Guillaume Lemaitre <[email protected]> * MAINT introduce _get_response_values and _check_response_methods (scikit-learn#23073) Co-authored-by: Thomas J. Fan <[email protected]> Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Extend message for large sparse matrices support (scikit-learn#25961) Co-authored-by: Meekail Zain <[email protected]> * MAINT Parameters validation for datasets.make_gaussian_quantiles (scikit-learn#25959) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.metrics.d2_tweedie_score (scikit-learn#25975) * MAINT Parameters validation for datasets.make_hastie_10_2 (scikit-learn#25967) * MAINT Parameters validation for preprocessing.minmax_scale (scikit-learn#25962) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for datasets.make_checkerboard (scikit-learn#25955) * MAINT Parameters validation for datasets.make_biclusters (scikit-learn#25945) * MAINT Parameters validation for datasets.make_moons (scikit-learn#25971) * DOC replace deviance by loss in docstring of GradientBoosting (scikit-learn#25968) * MAINT Fix broken link in feature_selection/_univariate_selection.py (scikit-learn#25984) * DOC Update model_persistence.rst to fix skops example (scikit-learn#25993) Co-authored-by: adrinjalali <[email protected]> * DOC Specified meaning for max_patches=None in extract_patches_2d (scikit-learn#25996) * DOC document that last step is never cached in pipeline (scikit-learn#25995) Co-authored-by: Guillaume Lemaitre <[email protected]> * FIX SequentialFeatureSelector throws IndexError when cv is a generator (scikit-learn#25973) * ENH Adds infrequent categories support to OrdinalEncoder (scikit-learn#25677) Co-authored-by: Tim Head <[email protected]> Co-authored-by: Olivier Grisel <[email protected]> Co-authored-by: Andreas Mueller <[email protected]> * MAINT make plot_digits_denoising deterministic by fixing random state (scikit-learn#26004) * DOC improve example of PatchExtractor (scikit-learn#26002) * MAINT Parameters validation for datasets.make_friedman2 (scikit-learn#25986) * MAINT Parameters validation for datasets.make_friedman3 (scikit-learn#25989) * MAINT Parameters validation for datasets.make_sparse_uncorrelated (scikit-learn#26001) * MAINT Parameters validation for datasets.make_spd_matrix (scikit-learn#26003) * MAINT Parameters validation for datasets.make_sparse_spd_matrix (scikit-learn#26009) * DOC Added the meanings of default=None for PatchExtractor parameters (scikit-learn#26005) * MAINT remove unecessary check covered by parameter validation framework (scikit-learn#26014) * MAINT Consistent cython types from _typedefs (scikit-learn#25942) Co-authored-by: Julien Jerphanion <[email protected]> * MAINT Parameters validation for datasets.make_swiss_roll (scikit-learn#26020) * MAINT Parameters validation for datasets.make_s_curve (scikit-learn#26022) * MAINT Parameters validation for datasets.make_blobs (scikit-learn#25983) Co-authored-by: Guillaume Lemaitre <[email protected]> * DOC fix SplineTransformer include_bias docstring (scikit-learn#26018) * ENH RocCurveDisplay add option to plot chance level (scikit-learn#25987) * DOC show from_estimator and from_predictions for Displays (scikit-learn#25994) * EXA Fix rst in plot_partial_dependence (scikit-learn#26028) * CI Adds coverage to docker jobs on Azure (scikit-learn#26027) Co-authored-by: Julien Jerphanion <[email protected]> Co-authored-by: Olivier Grisel <[email protected]> * API Replace `n_iter` in `Bayesian Ridge` and `ARDRegression` (scikit-learn#25697) Co-authored-by: Guillaume Lemaitre <[email protected]> * CLN Make _NumPyAPIWrapper naming consistent to _ArrayAPIWrapper (scikit-learn#26039) * CI disable coverage on Windows to keep CI times reasonable (scikit-learn#26052) * DOC Use Scientific Python Plausible instance for analytics (scikit-learn#25547) * MAINT Parameters validation for sklearn.preprocessing.scale (scikit-learn#26036) * MAINT Parameters validation for sklearn.metrics.pairwise.haversine_distances (scikit-learn#26047) * MAINT Parameters validation for sklearn.metrics.pairwise.laplacian_kernel (scikit-learn#26048) * MAINT Parameters validation for sklearn.metrics.pairwise.linear_kernel (scikit-learn#26049) * MAINT Parameters validation for sklearn.metrics.silhouette_samples (scikit-learn#26053) * MAINT Parameters validation for sklearn.preprocessing.add_dummy_feature (scikit-learn#26058) * Added Parameter Validation for metrics.cluster.normalized_mutual_info_score() (scikit-learn#26060) * DOC Typos in HistGradientBoosting documentation (scikit-learn#26057) * TST add global_random_seed fixture to sklearn/datasets/tests/test_rcv1.py (scikit-learn#26043) * MAINT Parameters validation for sklearn.metrics.pairwise.cosine_similarity (scikit-learn#26006) Co-authored-by: Jérémie du Boisberranger <[email protected]> * ENH Adds isdtype to Array API wrapper (scikit-learn#26029) * MAINT Parameters validation for sklearn.metrics.silhouette_score (scikit-learn#26054) Co-authored-by: Jérémie du Boisberranger <[email protected]> * FIX fix spelling mistake in _NumPyAPIWrapper (scikit-learn#26064) * CI ignore more non-library Python files in codecov (scikit-learn#26059) * MAINT Parameters validation for sklearn.metrics.pairwise.cosine_distances (scikit-learn#26046) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Introduce BinaryClassifierCurveDisplayMixin (scikit-learn#25969) Co-authored-by: Jérémie du Boisberranger <[email protected]> * ENH Forces shape to be tuple when using Array API's reshape (scikit-learn#26030) Co-authored-by: Olivier Grisel <[email protected]> Co-authored-by: Tim Head <[email protected]> * MAINT Parameters validation for sklearn.metrics.pairwise.paired_euclidean_distances (scikit-learn#26073) * MAINT Parameters validation for sklearn.metrics.pairwise.paired_manhattan_distances (scikit-learn#26074) * MAINT Parameters validation for sklearn.metrics.pairwise.paired_cosine_distances (scikit-learn#26075) * MAINT Parameters validation for sklearn.preprocessing.binarize (scikit-learn#26076) * MAINT Parameters validation for metrics.explained_variance_score (scikit-learn#26079) * DOC use correct template name for displays (scikit-learn#26081) * MAINT Parameters validation for sklearn.preprocessing.maxabs_scale (scikit-learn#26077) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.preprocessing.label_binarize (scikit-learn#26078) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT parameter validation for d2_absolute_error_score (scikit-learn#26066) Co-authored-by: jeremiedbb <[email protected]> * MAINT Parameter validation for roc_auc_score (scikit-learn#26007) Co-authored-by: jeremiedbb <[email protected]> * MAINT Parameters validation for sklearn.preprocessing.normalize (scikit-learn#26069) Co-authored-by: jeremiedbb <[email protected]> * MAINT Parameter validation for metrics.cluster.fowlkes_mallows_score (scikit-learn#26080) Co-authored-by: jeremiedbb <[email protected]> * MAINT Parameters validation for compose.make_column_transformer (scikit-learn#25897) Co-authored-by: jeremiedbb <[email protected]> * MAINT Parameters validation for sklearn.metrics.pairwise.polynomial_kernel (scikit-learn#26070) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.metrics.pairwise.rbf_kernel (scikit-learn#26071) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.metrics.pairwise.sigmoid_kernel (scikit-learn#26072) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Param validation: constraint for numeric missing values (scikit-learn#26085) * FIX Adds support for negative values in categorical features in gradient boosting (scikit-learn#25629) Co-authored-by: Julien Jerphanion <[email protected]> Co-authored-by: Tim Head <[email protected]> * MAINT Fix C warning in Cython module splitting.pyx (scikit-learn#26051) * MNT Updates _isotonic.pyx to use memoryviews instead of `cnp.ndarray` (scikit-learn#26068) * FIX Fixes memory regression for inspecting extension arrays (scikit-learn#26106) * PERF set openmp to use only physical cores by default (scikit-learn#26082) * MNT Update black to 23.3.0 (scikit-learn#26110) * MNT Adds black commit to git-blame-ignore-revs (scikit-learn#26111) * MAINT Parameters validation for sklearn.metrics.pair_confusion_matrix (scikit-learn#26107) * MAINT Parameters validation for sklearn.metrics.mean_poisson_deviance (scikit-learn#26104) * DOC Use notebook style in plot_lof_outlier_detection.py (scikit-learn#26017) Co-authored-by: Jérémie du Boisberranger <[email protected]> Co-authored-by: Guillaume Lemaitre <[email protected]> * MAINT utils._fast_dict uses types from utils._typedefs (scikit-learn#26025) * DOC remove sparse-matrix for `y` in ElasticNet (scikit-learn#26127) * ENH add exponential loss (scikit-learn#25965) * MAINT Parameters validation for sklearn.preprocessing.robust_scale (scikit-learn#26086) * MAINT Parameters validation for sklearn.datasets.fetch_rcv1 (scikit-learn#26126) * MAINT Parameters validation for sklearn.metrics.adjusted_rand_score (scikit-learn#26134) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.metrics.calinski_harabasz_score (scikit-learn#26135) * MAINT Parameters validation for sklearn.metrics.davies_bouldin_score (scikit-learn#26136) * MAINT: remove `from numpy.math cimport` statements (scikit-learn#26143) * MAINT Parameters validation for sklearn.inspection.permutation_importance (scikit-learn#26145) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.metrics.cluster.homogeneity_completeness_v_measure (scikit-learn#26137) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.metrics.rand_score (scikit-learn#26138) Co-authored-by: Jérémie du Boisberranger <[email protected]> * DOC update comment in metrics/tests/test_classification.py (scikit-learn#26150) * CI small cleanup of Cirrus CI test script (scikit-learn#26168) * MAINT remove deprecated is_categorical_dtype (scikit-learn#26156) * DOC Add skforecast to related projects page (scikit-learn#26133) Co-authored-by: Thomas J. Fan <[email protected]> * FIX Keeps namedtuple's class when transform returns a tuple (scikit-learn#26121) * DOC corrected letter case for better readability in sklearn/metrics/_classification.py / (scikit-learn#26169) * MAINT Parameters validation for sklearn.preprocessing.power_transform (scikit-learn#26142) * FIX `roc_auc_score` now uses `y_prob` instead of `y_pred` (scikit-learn#26155) * MAINT Parameters validation for sklearn.datasets.load_iris (scikit-learn#26177) * MAINT Parameters validation for sklearn.datasets.load_diabetes (scikit-learn#26166) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.datasets.load_breast_cancer (scikit-learn#26165) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.metrics.cluster.entropy (scikit-learn#26162) * MAINT Parameters validation for sklearn.datasets.fetch_species_distributions (scikit-learn#26161) Co-authored-by: Jérémie du Boisberranger <[email protected]> * ASV Fix tol in SGDRegressorBenchmark (scikit-learn#26146) Co-authored-by: jeremie du boisberranger <[email protected]> * MNT use api.openml.org URLs for fetch_openml (scikit-learn#26171) * MAINT Parameters validation for sklearn.utils.resample (scikit-learn#26139) * MAINT make it explicit that additive_chi2_kernel does not accept sparse matrix (scikit-learn#26178) * MNT fix circleci link in README.rst (scikit-learn#26183) * CI Fix circleci artifact redirector action (scikit-learn#26181) * GOV introduce rights for groups as discussed in SLEP019 (scikit-learn#25753) Co-authored-by: Julien <[email protected]> Co-authored-by: Thomas J. Fan <[email protected]> * MAINT Parameters validation for sklearn.neighbors.sort_graph_by_row_values (scikit-learn#26173) Co-authored-by: Jérémie du Boisberranger <[email protected]> * FIX improve convergence criterion for LogisticRegression(penalty="l1", solver='liblinear') (scikit-learn#25214) Co-authored-by: Thomas J. Fan <[email protected]> Co-authored-by: Olivier Grisel <[email protected]> * MAINT Fix several typos in src and doc files (scikit-learn#26187) * PERF fix overhead of _rescale_data in LinearRegression (scikit-learn#26207) * ENH add Huber loss (scikit-learn#25966) * MAINT Refactor GraphicalLasso and graphical_lasso (scikit-learn#26033) Co-authored-by: Guillaume Lemaitre <[email protected]> Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Cython linting (scikit-learn#25861) * DOC Add JupyterLite button in example gallery (scikit-learn#25887) * MAINT Parameters validation for sklearn.covariance.ledoit_wolf_shrinkage (scikit-learn#26200) * MAINT Parameters validation for sklearn.datasets.load_linnerud (scikit-learn#26199) * MAINT Parameters validation for sklearn.datasets.load_wine (scikit-learn#26196) * DOC Added redirect to Provost paper + minor refactor (scikit-learn#26223) * MAINT Parameter Validation for `covariance.graphical_lasso` (scikit-learn#25053) Co-authored-by: Guillaume Lemaitre <[email protected]> Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.datasets.load_digits (scikit-learn#26195) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.preprocessing.quantile_transform (scikit-learn#26144) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.model_selection.cross_validate (scikit-learn#26129) Co-authored-by: jeremiedbb <[email protected]> * DOC Adds TargetEncoder example explaining the internal CV (scikit-learn#26185) Co-authored-by: Tim Head <[email protected]> * spelling mistake corrected in documentation for script `plot_document_clustering.py` (scikit-learn#26228) Co-authored-by: Olivier Grisel <[email protected]> * FIX possible UnboundLocalError in fetch_openml (scikit-learn#26236) * ENH Adds PyTorch support to LinearDiscriminantAnalysis (scikit-learn#25956) Co-authored-by: Olivier Grisel <[email protected]> Co-authored-by: Tim Head <[email protected]> * MNT Use fixed version of Pyodide (scikit-learn#26247) * MNT Reset transform_output default in example to fix doc build build (scikit-learn#26269) * DOC Update example plot_nearest_centroid.py (scikit-learn#26263) * MNT reduce JupyterLite build size (scikit-learn#26246) * DOC term -> meth in GradientBoosting (scikit-learn#26225) * MNT speed-up html-noplot build (scikit-learn#26245) Co-authored-by: Thomas J. Fan <[email protected]> * MNT Use copy=False when creating DataFrames (scikit-learn#26272) * MAINT Parameters validation for sklearn.model_selection.permutation_test_score (scikit-learn#26230) * MAINT Parameters validation for sklearn.datasets.clear_data_home (scikit-learn#26259) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.datasets.load_files (scikit-learn#26203) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.datasets.get_data_home (scikit-learn#26260) Co-authored-by: Jérémie du Boisberranger <[email protected]> * DOC Fix y-axis plot labels in permutation test score example (scikit-learn#26240) * MAINT cython-lint ignores asv_benchmarks (scikit-learn#26282) * MAINT Parameter validation for metrics.cluster._supervised (scikit-learn#26258) Co-authored-by: Jérémie du Boisberranger <[email protected]> * DOC Improve docstring for tol in SequentialFeatureSelector (scikit-learn#26271) * MAINT Parameters validation for sklearn.datasets.load_sample_image (scikit-learn#26226) Co-authored-by: Jérémie du Boisberranger <[email protected]> * DOC Consistent param type for pos_label (scikit-learn#26237) * DOC Minor grammar fix to imputation docs (scikit-learn#26283) * MAINT Parameters validation for sklearn.calibration.calibration_curve (scikit-learn#26198) Co-authored-by: jeremie du boisberranger <[email protected]> * MAINT Parameters validation for sklearn.inspection.partial_dependence (scikit-learn#26209) Co-authored-by: jeremie du boisberranger <[email protected]> * MAINT Parameters validation for sklearn.model_selection.validation_curve (scikit-learn#26229) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MAINT Parameters validation for sklearn.model_selection.learning_curve (scikit-learn#26227) Co-authored-by: jeremie du boisberranger <[email protected]> * MNT Remove deprecated pandas.api.types.is_sparse (scikit-learn#26287) * CI Use Trusted Publishers for uploading wheels to PyPI (scikit-learn#26249) * MAINT Parameters validation for sklearn.metrics.pairwise.manhattan_distances (scikit-learn#26122) * PERF revert openmp use in csr_row_norms (scikit-learn#26275) * MAINT Parameters validation for metrics.check_scoring (scikit-learn#26041) Co-authored-by: Jérémie du Boisberranger <[email protected]> * MNT Improve error message when checking classification target is of a non-regression type (scikit-learn#26281) Co-authored-by: Adrin Jalali <[email protected]> Co-authored-by: Thomas J. Fan <[email protected]> * DOC fix link to User Guide encoder_infrequent_categories (scikit-learn#26309) * MNT remove unused args in _predict_regression_tree_inplace_fast_dense (scikit-learn#26314) * ENH Adds missing value support for trees (scikit-learn#23595) Co-authored-by: Tim Head <[email protected]> Co-authored-by: Julien Jerphanion <[email protected]> * CLN Clean up logic in validate_data and cast_to_ndarray (scikit-learn#26300) * MAINT refactor scorer using _get_response_values (scikit-learn#26037) Co-authored-by: Jérémie du Boisberranger <[email protected]> Co-authored-by: Adrin Jalali <[email protected]> * DOC Add HGBDT to "see also" section of random forests (scikit-learn#26319) Co-authored-by: ArturoAmorQ <[email protected]> Co-authored-by: Tim Head <[email protected]> * MNT Bump Github Action labeler version to use newer Node (scikit-learn#26302) * FIX thresholds should not exceed 1.0 with probabilities in `roc_curve` (scikit-learn#26194) Co-authored-by: Olivier Grisel <[email protected]> * ENH Allow for appropriate dtype us in `preprocessing.PolynomialFeatures` for sparse matrices (scikit-learn#23731) Co-authored-by: Aleksandr Kokhaniukov <[email protected]> Co-authored-by: Olivier Grisel <[email protected]> Co-authored-by: Julien Jerphanion <[email protected]> Co-authored-by: Thomas J. Fan <[email protected]> * DOC Fix minor typo (scikit-learn#26327) * MAINT bump minimum version for pytest (scikit-learn#26184) Co-authored-by: Loïc Estève <[email protected]> Co-authored-by: Adrin Jalali <[email protected]> Co-authored-by: Olivier Grisel <[email protected]> * DOC fix return type in isotonic_regression (scikit-learn#26332) * FIX fix available_if for MultiOutputRegressor.partial_fit (scikit-learn#26333) Co-authored-by: Guillaume Lemaitre <[email protected]> * FIX make pipeline pass check_estimator (scikit-learn#26325) * FEA Add multiclass support to `average_precision_score` (scikit-learn#24769) Co-authored-by: Geoffrey <[email protected]> Co-authored-by: gbolmier <[email protected]> Co-authored-by: Guillaume Lemaitre <[email protected]> Co-authored-by: Thomas J. Fan <[email protected]> --------- Signed-off-by: Julien Jerphanion <[email protected]> Co-authored-by: Jérémie du Boisberranger <[email protected]> Co-authored-by: Meekail Zain <[email protected]> Co-authored-by: Julien Jerphanion <[email protected]> Co-authored-by: Olivier Grisel <[email protected]> Co-authored-by: zeeshan lone <[email protected]> Co-authored-by: jeremiedbb <[email protected]> Co-authored-by: Adrin Jalali <[email protected]> Co-authored-by: Shiva chauhan <[email protected]> Co-authored-by: AymericBasset <[email protected]> Co-authored-by: Maren Westermann <[email protected]> Co-authored-by: Nishu Choudhary <[email protected]> Co-authored-by: Guillaume Lemaitre <[email protected]> Co-authored-by: Loïc Estève <[email protected]> Co-authored-by: Benedek Harsanyi <[email protected]> Co-authored-by: Pooja Subramaniam <[email protected]> Co-authored-by: Rushil Desai <[email protected]> Co-authored-by: Xiao Yuan <[email protected]> Co-authored-by: Omar Salman <[email protected]> Co-authored-by: 2357juan <[email protected]> Co-authored-by: Théophile Baranger <[email protected]> Co-authored-by: Thomas J. Fan <[email protected]> Co-authored-by: Andreas Mueller <[email protected]> Co-authored-by: Jovan Stojanovic <[email protected]> Co-authored-by: Rahil Parikh <[email protected]> Co-authored-by: Bharat Raghunathan <[email protected]> Co-authored-by: Sortofamudkip <[email protected]> Co-authored-by: Gleb Levitski <[email protected]> Co-authored-by: Christian Lorentzen <[email protected]> Co-authored-by: Ashwin Mathur <[email protected]> Co-authored-by: Sahil Gupta <[email protected]> Co-authored-by: Veghit <[email protected]> Co-authored-by: Itay <[email protected]> Co-authored-by: precondition <[email protected]> Co-authored-by: Marc Torrellas Socastro <[email protected]> Co-authored-by: Dominic Fox <[email protected]> Co-authored-by: futurewarning <[email protected]> Co-authored-by: Yao Xiao <[email protected]> Co-authored-by: Joey Ortiz <[email protected]> Co-authored-by: Tim Head <[email protected]> Co-authored-by: Christian Veenhuis <[email protected]> Co-authored-by: adienes <[email protected]> Co-authored-by: Dave Berenbaum <[email protected]> Co-authored-by: Lene Preuss <[email protected]> Co-authored-by: A.H.Mansouri <[email protected]> Co-authored-by: Boris Feld <[email protected]> Co-authored-by: Carla J <[email protected]> Co-authored-by: windiana42 <[email protected]> Co-authored-by: mdarii <[email protected]> Co-authored-by: murezzda <[email protected]> Co-authored-by: Peter Piontek <[email protected]> Co-authored-by: John Pangas <[email protected]> Co-authored-by: Dmitry Nesterov <[email protected]> Co-authored-by: Yuchen Zhou <[email protected]> Co-authored-by: Ekaterina Butyugina <[email protected]> Co-authored-by: Jiawei Zhang <[email protected]> Co-authored-by: Ansam Zedan <[email protected]> Co-authored-by: genvalen <[email protected]> Co-authored-by: farhan khan <[email protected]> Co-authored-by: Arturo Amor <[email protected]> Co-authored-by: Jiawei Zhang <[email protected]> Co-authored-by: Ralf Gommers <[email protected]> Co-authored-by: Jessicakk0711 <[email protected]> Co-authored-by: Ankur Singh <[email protected]> Co-authored-by: Seoeun(Sun☀️) Hong <[email protected]> Co-authored-by: Nightwalkx <[email protected]> Co-authored-by: VIGNESH D <[email protected]> Co-authored-by: Vincent-violet <[email protected]> Co-authored-by: Elabonga Atuo <[email protected]> Co-authored-by: Tom Dupré la Tour <[email protected]> Co-authored-by: André Pedersen <[email protected]> Co-authored-by: Ashish Dutt <[email protected]> Co-authored-by: Phil <[email protected]> Co-authored-by: Stanislav (Stanley) Modrak <[email protected]> Co-authored-by: hujiahong726 <[email protected]> Co-authored-by: James Dean <[email protected]> Co-authored-by: ArturoAmorQ <[email protected]> Co-authored-by: Aleksandr Kokhaniukov <[email protected]> Co-authored-by: c-git <[email protected]> Co-authored-by: annegnx <[email protected]> Co-authored-by: Geoffrey <[email protected]> Co-authored-by: gbolmier <[email protected]>

Add drop_intermediate kwarg to metrics.precision_recall_curve

498806d

github-actions bot added the module:metrics label Oct 15, 2022

betatim reviewed Oct 17, 2022

View reviewed changes

betatim approved these changes Oct 17, 2022

View reviewed changes

betatim approved these changes Oct 18, 2022

View reviewed changes

glemaitre changed the title ~~[MRG] Add drop_intermediate kwarg to metrics.precision_recall_curve~~ ENH Add drop_intermediate parameter to metrics.precision_recall_curve Nov 3, 2022

glemaitre reviewed Nov 3, 2022

View reviewed changes

dberenbaum and others added 8 commits November 3, 2022 19:46

Update sklearn/metrics/_plot/precision_recall_curve.py

22dadbf

Co-authored-by: Guillaume Lemaitre <[email protected]>

Update sklearn/metrics/_plot/precision_recall_curve.py

a2148f7

Co-authored-by: Guillaume Lemaitre <[email protected]>

Update sklearn/metrics/_plot/precision_recall_curve.py

1fb7a8f

Co-authored-by: Guillaume Lemaitre <[email protected]>

Update sklearn/metrics/_ranking.py

6765041

Co-authored-by: Guillaume Lemaitre <[email protected]>

Update sklearn/metrics/tests/test_ranking.py

26e5813

Co-authored-by: Guillaume Lemaitre <[email protected]>

Update sklearn/metrics/tests/test_ranking.py

3cdc102

Co-authored-by: Guillaume Lemaitre <[email protected]>

Update sklearn/metrics/tests/test_ranking.py

357f70b

Co-authored-by: Guillaume Lemaitre <[email protected]>

Addressed comments on PR scikit-learn#24668: Add drop_intermediate …

3e02ec5

…parameter to `metrics.precision_recall_curve`

glemaitre self-requested a review November 4, 2022 19:09

glemaitre added 2 commits November 4, 2022 20:13

FIX move drop_intermediate in docstring

b2bda83

FIX double versionadded

c1423eb

glemaitre reviewed Nov 4, 2022

View reviewed changes

dberenbaum and others added 4 commits November 4, 2022 17:04

Update sklearn/metrics/_ranking.py

918bb34

Co-authored-by: Guillaume Lemaitre <[email protected]>

Update sklearn/metrics/_plot/precision_recall_curve.py

f4a3071

Co-authored-by: Guillaume Lemaitre <[email protected]>

Update sklearn/metrics/_plot/precision_recall_curve.py

002dac4

Co-authored-by: Guillaume Lemaitre <[email protected]>

Update sklearn/metrics/_plot/precision_recall_curve.py

68f7fd0

Co-authored-by: Guillaume Lemaitre <[email protected]>

Update sklearn/metrics/_plot/precision_recall_curve.py

5ed273e

Co-authored-by: Guillaume Lemaitre <[email protected]>

glemaitre approved these changes Nov 4, 2022

View reviewed changes

Merge branch 'main' into prc-drop-intermediate

211d599

betatim mentioned this pull request Nov 18, 2022

Control default behavior of PR curve #24976

Closed

betatim approved these changes Nov 22, 2022

View reviewed changes

glemaitre added the Waiting for Second Reviewer First reviewer is done, need a second one! label Jan 9, 2023

glemaitre added this to the 1.3 milestone Jan 11, 2023

stephanecollot mentioned this pull request Mar 14, 2023

Sampling uncertainty on precision-recall and ROC curves #25856

Open

jeremiedbb added 2 commits March 24, 2023 16:15

Merge remote-tracking branch 'upstream/main' into pr/dberenbaum/24668

34a6b5a

target 1.3

b893761

jeremiedbb approved these changes Mar 24, 2023

View reviewed changes

jeremiedbb merged commit 01e1f97 into scikit-learn:main Mar 24, 2023

Veghit pushed a commit to Veghit/scikit-learn that referenced this pull request Apr 15, 2023

ENH Add drop_intermediate parameter to metrics.precision_recall_curve (…

998bbda

…scikit-learn#24668) Co-authored-by: Guillaume Lemaitre <[email protected]>

Uh oh!

ENH Add drop_intermediate parameter to metrics.precision_recall_curve #24668

ENH Add drop_intermediate parameter to metrics.precision_recall_curve #24668

Uh oh!

Conversation

dberenbaum commented Oct 15, 2022

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Uh oh!

betatim Oct 17, 2022

Choose a reason for hiding this comment

Uh oh!

betatim Oct 17, 2022

Choose a reason for hiding this comment

Uh oh!

betatim Oct 17, 2022

Choose a reason for hiding this comment

Uh oh!

dberenbaum Oct 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

betatim Oct 18, 2022

Choose a reason for hiding this comment

Uh oh!

betatim left a comment

Choose a reason for hiding this comment

Uh oh!

dberenbaum commented Oct 17, 2022

Uh oh!

betatim commented Oct 18, 2022

Uh oh!

glemaitre left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dberenbaum commented Nov 4, 2022

Uh oh!

glemaitre left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dberenbaum commented Nov 4, 2022

Uh oh!

glemaitre left a comment

Choose a reason for hiding this comment

Uh oh!

betatim left a comment

Choose a reason for hiding this comment

Uh oh!

dberenbaum commented Jan 6, 2023

Uh oh!

glemaitre commented Jan 9, 2023

Uh oh!

betatim commented Jan 11, 2023

Uh oh!

jeremiedbb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ENH Add `drop_intermediate` parameter to `metrics.precision_recall_curve` #24668

ENH Add `drop_intermediate` parameter to `metrics.precision_recall_curve` #24668

dberenbaum Oct 17, 2022 •

edited

Loading