-
-
Notifications
You must be signed in to change notification settings - Fork 25.9k
Precision/Recall AUC scoring function #5992
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Have you looked at There is an example also: >>> from sklearn.metrics import fbeta_score, make_scorer
>>> ftwo_scorer = make_scorer(fbeta_score, beta=2)
>>> ftwo_scorer
make_scorer(fbeta_score, beta=2)
>>> from sklearn.model_selection import GridSearchCV
>>> from sklearn.svm import LinearSVC
>>> grid = GridSearchCV(LinearSVC(), param_grid={'C': [1, 10]},
... scoring=ftwo_scorer) |
Ahh yes. In fact, I am using make_scorer and a custom implementation of pr_auc right now. Here's my code (feedback welcome). def pr_auc_score(y_true, y_score):
"""
Generates the Area Under the Curve for precision and recall.
"""
precision, recall, thresholds = \
precision_recall_curve(y_true, y_score[:, 1])
return auc(recall, precision, reorder=True)
pr_auc_scorer = make_scorer(pr_auc_score, greater_is_better=True,
needs_proba=True) It just seems to me that, if roc_auc is directly available and it's easy to implement, pr_auc should be available too. |
that is "average_precision" right? |
http://scikit-learn.org/stable/modules/generated/sklearn.metrics.average_precision_score.html (the curve is not interpolated btw, see #4577) |
Oh great! I did not realize that. Thank you for pointing it out. From the docs:
|
I'd like to run GridSearch optimizing for area under the precision/recall curve. It's great that "roc_auc" is readily available, but I'd like to be able to optimize for "pr_auc" too. See http://pages.cs.wisc.edu/~jdavis/davisgoadrichcamera2.pdf for a discussion of the meaningful differences between PR_AUC and ROC_AUC.
See also precision_recall_curve() and auc().
The text was updated successfully, but these errors were encountered: