Open
Description
Describe the workflow you want to enable
Enable users to conveniently calculate "Precision at Recall K" or "Recall at Precision K" on the probabilities from clf.predict_proba.
clf.fit(X_train, y_train)
y_pred = clf.predict_proba(X_test)
p_at_r99 = precision_at_recall_k(y_test, y_pred, k=.99)
Describe your proposed solution
precision_at_recall_k
(also it's complement recall_at_precision_k
)
Describe alternatives you've considered, if relevant
I've written this code myself probably 10 separate times. Facebook uses this metric pretty heavily, and they have coded up this functionality here: https://github.com/facebookresearch/mmf/blob/322bd33d4a67f5bbe806624d9afd3c2cf921ec50/mmf/modules/metrics.py#L1020