MNT Use integer cumulative sum in confusion_matrix_at_thresholds for unweighted inputs - #34817
Conversation
… unweighted inputs - Avoids float32 precision swamping/saturation on float32-only array API devices when N > 2^24. - Ensures type-safe integer arithmetic between threshold indices and true positive counts. - Preserves y_score floating-point dtype at the output boundary. - Closes scikit-learn#34813 (part 1).
|
Thank you for opening your first pull request to scikit-learn! 🎉 To help get your contribution reviewed, please make sure that:
|
|
Thanks for the PR, however the test is too weak as it would not reproduce the numerical stability problem on a float32 only device. I concurrently vibe-coded a fix for this issue and the test in my branch seems stronger. Let me push it to your branch. |
Use a float32-only large-n non-regression test that fails when cumulative sums saturate past 2**24, covering the bug fixed in scikit-learn#34813. Co-authored-by: Cursor <[email protected]>
|
Pushed a follow-up commit replacing The previous smoke test only checked small float32 arrays / all-pos / all-neg edge cases and would not catch float32 cumsum saturation. The new test uses array-api-strict's |
ogrisel
left a comment
There was a problem hiding this comment.
I think the following lines are redundant.
Co-authored-by: Olivier Grisel <[email protected]>
I already have a local branch that implement a different strategy as what I originally suggested in the issue does not seem to be stable enough. |
|
I ran the Intel GPU tests there and they pass: https://github.com/probabl-ai/scikit-learn-intel-workflow/actions/runs/33060755214 |
Ok
I have actually hit the 1 open PR limit. Can you change the status from your end if its alright? |
|
I reviewed your other PR (#34789). Since it seems to need some more work, maybe you can set that one to draft and mark this one ready for review instead. |
OmarManzoor
left a comment
There was a problem hiding this comment.
Thank you for the PR @Kaynup . A few minor comments otherwise looks good
Co-authored-by: Omar Salman <[email protected]> Co-authored-by: Olivier Grisel <[email protected]>
|
Thank you! But it was mostly Olivier driving the implementation forward! I had a hunch that the unweighted case could serve as a solid foundation for the weighted follow-up (#34827). It's been really exciting to see this develop, and I look forward to contributing more to the Array API initiatives and scikit-learn as I continue learning. Thank you again for the thorough reviews, guidance, and approvals! @ogrisel and @OmarManzoor |
Resolve conflicts with scikit-learn#34817 by keeping both the unweighted int64 cumsum path and the weighted float32-only fixed-point scale path, and retaining both corresponding large-n float32-only tests. Co-authored-by: Olivier Grisel <[email protected]>
…unweighted inputs (scikit-learn#34817) Co-authored-by: Olivier Grisel <[email protected]> Co-authored-by: Cursor <[email protected]>
Reference Issues/PRs
Towards #34813. See also #33200.
What does this implement/fix? Explain your changes.
As discussed in #34813,$2^{24} = 16,777,216$ samples due to floating-point swamping (where adding $+1.0$ rounds down to itself).
confusion_matrix_at_thresholdscurrently computes cumulative counts using floating-point cumulative sums. Onfloat32-only devices (e.g. PyTorch MPS orarray-api-strict's single-precision mock devices),_max_precision_float_dtypereturnsfloat32. Because single-precision floats only have 24 effective bits of mantissa, cumulative sums freeze once counts exceedThis PR implements the unweighted integer accumulation strategy proposed in #34813:
sample_weight is None,y_trueis accumulated in integer space (xp.int64or fallbackxp.int32), giving 100% exact counts and completely preventingfloat32saturation on large datasets.fps_int = (threshold_idxs + 1) - tps_intin the integer domain, avoiding mixed-integer type promotion errors in strict array API modes.tpsandfpsto the floating-pointdtypeofy_scoreat the output boundary.test_confusion_matrix_at_thresholds_unweighted_integer_accumulationinsklearn/metrics/tests/test_ranking.pytesting float32 dtypes, ties, all-positive, and all-negative edge cases.doc/whats_new/upcoming_changes/sklearn.metrics/34813.enhancement.rst.Introduce yourself
Hello everyone, I'm contributing to scikit-learn to help improve array API support and numerical stability across accelerated and single-precision array devices.
AI usage disclosure
I used AI assistance for:
Any other comments?