balanced_accuracy_score(adjusted=True) returns -inf when a single class survives the NaN filter #34889

Introduce yourself

I'm a data scientist in Toronto working on forecasting and causal inference. I've recently started contributing to scikit-learn — I have an open PR at #34888 (array API support for ShrunkCovariance) — and I found this while looking into #33712, where I was trying to work out which of two distinct undefined cases in balanced_accuracy_score that issue refers to. This one turned out to be independent of it, so I'm filing it separately.

Describe the bug and give evidence about its user-facing impact

balanced_accuracy_score(..., adjusted=True) returns -inf when a single class survives the internal NaN filter. The adjusted score is bounded — it spans [-1, 1], reaching -1 only at n_classes=2 — so -inf is outside the documented range of the metric.

This surfaces in cross-validation on small or imbalanced folds, where a fold can easily contain a single class. One -inf propagates through any mean over folds, turning the whole aggregate into -inf and silently destroying an otherwise valid CV score. The related nan case is at least absorbed by nan-aware aggregation; -inf is not.

Mechanism: n_classes is read after the NaN filter at _classification.py:2940, so when one class survives, chance = 1 / 1 and score /= 1 - chance at line 2946 divides by zero. The np.errstate guard wraps the per-class division only, not this one, and has been scoped that way since the original commit (e888c0d, "ENH multiclass balanced accuracy" #10587, 2018) — the guard and the unguarded division were introduced together.

This is separate from #33712: it predates it and reproduces without touching replace_undefined_by.

Steps/Code to Reproduce

from sklearn.metrics import balanced_accuracy_score

# Case 1
print(balanced_accuracy_score([0, 0, 0], [0, 0, 0], adjusted=True))

# Case 2
print(balanced_accuracy_score([0, 0, 0], [0, 1, 0], adjusted=True))

Expected Results

A value within the documented range of the adjusted metric, [-1, 1]. A score documented as bounded should not return -inf.

Actual Results

On scikit-learn 1.9.0 (installed from PyPI), case 1 returns nan:

/tmp/sk19/lib/python3.12/site-packages/sklearn/metrics/_classification.py:614: UserWarning: A single label was found in 'y_true' and 'y_pred'. For the confusion matrix to have the correct shape, use the 'labels' parameter to pass all known labels.
  warnings.warn(
/tmp/sk19/lib/python3.12/site-packages/sklearn/metrics/_classification.py:2946: RuntimeWarning: invalid value encountered in scalar divide
  score /= 1 - chance
nan

Case 2 returns -inf:

/tmp/sk19/lib/python3.12/site-packages/sklearn/metrics/_classification.py:2939: UserWarning: y_pred contains classes not in y_true
  warnings.warn("y_pred contains classes not in y_true")
/tmp/sk19/lib/python3.12/site-packages/sklearn/metrics/_classification.py:2946: RuntimeWarning: divide by zero encountered in scalar divide
  score /= 1 - chance
-inf

Both RuntimeWarnings come from the same unguarded statement at line 2946. The texts differ — "invalid value" for case 1, "divide by zero" for case 2 — which maps onto the nan/-inf split: case 1 is 0/0, case 2 is -0.333/0.

Also reproduces unchanged on main (1.10.dev0), where the filter is at line 2969 and the division at line 2975.

Expected Results

A value within the documented range of the adjusted metric, [-1, 1]. A score documented as bounded should not return -inf.

Actual Results

On scikit-learn 1.9.0 (installed from PyPI), case 1 returns nan:

/tmp/sk19/lib/python3.12/site-packages/sklearn/metrics/_classification.py:614: UserWarning: A single label was found in 'y_true' and 'y_pred'. For the confusion matrix to have the correct shape, use the 'labels' parameter to pass all known labels.
  warnings.warn(
/tmp/sk19/lib/python3.12/site-packages/sklearn/metrics/_classification.py:2946: RuntimeWarning: invalid value encountered in scalar divide
  score /= 1 - chance
nan

Case 2 returns -inf:

/tmp/sk19/lib/python3.12/site-packages/sklearn/metrics/_classification.py:2939: UserWarning: y_pred contains classes not in y_true
  warnings.warn("y_pred contains classes not in y_true")
/tmp/sk19/lib/python3.12/site-packages/sklearn/metrics/_classification.py:2946: RuntimeWarning: divide by zero encountered in scalar divide
  score /= 1 - chance
-inf

Both RuntimeWarnings come from the same unguarded statement at line 2946. The texts differ — "invalid value" for case 1, "divide by zero" for case 2 — which maps onto the nan/-inf split: case 1 is 0/0, case 2 is -0.333/0.

Also reproduces unchanged on main (1.10.dev0), where the filter is at line 2969 and the division at line 2975.

Versions

System:
    python: 3.12.14 (main, Sep  3 2026, 03:46:19) [Clang 21.1.8 ]
executable: /tmp/sk19/bin/python
   machine: macOS-26.6.2-arm64-arm-64bit

Python dependencies:
      sklearn: 1.9.0
          pip: 25.0.1
   setuptools: None
        numpy: 2.5.2
        scipy: 1.18.1
       Cython: None
       pandas: None
   matplotlib: None
       joblib: 1.6.0
threadpoolctl: 3.6.0
     narwhals: 2.25.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
    num_threads: 10
         prefix: libomp
       filepath: /private/tmp/sk19/lib/python3.12/site-packages/sklearn/.dylibs/libomp.dylib
        version: None

Interest in fixing the bug

Yes. The root cause is above; I'd rather not propose a return value before triage, since the choice interacts with the replace_undefined_by discussion in #33712.