[BUG] RidgeClassifierCV(scoring="accuracy", cv=None) returns incorrect best_score_ for binary classification #34942

Introduce yourself

hello, I am a professor of computer science and a core developer at aeon . We use RidgeClassifierCV as a classifier head for many of our transformation based pipelines and ensembles, particularly the convolution based ones, and this bug was found by our core dev Matthew (@MatthewMiddlehurst) when reviewing [this PR] (aeon-toolkit/aeon#3655) that adapted one of these pipeline classifiers, the Arsenal, which is a component of our HIVE-COTEv2 classifier and is one of my finest moments in algorithm naming :)

Describe the bug and give evidence about its user-facing impact

RidgeClassifierCV(scoring="accuracy", cv=None) computes an incorrect best_score_ for binary classification problems: it always returns accuracy of 1.0, even when the problem contains substantial label noise and the fitted classifier is clearly not perfect.

We use best_score_ as an estimate of each ensemble member's classification accuracy and use it as an ensemble weight. For multiclass problems this appears to work as expected. For binary problems, however, all ensemble members receive a weight of 1.0, effectively disabling the weighting mechanism.

The behaviour appears specific to the default cv=None path. Using an explicit cross-validation splitter such as cv=5 gives plausible accuracy scores.

Looking at the implementation, the cause may be the classification scoring code in the GCV path reconstructing class labels using y.argmax(axis=1). For binary classification, LabelBinarizer produces a single-column target, so argmax(axis=1) is always zero.

Steps/Code to Reproduce

import numpy as np
from sklearn.linear_model import RidgeClassifierCV
rng = np.random.RandomState(0)
X = rng.normal(size=(200, 20))
y = rng.randint(0, 2, size=200)

clf_loo = RidgeClassifierCV( alphas=[0.1, 1.0, 10.0], scoring="accuracy", ).fit(X, y)
clf_cv = RidgeClassifierCV( alphas=[0.1, 1.0, 10.0], scoring="accuracy", cv=5, ).fit(X, y)
print("cv=None best_score_:", clf_loo.best_score_)
print("cv=5 best_score_:", clf_cv.best_score_)
print("training accuracy:", clf_loo.score(X, y))

import sklearn;
sklearn.show_versions()

Outputs

cv=None best_score_: 1.0
cv=5 best_score_: 0.515
training accuracy: 0.65

on random data.

Expected Results

an estimate of accuracy specific to the training data, not always 1.0.

chatgpt traced this back and thinks it was introduced in 0.23 by PR #14848, merged on 6 January 2020. I have not verified this myself.

Actual Results

For binary classification with cv=None, best_score_ is always reported as 1.0

Versions

System:
    python: 3.12.10 (tags/v3.12.10:0cc8128, Apr  8 2025, 12:21:36) [MSC v.1943 64 bit (AMD64)]
executable: C:\Code\tsml-eval\venv\Scripts\python.exe
   machine: Windows-11-10.0.26200-SP0

Python dependencies:
      sklearn: 1.7.2
          pip: 26.0.1
   setuptools: 82.0.1
        numpy: 2.2.6
        scipy: 1.15.3
       Cython: 3.1.3
       pandas: 2.2.3
   matplotlib: 3.10.5
       joblib: 1.5.1
threadpoolctl: 3.6.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libscipy_openblas
       filepath: C:\Code\tsml-eval\venv\Lib\site-packages\numpy.libs\libscipy_openblas64_-13e2df515630b4a41f92893938845698.dll
        version: 0.3.29
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: vcomp
       filepath: C:\Code\tsml-eval\venv\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libscipy_openblas
       filepath: C:\Code\tsml-eval\venv\Lib\site-packages\scipy.libs\libscipy_openblas-f07f5a5d207a3a47104dca54d6d0c86a.dll
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell

Interest in fixing the bug

Yes I would be happy to contribute, as would @MatthewMiddlehurst