Partial dependence broken in sklearn 1.6.1 when grid has only two values #30938

MarcBresson · 2025-03-04T09:53:23Z

Describe the bug

When our input feature has two possible values (and that the grid built in that function hence has two values), partial_dependence will raise an error ValueError: cannot reshape array of size 1 into shape (2)

What I suspect is happening is that inside _partial_dependence_brute function, there is a (wrongful) check to see if there are only two predicted values. This check should not be here because there _get_response_values seems to do the job of only getting the positive class already.

Steps/Code to Reproduce

from sklearn.tree import DecisionTreeClassifier
import numpy as np
from sklearn.inspection._partial_dependence import _partial_dependence_brute
from sklearn.inspection import partial_dependence

X_test = np.array([[1., 0], [0., 1], [0., 1], [0., 0], [1., 0], [0., 0], [0., 0]])
clf = DecisionTreeClassifier()
clf.fit(X_test, np.array([0, 1, 1, 0, 0, 0, 0]))

partial_dependence(clf, X=X_test, features=[0], grid_resolution=10, response_method="predict_proba")

Expected Results

.

Actual Results

File /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:216, in validate_params.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    [210](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:210) try:
    [211](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:211)     with config_context(
    [212](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:212)         skip_parameter_validation=(
    [213](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:213)             prefer_skip_nested_validation or global_skip_validation
    [214](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:214)         )
    [215](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:215)     ):
--> [216](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:216)         return func(*args, **kwargs)
    [217](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:217) except InvalidParameterError as e:
    [218](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:218)     # When the function is just a wrapper around an estimator, we allow
    [219](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:219)     # the function to delegate validation to the estimator, but we replace
    [220](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:220)     # the name of the estimator by the name of the function in the error
    [221](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:221)     # message to avoid confusion.
    [222](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:222)     msg = re.sub(
    [223](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:223)         r"parameter of \w+ must be",
    [224](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:224)         f"parameter of {func.__qualname__} must be",
    [225](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:225)         str(e),
    [226](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/utils/_param_validation.py:226)     )

File /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/inspection/_partial_dependence.py:688, in partial_dependence(estimator, X, features, sample_weight, categorical_features, feature_names, response_method, percentiles, grid_resolution, method, kind)
    [684](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/inspection/_partial_dependence.py:684) print("value shapes", [val.shape[0] for val in values])
    [686](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/inspection/_partial_dependence.py:686) # reshape averaged_predictions to
    [687](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/inspection/_partial_dependence.py:687) # (n_outputs, n_values_feature_0, n_values_feature_1, ...)
--> [688](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/inspection/_partial_dependence.py:688) averaged_predictions = averaged_predictions.reshape(
    [689](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/inspection/_partial_dependence.py:689)     -1, *[val.shape[0] for val in values]
    [690](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/inspection/_partial_dependence.py:690) )
    [691](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/inspection/_partial_dependence.py:691) pdp_results = Bunch(grid_values=values)
    [693](https://file+.vscode-resource.vscode-cdn.net/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/inspection/_partial_dependence.py:693) if kind == "average":

ValueError: cannot reshape array of size 1 into shape (2)

Versions

System:
    python: 3.10.11 (v3.10.11:7d4cc5aa85, Apr  4 2023, 19:05:19) [Clang 13.0.0 (clang-1300.0.29.30)]
executable: /usr/local/bin/python3.10
   machine: macOS-14.4.1-arm64-arm-64bit

Python dependencies:
      sklearn: 1.6.1
          pip: 24.2
   setuptools: 74.0.0
        numpy: 1.26.4
        scipy: 1.13.1
       Cython: 3.0.10
       pandas: 1.5.3
   matplotlib: 3.8.4
       joblib: 1.2.0
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: armv8

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.27
threading_layer: pthreads
   architecture: neoversen1

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/sklearn/.dylibs/libomp.dylib
        version: None

The text was updated successfully, but these errors were encountered:

MarcBresson · 2025-03-04T11:09:31Z

it seems like it is fixed on the main branch. Do you happen to know when will the 1.7 be released?

ogrisel · 2025-03-04T15:45:43Z

We release twice a year, so approximately 6 months after 1.6.0 (around June, I think).

MarcBresson · 2025-03-04T15:48:06Z

Will a 1.6.2 get out before then? I could fix that in a separate PR

ogrisel · 2025-03-04T15:54:43Z

We could also backport the fix into 1.6.2 if we can identify which PR is responsible for the fix.

I did a quick git log sklearn/inspection/_partial_dependence.py but I am not sure which PR is responsible for the fix.

fabae96 DOC Inspection Examples links in User Guide (DOC Inspection Examples links in User Guide #30752) (3 weeks ago)
3b75eb9 MAINT postpone erroring in 1.9 when dealing with integer in PDP (MAINT postpone erroring in 1.9 when dealing with integer in PDP #30432) (3 months ago)
4a7f96e FIX deprecate integer valued numerical features for PDP (FIX: deprecate integer valued numerical features for PDP #30409) (3 months ago)
17ab8c0 ENH Add custom_range argument for partial dependence - version 2 (ENH Add custom_range argument for partial dependence - version 2 #26202) (4 months ago)
bfa0d65 MAINT small refactoring in partial_dependence (MAINT small refactoring in partial_dependence #30104) (4 months ago)
1c47967 API Rename force_all_finite into ensure_all_finite (API Rename force_all_finite into ensure_all_finite #29404) (7 months ago)
51c8e0e CLN replace authors with the standard line (CLN replace authors with the standard line #29250) (9 months ago)
55sca335e42 CLN use new license notice line (CLN use new license notice line #29246) (9 months ago)

For the record, here is the full traceback (including the actual exception type and message) when I run the reproducer on 1.6:

Traceback (most recent call last):
  File "<python-input-0>", line 10, in <module>
    partial_dependence(clf, X=X_test, features=[0], grid_resolution=10, response_method="predict_proba")
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ogrisel/miniforge3/envs/sklearn-1.6/lib/python3.13/site-packages/sklearn/utils/_param_validation.py", line 216, in wrapper
    return func(*args, **kwargs)
  File "/Users/ogrisel/miniforge3/envs/sklearn-1.6/lib/python3.13/site-packages/sklearn/inspection/_partial_dependence.py", line 682, in partial_dependence
    averaged_predictions = averaged_predictions.reshape(
        -1, *[val.shape[0] for val in values]
    )
ValueError: cannot reshape array of size 1 into shape (2)

MarcBresson · 2025-03-04T16:02:05Z

In

17ab8c0 ENH Add custom_range argument for partial dependence - version 2 (#26202) (4 months ago)

I can see that change (and this documentation change) that seems to be it. I did not try it tho.

However, this commit does more than just fixing the bug, so i don't know how it should be handled. We could backport only the patch so there will be no (or limited) conflicts later on.

MarcBresson · 2025-03-26T08:57:28Z

Hello, do you have an opinion on how things should be handled?

MarcBresson · 2025-03-26T15:57:03Z

btw, the bug is also on scikit-learn 1.5.2

For reference:

original bug detection: partial_dependence errors when given only two grid points #30271
duplicate issue: Possible bug in sklearn 1.6.1 PartialDependenceDisplay.from_estimator when target and feature are both binary #30675

MarcBresson added Bug Needs Triage Issue requires triage labels Mar 4, 2025

ogrisel removed the Needs Triage Issue requires triage label Mar 4, 2025

ogrisel added the Regression label Mar 4, 2025

ogrisel added this to the 1.6.2 milestone Mar 4, 2025

glemaitre removed this from the 1.6.2 milestone May 5, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Partial dependence broken in sklearn 1.6.1 when grid has only two values #30938

Partial dependence broken in sklearn 1.6.1 when grid has only two values #30938

MarcBresson commented Mar 4, 2025 •

edited by ogrisel

Loading

MarcBresson commented Mar 4, 2025

Uh oh!

ogrisel commented Mar 4, 2025

Uh oh!

MarcBresson commented Mar 4, 2025

Uh oh!

ogrisel commented Mar 4, 2025

Uh oh!

MarcBresson commented Mar 4, 2025

Uh oh!

MarcBresson commented Mar 26, 2025

Uh oh!

MarcBresson commented Mar 26, 2025

Uh oh!

Uh oh!

Partial dependence broken in sklearn 1.6.1 when grid has only two values #30938

Partial dependence broken in sklearn 1.6.1 when grid has only two values #30938

Comments

MarcBresson commented Mar 4, 2025 • edited by ogrisel Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Describe the bug

Steps/Code to Reproduce

Expected Results

Actual Results

Versions

MarcBresson commented Mar 4, 2025

Uh oh!

ogrisel commented Mar 4, 2025

Uh oh!

MarcBresson commented Mar 4, 2025

Uh oh!

ogrisel commented Mar 4, 2025

Uh oh!

MarcBresson commented Mar 4, 2025

Uh oh!

MarcBresson commented Mar 26, 2025

Uh oh!

MarcBresson commented Mar 26, 2025

Uh oh!

MarcBresson commented Mar 4, 2025 •

edited by ogrisel

Loading