FIX check_array with narwhals.DataFrame input - #34051

maybe we shoudl already define df_pandas here, instead of asking and array_df.implementation.is_pandas() afterwards ?

I like relying on nw for which backend is used.

yeah but the flow is weird here. We only use array_df if it comes from pandas.

if _nw_into_df_or_series(array):
    array_df = nw.from_native(array, allow_series=True)
else:
    array_df = None

if array_df is not None and array_df.implentation.is_pandas():
    df_pandas = array_df.to_native()

    # and then only use df_pandas in this branch

I think the following would be more readable:

if _nw_into_df_or_series(array):
    array_df = nw.from_native(array, allow_series=True)
    array_df = array_df.to_native() if array_df.implentation.is_pandas() else None
else:
    array_df = None

if array_df is not None:
    # do everything with array_df

This part is not touched by this PR. Also, I anticipate that array_df will be used more within this function in the future.

Does the warning result from the changes in check_array ?

No. a8d8983 is not related to the bugfix, but removes all warnings in the tests such that I could run pytest -x -We sklearn/utils/tests/tests_validation.py successfully.

Haha if that's your goal, you have a loooong path in front of you 😄 (#29516)

I‘ve already been quite successful in linear models (the ones I touched).

I believe this change is here to avoid the SparseEfficiencyWarning, is that correct ?

Correct.

feels like maybe constructor_name is not the best place for it. To me we should be able to use _convert_container to create a narwhals dataframe backed by a pandas or polars dataframe without having to create a dataframe first. Something like

X = np.array([[1, 2], [3, 4]])
df = _convert_container(X, "pandas", convert_to_narwhals=True)

I see it different. Either nw.df is an option in constructor name, or we don’t deal with nw in _convert_container.
We could also simplify and require in the tests that, e.g., pandas is available to construct nw.df (and rely on ne testing that the backend does not matter).

Still it's weird that you can't chose the backend and that the backend that one gets is based on an hardcoded ordering. For instance, in the test that check sparse you added:

    sdf = pd.DataFrame.sparse.from_spmatrix(sp_mat)
    if convert_to_narwhals:
        sdf = _convert_container(sdf, constructor_name="narwhals")

If the order was different, it would not do what's expected. That's fragile.
(in particular there's no way, with this ordering, to get a narwhals dataframe with a polars backend if pandas is installed)

Maybe here it would just be more appropriate and robust to do

    sdf = pd.DataFrame.sparse.from_spmatrix(sp_mat)
    if convert_to_narwhals:
        sdf = nw.from_native(sdf)