Would also make something like hv.Dataset(df_pandas).select(selection_expr=hv.dim('a')) fail

Open for updating the position here.

Position seems fine, though question is at what point we deprecate dask and cuDF interfaces (if ever).

Which LazyFrame types does narwhals support these days? Can we check if it's backed by a dask dataframe and avoid collect for that case?

yup you can do if df.implementation.is_dask():

Have done this in 2c6388e

Would implement type as well if possible.

Implemented like this:

    @property
    def type(self):
        return type(self.dtype)

Hmm, there's no nanmin/nanmax?

Polars defaults to ignoring NaN values, so Polars' nan_min actually propagates nan instead of ignoring them

In general NaN values are rare to encounter outside of pandas, as other libraries have proper null value support (and uniformly use null to indicate missing data) and only result from undefined mathematical operations like 0/0 or log(-1). My general advise is to just deal with null values (as you're already doing here with drop_nulls) and let each backend deal with its own definition of null, and then let NaN be treated as a user-error. Alternatively, you can call fill_nan but note that it's only supported on float columns

There was some discussion previously #6567 (comment) and #6567 (comment).

Hmm, that sucks.

hey - I just checked and Polars allows this:

In [4]: df = pl.DataFrame({'a': [3,3,2,1], 'b': [3,2,2,1], 'c': [1,2,3,4]})

In [5]: df.filter((df['a']>1).to_numpy())
Out[5]:
shape: (3, 3)
┌─────┬─────┬─────┐
│ a   ┆ b   ┆ c   │
│ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ i64 │
╞═════╪═════╪═════╡
│ 3   ┆ 3   ┆ 1   │
│ 3   ┆ 2   ┆ 2   │
│ 2   ┆ 2   ┆ 3   │
└─────┴─────┴─────┘

whereas Narwhals errors

I think we should allow for this in Narwhals (at least, for the eager case). Would it be OK for you to have this supported just for the eager case?

Yes, I don't think it can be supported properly for the lazy case, e.g. for our dask interface we don't support it either.

Hey sorry to chime in. Without waiting for a new narwhals release/having to pin narwhals to latest one, it is still possible to avoid casting to list by converting numpy to a narwhals series backed by the backend:

import narwhals as nw
import numpy as np
import pyarrow as pa

frame = nw.from_native(pa.table({"a": [1,2,3]}), eager_only=True)
mask_np = np.array([True, False, True])
mask_nw = nw.new_series(
    name="mask",
    values=mask_np,
    backend=frame.implementation,
)

# or in more recent versions, even better
# mask_nw = nw.Series.from_numpy("mask", mask_np, backend=frame.implementation)

frame.filter(mask_nw)
┌──────────────────┐
|Narwhals DataFrame|
|------------------|
|  pyarrow.Table   |
|  a: int64        |
|  ----            |
|  a: [[1,3]]      |
└──────────────────┘

That seems more elegant, thanks @FBruzzesi.

And you are always welcome to review the code 👍

Can see I already did something like that here:

https://github.com/holoviz/holoviews/pull/6567/files#diff-f29da10e5d3aa697b91b59b0b5765e41b95ac043010b8a73f7c2d652e1a554e1R321-R325

What's the return type here, e.g. pandas this would be a pd.Series if keep_index else np.ndarray?

It will return a nw.Series, except if compute is False, then a LazyFrame input will return a single-column LazyFrame.

Definitely a considerable broadening of our definition of DataFrame.

Do you want to force it to pandas? Or should we try this out for now and enable it if we see too many problems with it?