Thanks to visit codestin.com
Credit goes to github.com

Skip to content

TST mark test as xfail due to bug fix in pandas-dev - #26344

Closed
glemaitre wants to merge 1 commit into
scikit-learn:mainfrom
glemaitre:fix_scipy_dev_ci
Closed

TST mark test as xfail due to bug fix in pandas-dev#26344
glemaitre wants to merge 1 commit into
scikit-learn:mainfrom
glemaitre:fix_scipy_dev_ci

Conversation

@glemaitre

Copy link
Copy Markdown
Member

Partially address #26154

Solving the issue pointed out here: #26154 (comment)

In short, pandas will better infer type during DataFrame concatenation with missing values. Previously, due to the way we read by chunk in the liac-arff parser, we could end up with None and np.nan in the same column. The new version of pandas will identify both values are missing values.

Since the new behaviour is what one would expect but we cannot make a backport, a way is to mark the test as xfail.

@adrinjalali

Copy link
Copy Markdown
Member

So this means we won't be really supporting as_frame=True with parser="liac-arff", right? Should we then at least deprecated that usage?

@glemaitre

Copy link
Copy Markdown
Member Author

So this means we won't be really supporting as_frame=True with parser="liac-arff", right?

This does not change. Here, it is just that the number of column detected as numerical or categorical will changed depending if None will be map to a proper missing value (which was not in the passed).

@thomasjpfan

Copy link
Copy Markdown
Member

I think we can adjust the implementation to infer better dtypes. I opened #26386 as an alternative to this PR.

@lesteve

lesteve commented May 23, 2023

Copy link
Copy Markdown
Member

Closing since #26386 has been merged

@lesteve lesteve closed this May 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants