Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[ENH]: Support pandas.NA #22126

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
randolf-scholz opened this issue Jan 6, 2022 · 3 comments
Open

[ENH]: Support pandas.NA #22126

randolf-scholz opened this issue Jan 6, 2022 · 3 comments
Labels
New feature third-party integration: pandas

Comments

@randolf-scholz
Copy link
Contributor

Problem

Since a while now, pandas is moving towards better support for data with missing values: https://pandas.pydata.org/pandas-docs/stable/user_guide/missing_data.html

In particular, they introduced nullable data types:

which are all based on pandas.NA. Because missing values are becoming more and more important to handle, we see other projects supporting this feature as well, for example

Proposed solution

Matplotlib should detect pandas.NA and treat it like a missing value. (e.g. like numpy.nan)
@jklymak
Copy link
Member

jklymak commented Jan 6, 2022

What does to_numpy or .values do? We don't directly do anything with pandas data types except immediately turn them into numpy arrays.

If numpy starts supporting other invalid flags we can discuss what to do (probably just add a bunch of redundant checks) but from Matlotlibs point of view I'm not sure there is an action here?

@oscargus
Copy link
Member

It seems like to_numpy is not converting the NA values. Conditionally if dtype=object.

In [5]: df
Out[5]: 
   col1  col2
0     1     3
1     2  <NA>

In [6]: df.to_numpy()
Out[6]: 
array([[1, 3],
       [2, <NA>]], dtype=object)

@oscargus
Copy link
Member

Pandas has a keyword argument na_value that can be use to replace the NA values during conversion. https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_numpy.html

However, xarray does not.

@oscargus oscargus mentioned this issue Apr 23, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
New feature third-party integration: pandas
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jklymak @dstansby @oscargus @randolf-scholz