Thanks to visit codestin.com
Credit goes to github.com

Skip to content

numpy 1.24.0rc1 gives scalar False in masked_invalid when input is plain array #22720

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
pllim opened this issue Dec 2, 2022 · 10 comments · Fixed by #22849
Closed

numpy 1.24.0rc1 gives scalar False in masked_invalid when input is plain array #22720

pllim opened this issue Dec 2, 2022 · 10 comments · Fixed by #22849
Labels
06 - Regression

Comments

@pllim
Copy link
Contributor

pllim commented Dec 2, 2022

Is this change intentional? I cannot tell.

As a result of this change, when someone pass in plain array and then try to grab the .mask, it has ndim=0 and the shape no longer matches the input, causing some downstream assumptions to break.

xref astropy/specreduce#153

numpy 1.23.5

>>> import numpy as np
>>> np.ma.masked_invalid([1,2,3])  # Same if I pass in np.array([1,2,3])
masked_array(data=[1, 2, 3],
             mask=[False, False, False],
       fill_value=999999)

numpy 1.24.0rc1

>>> import numpy as np
>>> np.ma.masked_invalid([1,2,3])  # Same if I pass in np.array([1,2,3])
masked_array(data=[1, 2, 3],
             mask=False,
       fill_value=999999)
@pllim pllim mentioned this issue Dec 2, 2022
Closed
@charris charris added this to the 1.24.0 release milestone Dec 2, 2022
@seberg
Copy link
Member

seberg commented Dec 3, 2022 •
edited
Loading

Must have been a side-effect of pluggin in masked_where here, so not intentional. I suppose it is not really incorrect in the sense that arr.mask is False is np.ma.nomask and thus a correct spelling for "the mask is false everywhere".

But I don't see why not to just add a get_mask() call (if that was it) to materialize the full mask (and retain behavior), it isn't like we are aiming for consistency in masked arrays at this point/time. (and if we did, I wonder if we should just always materialize the mask.)

@pllim
Copy link
Contributor Author

pllim commented Dec 3, 2022

Thanks for the reply!

In 1.24.0rc1, get_mask also gives a scalar False. But perhaps I have misunderstood what you are saying.

>>> np.ma.core.get_mask([1,2,3])
False

@seberg
Copy link
Member

seberg commented Dec 3, 2022

Oh, it does... I think there should be a way to materialize the full mask, I thinkt here was an internal _get_mask or so for that.
(I also wrote is False up there, that is incorrect, it is is np.False_...)

@seberg
Copy link
Member

seberg commented Dec 3, 2022

Sorry... the point was, there is some utility function that converts the nomask to a full mask... apparently get_mask is not it :).

@pllim
Copy link
Contributor Author

pllim commented Dec 5, 2022

So, this feature will ship for sure with 1.24? Just want to know if I should code in check for this or will this be reverted if it wasn't intentional. Thanks!

@seberg
Copy link
Member

seberg commented Dec 5, 2022

Honestly, I would go with whatever you prefer/think here :). But since you noticed the change, the conservative thing might be to just change back that particular part.

@pllim
Copy link
Contributor Author

pllim commented Dec 5, 2022

I am neutral as well but then again I am not the main devs for the affected package. I only noticed it in "integration testing". The patch isn't horrible and I really don't know how many more packages use it like this.

astropy/specreduce#153

@seberg
Copy link
Member

seberg commented Dec 5, 2022

Well, I guess if we are neutral, it means we are not enthusiastic enough about cleaning up the behavior and should just add the hack to retain the old one ;).

@pllim
Copy link
Contributor Author

pllim commented Dec 5, 2022

Actually, since I replied, specreduce has decided to remove this particular usage altogether, so with astropy/specreduce#155 , this issue no longer affects us but still might for a different package, not sure. cc @ojustino

@ojustino
Copy link

ojustino commented Dec 5, 2022

As @pllim said, this is no longer causing problems for specreduce, so it's a no-urgency issue for us. As a general user, it would be nice to have a way to get an all-False array in this scenario, whether that's through get_mask() or a mask attribute.

@charris charris removed this from the 1.24.0 release milestone Dec 13, 2022
@seberg seberg mentioned this issue Dec 19, 2022
Closed
seberg added a commit to seberg/numpy that referenced this issue Dec 19, 2022
@seberg
BUG: Do not use getdata() in np.ma.masked_invalid
45cb037 
This is the minimal solution to fix numpygh-22826 with as little change
as possible.
We should fix `getdata()` but I don't want to do that in a bug-fix
release really.

IMO the alternative is to revert numpygh-22046 which would also revert
the behavior noticed in numpygh-22720  (which seems less harmful though).

Closes numpygh-22826
@seberg seberg mentioned this issue Dec 19, 2022
Merged
charris pushed a commit to charris/numpy that referenced this issue Dec 19, 2022
@seberg @charris
BUG: Do not use getdata() in np.ma.masked_invalid
b75d897 
This is the minimal solution to fix numpygh-22826 with as little change
as possible.
We should fix `getdata()` but I don't want to do that in a bug-fix
release really.

IMO the alternative is to revert numpygh-22046 which would also revert
the behavior noticed in numpygh-22720  (which seems less harmful though).

Closes numpygh-22826
@charris charris mentioned this issue Dec 19, 2022
Merged
This was referenced Dec 21, 2022
Closed
Merged
seberg added a commit to seberg/numpy that referenced this issue Dec 21, 2022
@seberg
API: Ensure a full mask is returned for masked_invalid
954aee7 
Matplotlib relies on this, so we don't seem to have much of a choice.
I am surprised that we were not notified of the issue before release
time.

Closes numpygh-22720, numpygh-22720
charris pushed a commit to charris/numpy that referenced this issue Dec 21, 2022
@seberg @charris
API: Ensure a full mask is returned for masked_invalid
af7ee55 
Matplotlib relies on this, so we don't seem to have much of a choice.
I am surprised that we were not notified of the issue before release
time.

Closes numpygh-22720, numpygh-22720
@charris charris mentioned this issue Dec 21, 2022
Merged
@EwoutH EwoutH mentioned this issue Dec 28, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
06 - Regression
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

API: Ensure a full mask is returned for masked_invalid seberg/numpy
4 participants
@seberg @charris @pllim @ojustino