Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ENH: Add ma.convolve and ma.correlate for #6458 #7922

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 20, 2016

Conversation

eric-wieser
Copy link
Member

@eric-wieser eric-wieser commented Aug 9, 2016

Implements the function requested in #6458

contagious parameter could probably be better named

No tests for correlate, but it should be pretty apparent that if it works for one it works for both

@eric-wieser eric-wieser force-pushed the np-ma-convolve branch 2 times, most recently from cec08b7 to a2f2ac7 Compare August 9, 2016 16:55
@eric-wieser
Copy link
Member Author

Is core.py or extras.py a better place for this?

@ahaldane
Copy link
Member

It looks like the right file to me, and the code looks good.

The only thing is whether "contagious" is the right name.

The masked NEP distinguishes between "Unknown Yet Existing Data (NA)" and "Doesn’t Exist Or Is Being Skipped (IGNORE)" interpretations of the mask, which I think correspond to contagious vs not contagious. That NEP also uses the work "propagate" in a few places, to mean the "NA" case.

"contagious" was already pretty good to me. I propose "propagate_mask", as another possibility. Thoughts?

@eric-wieser
Copy link
Member Author

eric-wieser commented Oct 14, 2016

Is that NEP superseded by np.ma, or intended to replace it?

Looking at that document, it seems that contagious=False should be named skipna=True. Of course, that doesn't make any sense until np.NA is a thing

@ahaldane
Copy link
Member

The NEP was intended to replace it, but it was never implemented.

I would consider the NEP as a collection of ideas on how maskedarray could be reimplemented in the future.

@ahaldane
Copy link
Member

(as far as I understand... it all happened before I was a contributor)

@eric-wieser
Copy link
Member Author

eric-wieser commented Oct 14, 2016

IMO, propogate_mask: bool is worse, as propogate_mask=False seems to (incorrectly) imply that the return value is never masked. Perhaps propogate_mask: Either['any'|'all'] would be better.

@ahaldane
Copy link
Member

ahaldane commented Oct 14, 2016

Yeah, I think I agree, and personally I think "contagious" is pretty intuitive. Once we make the decision it is set in stone though, so we shouldn't take the decision too lightly.

At least, it is set in stone until the time MaskedArray is overhauled somehow (which a fair number of people want to happen soon).

In any case, I think we should write to the list to ask if contagious is the right word. Would you mind doing that?

What about contagious_mask? (to emphasize it affects the mask)

@eric-wieser
Copy link
Member Author

eric-wieser commented Oct 14, 2016

contagious_mask is a little wordy, but I'm not opposed to it.

I guess the other question here is whether the default should be contagious=True. Arguably it should be False for consistency with the present behaviour of np.ma.sum. And arguably sum and all the other accumulators should be augmented with a similar parameter.

I'm not really sure what "the list" is, so would prefer it if you could do that

@ahaldane
Copy link
Member

All right, I put a message on the mailing list. Archive is here.

@ahaldane
Copy link
Member

All right, the consensus from the list discussion was that we should use propagate_mask=True as the keyword and default.

I don't think there's anything else in the code to change.

Could you also add a note in the release notes, in doc/release/1.12.0-notes.rst, in the "New Features" section?

@eric-wieser
Copy link
Member Author

Pulling from the mailing list:

Just as convolution functions often add zero-padding around an image,
here the mask behavior would allow you to have different borders, eg
[m,m,m,1,1,1,1,m,m,m,m]
using my notation from before.

This raises the question of whether out-of-range values should be considered masked - should mode='full', propagate_mask=True pad the result with masks at the start and end?

@eric-wieser
Copy link
Member Author

Also, is numpy/ma/API_CHANGES.txt#New features (non exhaustive list) relevant any more?

@ahaldane
Copy link
Member

Hmm. I don't like that it would mean np.convolve([1,1,1],[1,1,1,1]) would give a different result from np.ma.convolve([1,1,1],[1,1,1,1]).

I hadn't noticed numpy/ma/API_CHANGES.txt, and it hasn't been updated in 3 years. It also looks like a summary of changes from a past MaskedArray overhaul, rather than an ongoing list of changes, so I would leave it alone.

@eric-wieser
Copy link
Member Author

But it would mean that np.convolve([1,1,1],[1,1,1,1]) gives the same result as np.convolve([1,1,1],[1,1,1,1], propagate_mask=False), which is a potential argument for keeping the default as it was

@eric-wieser
Copy link
Member Author

Darn, guess I'm rebasing now

@ahaldane
Copy link
Member

I'm fairly convinced that the "True" default is the right one, because as we saw on the list it's not totally clear in which cases "False" actually makes sense. So my instinct is to focus on the "True" case so it works as sensibly as possible.

Also, I just noticed a bug, that you can't access v.shape or a.shape if they aren't ndarrays (eg in np.ma.convolve([1,1,1],[1,1,1,1])). You probably need to wrap a and v with asanyarray at the start of the function.

@eric-wieser
Copy link
Member Author

eric-wieser commented Oct 19, 2016

Wouldn't np.shape(a) be better? Or does that defer to np.asarray anyway? Either way, committed, and added some tests.

@eric-wieser
Copy link
Member Author

Unrelatedly, does numpy have a nanconvolve, and should it?

@eric-wieser
Copy link
Member Author

Ok, this PR should be ready for review - docs updated, argument name fixed, and the bug you mentioned earlier is fixed and tested.

@ahaldane ahaldane merged commit 8a9b04c into numpy:master Oct 20, 2016
@ahaldane
Copy link
Member

np.shape is fine too. It possibly might be a little slower since you will effectively be converting to an ndarray twice (since you convert v later anyway), but that shouldn't matter.

So I'm going to merge. Thanks @eric-wieser !

@homu homu mentioned this pull request Oct 20, 2016
@eric-wieser eric-wieser deleted the np-ma-convolve branch October 20, 2016 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants