Thanks to visit codestin.com
Credit goes to github.com

Skip to content

MAINT: add overlap checks to choose, take, put, putmask #13182

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 27, 2019

Conversation

ahaldane
Copy link
Member

Fixes #9293, #6272

This adds the overlap checks as suggested in #6272 to np.choose, np.take, np.put, and np.putmask.

It is arguable whether or not this "fixes" #6272, since there are potentially more places overlap checks are needed, in particular in inplace operations. But at the very least this PR completes the checks listed by @seberg in #6272.

(Incidentally, I don't like that we always make a copy for mode='raise' in np.take/put. It defeats the point of having an out argument, which is to get no-copy performance!)

@mattip
Copy link
Member

mattip commented Mar 24, 2019

LGTM.
Is out so rarely used with overlap that this issue was never reported by a user?

Incidentally, I don't like that we always make a copy for mode='raise' in np.take/put. It defeats the point of having an out argument, which is to get no-copy performance!

Is there an open ENH issue for this?

@seberg
Copy link
Member

seberg commented Mar 24, 2019

The problem with "raise" is that it leaves the output array mutated when an error occurs (but not on the first index). So changing it could be seen as introducing a small issue itself.

Advanced indexing assignment currently has similar issues but will simply allow for such mutation, which is not really ideal in a sense (many paths will check the index first, but not all do IIRC, have to check).

Copy link
Member

@seberg seberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me as well. Could be merged as is, but maybe we should add a short release note in any case.

@ahaldane ahaldane force-pushed the overlap_check_choose_take branch from 18a23bf to aa1f3b4 Compare March 24, 2019 15:13
@ahaldane
Copy link
Member Author

ahaldane commented Mar 24, 2019

Fixed. I also tweaked the np.take and np.choose docstring to clarify that out gives no performance benefit when clip='raise'.

Also, I notice in np.put that the logic for clip='raise' buffering is a no-op. That is, it looks like the intention was to make np.put buffer if clip='raise', but the clause was inside an if-block that only runs if the input was non-contiguous, so in effect np.put usually runs unbuffered and will clobber the input even on exception.

That code should be cleaned up. My suggestion is to just remove the clip='raise' test and run unbuffered, even though that's a little inconsistent what np.take and np.choose do. The major alternative is to make it buffer for mode='raise' but add a note in the docstring about it. Thoughts?

@ahaldane ahaldane force-pushed the overlap_check_choose_take branch from aa1f3b4 to a2df7b6 Compare March 24, 2019 15:37
@seberg
Copy link
Member

seberg commented Mar 24, 2019

Hmm, can't say I care much which way we go on that function, both sounds fine to me (or even just leaving it, heh). For similarity with those other functions, I think I would tend slightly towards doing the copy, but contiguous input is probably the most common, so going the other way should actually be the smaller change.
But maybe my enthusiasm on these functions is a bit low anyway ;), I always think of them as solving what indexing already solves.

@ahaldane ahaldane force-pushed the overlap_check_choose_take branch from a2df7b6 to a9d3ee9 Compare March 24, 2019 17:58
@seberg
Copy link
Member

seberg commented Mar 25, 2019

Thanks, I see, you fixed the documentation, sounds good to me. If nobody else has a comment, I will merge it tomorrow or so.

@seberg
Copy link
Member

seberg commented Mar 27, 2019

I guess this was ready to go in. Thanks Allan!

@seberg seberg merged commit e6147b9 into numpy:master Mar 27, 2019
@mattip mattip mentioned this pull request Aug 18, 2019
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Wrong output in take function
3 participants