Thanks to visit codestin.com
Credit goes to github.com

Skip to content

DOC, MAINT: Enforce np.ndarray arg for np.put and np.place #7000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 15, 2016
Merged

DOC, MAINT: Enforce np.ndarray arg for np.put and np.place #7000

merged 1 commit into from
Jan 15, 2016

Conversation

gfyoung
Copy link
Contributor

@gfyoung gfyoung commented Jan 12, 2016

np.put and np.place do something only when the first argument is an instance of np.ndarray. These changes will cause a TypeError to be thrown in either function should that requirement not be satisfied, which would be more useful to see when debugging.

@gfyoung
Copy link
Contributor Author

gfyoung commented Jan 12, 2016

Could someone explain to me what the purpose of the failing test (test_mem_insert) is? I can't seem to find the relevant ticket for this test. It seems like what I did is backwards-incompatible, but I just don't know how. I triednp.place(1, 1, 1) on the most recently released version of numpy, but I don't know what it is doing.

@@ -445,6 +445,10 @@ def put(a, ind, v, mode='raise'):
array([ 0, 1, 2, 3, -5])

"""
if not isinstance(a, np.ndarray):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My preference for ducktyping would suggest using a try/except? I.e.,

try:
    return a.put(ind, v, mode)
except AttributeError:
    if not isinstance(a, np.ndarray):
        raise TypeError(...)
    raise

This should also be a little faster for the most common case, where a is in fact an ndarray.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm...I'm a little on the fence on this one. While I do agree it should be faster in the most common case (though how much faster I'm not really sure), coupling the type checking with the function call makes us dependent on an AttributeError when calling the C API should a not be an ndarray. Should the API change, say to something like what is done with np.place, that try...except would become moot.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest using the _wrapit functionality here, as used elsewhere in this file. _wrapit uses duck typing, similarly to @mhvk's suggestion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shoyer : IMHO using _wrapit does not help here because I want to enforce that an ndarray must be passed in. Because np.put modifies the ndarray in-place, passing in a non-ndarray object will do nothing (the array_like object itself is not changed because it has to be cast to an ndarray object first) and will not raise any errors / warnings, which is what I am trying to avoid. This was the issue that I saw with np.place.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, good point. It's still not entirely clear to me why the current duck typing solution dies not suffice, though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I would suggest:

try:
    put = a.put
except AttributeError:
    raise TypeError
return put(...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shoyer : Ah, fair enough. I do see lots of similar wrappings in this file, so I'll follow suit then. By the way, might you happen to know what that test_mem_insert is all about? Even with your suggestion, that test will fail, but I'm not sure what it is or what the purpose of that test was. I'm tempted to remove it, but it was located in the test_regression.py file, so I'm hesitant to do so until I have more information.

@mhvk : Thanks for your input as well!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like @shoyer's solution best too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@mhvk
Copy link
Contributor

mhvk commented Jan 13, 2016

@Gyoung - the relevant PR is #1170 -- there apparently was a memory leak, although to be honest I don't see how this test would actually be useful to determine that the leak had returned.

@gfyoung
Copy link
Contributor Author

gfyoung commented Jan 13, 2016

@mhvk : I think I'll take the test out. I'm doing a complete revamp of np.place in another PR that is very similar to np.putmask, which as far as I can tell has had no memory leak issues.

@gfyoung
Copy link
Contributor Author

gfyoung commented Jan 13, 2016

Changes are in, and Travis + Appveyor are both happy. Should be good to merge if there is nothing else.

@@ -1801,6 +1801,10 @@ def place(arr, mask, vals):
[44, 55, 44]])

"""
if not isinstance(arr, np.ndarray):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use the same duck-typing solution as above, with the try/except clause?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels little more awkward in this case because _insert is not an ndarray method. I suppose you could do something like the following:

try:
   put = arr.put
except AttributeError:
   raise TypeError(...)
return _insert(arr, mask, vals)

However, it seems like more contrived compared to the others since we're relying on a method that has nothing to do with what is being called.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What error message do you get when place is not valid? What sort of duck typing does it currently support, if any?

It might be worth updating _insert instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shoyer : See #7003, which does a complete revamp of arr_insert (that is tied to _insert) and does the type checking. In the case of np.place, this PR should serve as a stopgap measure until #7003 can be merged.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shoyer : Sorry, I didn't answer the remainder of your question; I only had seen the "It might be worth" part on my phone when I was reading it. place doesn't actually do any duck-typing if you pass in a non-ndarray to it. Rather, it does nothing, which is why I added that isinstance check to the code. However, #7003 will embed that check into the C code, which is preferable. However, I wouldn't mind merging the isinstance check into the codebase FTTB.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, agreed!

@shoyer
Copy link
Member

shoyer commented Jan 13, 2016

It looks like we still need a unit test for the put fix?

@gfyoung
Copy link
Contributor Author

gfyoung commented Jan 13, 2016

@shoyer : Oh, yes, of course! Let me do that quickly.

@gfyoung
Copy link
Contributor Author

gfyoung commented Jan 13, 2016

@shoyer : Actually, quick question : where should that test go? It seems that there are no tests for the fromnumeric.py file IINM.

@shoyer
Copy link
Member

shoyer commented Jan 13, 2016

Looks like test_place is in test_function_base.py.

On Wed, Jan 13, 2016 at 11:45 AM, gfyoung [email protected] wrote:

@shoyer https://github.com/shoyer : Actually, quick question : where
should that test go? It seems that there are no tests for the
fromnumeric.py file IINM.


Reply to this email directly or view it on GitHub
#7000 (comment).

@gfyoung
Copy link
Contributor Author

gfyoung commented Jan 13, 2016

@shoyer : I'm referring to np.put, not np.place. That was why I had said I couldn't find any file that tested the fromnumeric.py module.

@shoyer
Copy link
Member

shoyer commented Jan 13, 2016

Looks like that is in test_multiarray.py.

On Wed, Jan 13, 2016 at 11:52 AM, gfyoung [email protected] wrote:

@shoyer https://github.com/shoyer : I'm referring to np.put, not
np.place.


Reply to this email directly or view it on GitHub
#7000 (comment).

@gfyoung
Copy link
Contributor Author

gfyoung commented Jan 13, 2016

It's for the array method itself, not np.put.

@shoyer
Copy link
Member

shoyer commented Jan 13, 2016

Hmm. My search for np.put in test_* turns up nothing. So no wonder this was
broken! I guess you'll have to make a judgment call here on where these
belong... (I'm also pretty new to the numpy codebase so I don't have
immediate guidance)

On Wed, Jan 13, 2016 at 11:54 AM, gfyoung [email protected] wrote:

It's for the array method itself, not np.put.


Reply to this email directly or view it on GitHub
#7000 (comment).

@gfyoung
Copy link
Contributor Author

gfyoung commented Jan 13, 2016

Oh, okay! I'm relatively new as well (I started contributing last month), but I think I'm going to go for creating a new file called test_fromnumeric.py for organizational reasons.

@gfyoung
Copy link
Contributor Author

gfyoung commented Jan 13, 2016

@shoyer : I'm going to create a new file test_fromnumeric.py, and it will only contain a test for checking that TypeError. However, as a follow-up, I will develop this testing module to include tests for the other methods, where I can just port tests for the relevant array methods.

np.put and np.place do something only when the first argument
is an instance of np.ndarray. These changes will cause a TypeError
to be thrown in either function should that requirement not be
satisfied.
@gfyoung
Copy link
Contributor Author

gfyoung commented Jan 14, 2016

Test added and passing on both Travis and Appveyor. Should be good to merge if there is nothing else.

@shoyer
Copy link
Member

shoyer commented Jan 15, 2016

OK, I'm pretty happy with this. Going to merge unless someone else raises objections....

@shoyer
Copy link
Member

shoyer commented Jan 15, 2016

I'm not 100% sure that test_fromnumeric.py is a good idea but it's easy enough to remove later if someone cares.

shoyer added a commit that referenced this pull request Jan 15, 2016
DOC, MAINT: Enforce np.ndarray arg for np.put and np.place
@shoyer shoyer merged commit 54224f4 into numpy:master Jan 15, 2016
@shoyer
Copy link
Member

shoyer commented Jan 15, 2016

Thanks @gfyoung!

@gfyoung gfyoung deleted the ndarray_arg_enforce branch January 15, 2016 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants