Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BUG: fix argmin() with NaTs #6031

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 9, 2015
Merged

BUG: fix argmin() with NaTs #6031

merged 3 commits into from
Jul 9, 2015

Conversation

pitrou
Copy link
Member

@pitrou pitrou commented Jul 1, 2015

np.min() skips NaT values, but np.argmin() wouldn't: since the internal
representation of NaT is smaller than every other datetime64 or
timedelta64 value, np.argmin() would return results inconsistent with np.min().

Closes #6030.

np.min() skips NaT values, but np.argmin() wouldn't: since the internal
representation of NaT is smaller than every other datetime64 or
timedelta64 value, np.argmin() would return results inconsistent with np.min().

Closes numpy#6030.
@jaimefrio
Copy link
Member

Since there is a specific code path for it, could we have a test that checks that the argmin is properly calculated when there is a bunch of leading 'NaT's followed by some actual data? Other than that, LGTM.

Given the change in behavior, perhaps it is worth a quick note in the release notes as well.

@pitrou
Copy link
Member Author

pitrou commented Jul 1, 2015

Funny... that uncovers a bug in np.min() as well:

>>> a = np.array([4, 'nat', 'nat', 2, 3], dtype='m8[s]')
>>> np.min(a)
numpy.timedelta64(2,'s')
>>> a = np.array(['nat', 'nat', 2, 3], dtype='m8[s]')
>>> np.min(a)
numpy.timedelta64('NaT','s')

@pitrou
Copy link
Member Author

pitrou commented Jul 1, 2015

Ok, I've pushed the changes. Should I squash the commits?

@jaimefrio
Copy link
Member

Can we really get rid of the specialized branch for binary reduces? I'd rather keep it, or at least have some timings supporting that the impact to performance is negligible.

@pitrou
Copy link
Member Author

pitrou commented Jul 2, 2015

Well, here are some runtimes before the patch:

>>> a = np.arange(1000).astype('m8[s]')
>>> %timeit np.min(a)
The slowest run took 41.48 times longer than the fastest. This could mean that an intermediate result is being cached 
100000 loops, best of 3: 5.49 µs per loop
>>> a[:a.size // 2] = np.timedelta64('nat', 's')
>>> %timeit np.min(a)
The slowest run took 10.66 times longer than the fastest. This could mean that an intermediate result is being cached 
100000 loops, best of 3: 5.51 µs per loop

And after:

>>> a = np.arange(1000).astype('m8[s]')
>>> %timeit np.min(a)
The slowest run took 13.43 times longer than the fastest. This could mean that an intermediate result is being cached 
100000 loops, best of 3: 6.72 µs per loop
>>> a[:a.size // 2] = np.timedelta64('nat', 's')
>>> %timeit np.min(a)
The slowest run took 10.43 times longer than the fastest. This could mean that an intermediate result is being cached 
100000 loops, best of 3: 5.72 µs per loop

@jaimefrio
Copy link
Member

20% slow down is not negligible, but correctness is sure worth at least that much. And it doesn't seem possible to do things "right" without manually expanding the BINARY_REDUCE_LOOP macro... I'm not 100% sure, but am leaning towards merging it as is.

@charris, @juliantaylor, anyone else: do you have a strong opinion about whether we should squeeze the last drop of performance out of minimum/maximum reduction, or are you OK with reusing the normal elementwise loop for this one case?

@pitrou
Copy link
Member Author

pitrou commented Jul 2, 2015

And it doesn't seem possible to do things "right" without manually expanding the BINARY_REDUCE_LOOP macro

It sounds like that to me as well.

jaimefrio added a commit that referenced this pull request Jul 9, 2015
BUG: fix argmin() with NaTs
@jaimefrio jaimefrio merged commit e9a7534 into numpy:master Jul 9, 2015
@jaimefrio
Copy link
Member

That seems about enough waiting time, in it goes. Thanks Antoine!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants