Codestin Search App

eric-wieser · 2019-10-15T22:04:02Z

A lighter-weight alternative to gh-13061, fixes #12927

seberg

Looks good to me (lets see if anything else needs fixing up). As a note for others: This is the minimal fix, since it solves the biggest/new issue while modifying only the strangest part of casting timedelta to ints.

Casting timedelta64 to ints could be disabled more generally at some point. Although that general cast might be best to synchronize to any thoughts about removing the timedelta64 without a unit (base).

seberg · 2019-10-15T23:07:10Z

I just realized this does add a small inconsistency, since we also disable "same_kind" casting as is. And "same_kind" casiting does work for Q->q. We could probably enable/hack the same kind casting without affecting the behaviour w.r.t. to ufuncs though.

seberg · 2019-10-16T00:59:02Z

I pushed a commit to make same kind casting possible np.can_cast("uint64", "timedelta64") (arr.astype() also works, should add tests for both still probably).

I do not quite love this, but I am not sure if there is much of an option if we want consistency. I am also a bit scared that there may be some corner case missing somewhere...

outdated

That is, uint64 <-> timedelta64. Allowing this casting was weird, because Q<->q is forbidden, yet q and m are both int64. This fixes numpygh-12927

eric-wieser · 2019-10-16T08:36:41Z

@seberg: I rebased your commit on top of my local changes to add some more tests. I think your change looks reasonable, but I'd be tempted to pursue it as a separate PR.

seberg · 2019-10-16T17:58:35Z

Oh, nice, you added some tests. I think I am about ±0 on defining the "same kind" casting. It did seem like a slight inconsistency to not allow it, since by definition same kind casting should be allowed to demote the integer before attempting to "safe" cast to the timedelta.

I am OK with living with that inconsistency or splitting it out though.

eric-wieser · 2019-10-23T19:52:14Z

I think this would be fine to put in - if nothing else, np.can_cast('u8', 'm8', casting='same_kind') is True on master, so is probably ok to leave that way.

tylerjereddy

In the recent review of my PR you indicate

Also a little worried that issubclass(np.timedelta64, numbers.Number) appears to still be true.

I don't see a test enforcing that this is no longer the case? Is that just a separate concern then?

This does look a lot less invasive than my hard blocking of all integer -> m8 conversions. Actually, I thought that was motivated by concerns about unit sloppiness, but maybe that discussion evolved in my absence (I'm 700 emails behind on NumPy now, I think :/).

Overall, looks like you and Sebastian are happier with this & you haven't modified any old unit tests, which is always nice for NumPy.

So I think I'm +1, though I'd say at least good to ping @jbrockmendel on whether this will empower the favorable refactoring inside pandas that was discussed in the context of the original/more invasive change. Perhaps a step in that direction, anyway?

tylerjereddy · 2019-10-25T02:56:25Z

+        # issue gh-12927
+        # this only applies to in-place floordiv //=, because the output type
+        # promotes to float which does not fit
+        a = np.array([1, 2], np.int64)


Any reason you changed the original regression test, based on the case originally reported in the issue? It looks like it should have no effect, but then why bother to change it from the original report /my PR?

Perhaps a larger array is more likely to detect problems, I suppose.

I think I wanted to avoid any weird special cases caused by scalar casting rules. I suppose I should have tested both.

tylerjereddy · 2019-10-25T03:22:04Z

+        # Cannot cast safely from unsigned integer of the same size, which
+        # could overflow
+        assert_(np.can_cast('u8', 'm8', casting='same_kind'))
+        assert_(not np.can_cast('u8', 'm8', casting='safe'))


Yes, these tests are very clean/nice. Seems pretty clear that this is type-size, not value-size.

jbrockmendel · 2019-10-25T14:55:47Z

whether this will empower the favorable refactoring inside pandas that was discussed in the context of the original/more invasive change. Perhaps a step in that direction, anyway?

I don't know the numpy code well enough to infer from the diff how this will effect downstream behavior. Two candidates come to mind. Will it affect isinstance(td64, np.integer)? how about td64 + 1?

eric-wieser · 2019-10-25T16:05:52Z

It might affect d64 + np.uint64(1) I suppose, but it won't affect anything that uses the signed int64.

eric-wieser · 2019-10-29T21:52:34Z

@jbrockmendel

All this affects the behavior of outside the existing tests is code like

d = np.datetime64('now')
np.add(d, np.uint64([1, 2]), casting='safe')

td = np.timedelta64(1, 'D')
np.add(td, np.uint64([1, 2]), casting='safe')

that is:

Explicit safe casting
uint64 addition
Non-scalar operands

I doubt this will impact anything

seberg · 2019-11-17T04:54:26Z

Should we go forward with this, I think I am in favor, although it is now pretty close to 1.18 as a small downside (although the real test is usually the rc release in any case).

mattip · 2019-11-24T01:09:09Z

@seberg please check the release notes reflect the second commit.

eric-wieser · 2019-11-24T11:22:54Z

Since this fixes a regression, I think we should go ahead with it.

seberg · 2019-11-25T16:25:16Z

OK, I think we can just squash-merge. The release notes are fine, without the second commit, same_kind also slightly changes, with the second commit, the same kind logic does not change.

mattip · 2019-11-25T19:07:54Z

Thanks @eric-wieser, @seberg

eric-wieser added this to the 1.18.0 release milestone Oct 15, 2019

eric-wieser requested review from seberg and tylerjereddy October 15, 2019 22:04

eric-wieser force-pushed the disable-Q_-_m_casting branch from a322678 to ce2c865 Compare October 15, 2019 22:04

seberg changed the title ~~Forbid Q<->m safe casting~~ API: Forbid Q<->m safe casting Oct 15, 2019

eric-wieser force-pushed the disable-Q_-_m_casting branch from ce2c865 to 63efb2d Compare October 15, 2019 22:13

seberg previously approved these changes Oct 15, 2019

View reviewed changes

eric-wieser force-pushed the disable-Q_-_m_casting branch from 63efb2d to edcd37a Compare October 15, 2019 22:24

eric-wieser added 30 - API component: numpy.datetime64 (and timedelta64) labels Oct 15, 2019

eric-wieser force-pushed the disable-Q_-_m_casting branch from edcd37a to 6d19421 Compare October 15, 2019 22:28

seberg reviewed Oct 15, 2019

View reviewed changes

Comment thread doc/release/upcoming_changes/14718.compatibility.rst Outdated

eric-wieser and others added 2 commits October 16, 2019 09:31

API: Forbid Q<->m safe casting

30b2b58

That is, uint64 <-> timedelta64. Allowing this casting was weird, because Q<->q is forbidden, yet q and m are both int64. This fixes numpygh-12927

Followup: Make "same_kind" casting possible for consistency

ac757b5

eric-wieser force-pushed the disable-Q_-_m_casting branch from 4496add to ac757b5 Compare October 16, 2019 08:36

tylerjereddy reviewed Oct 25, 2019

View reviewed changes

mattip merged commit 4de3997 into numpy:master Nov 25, 2019

seberg mentioned this pull request Nov 25, 2019

MAINT: casting changes for int -> m8 #13061

Closed

Uh oh!

Conversation

eric-wieser commented Oct 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seberg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

seberg commented Oct 15, 2019

Uh oh!

seberg commented Oct 16, 2019

Uh oh!

eric-wieser commented Oct 16, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seberg commented Oct 16, 2019

Uh oh!

eric-wieser commented Oct 23, 2019

Uh oh!

tylerjereddy left a comment

Choose a reason for hiding this comment

Uh oh!

tylerjereddy Oct 25, 2019

Choose a reason for hiding this comment

Uh oh!

eric-wieser Oct 25, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tylerjereddy Oct 25, 2019

Choose a reason for hiding this comment

Uh oh!

jbrockmendel commented Oct 25, 2019

Uh oh!

eric-wieser commented Oct 25, 2019

Uh oh!

eric-wieser commented Oct 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seberg commented Nov 17, 2019

Uh oh!

mattip commented Nov 24, 2019

Uh oh!

eric-wieser commented Nov 24, 2019

Uh oh!

seberg commented Nov 25, 2019

Uh oh!

mattip commented Nov 25, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

eric-wieser commented Oct 15, 2019 •

edited

Loading

eric-wieser commented Oct 16, 2019 •

edited

Loading

eric-wieser commented Oct 29, 2019 •

edited

Loading