Thanks to visit codestin.com
Credit goes to github.com

Skip to content

FIX: datetime64 now recognized if in a list #12277

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Oct 7, 2018
Merged

FIX: datetime64 now recognized if in a list #12277

merged 2 commits into from
Oct 7, 2018

Conversation

jklymak
Copy link
Member

@jklymak jklymak commented Sep 25, 2018 •
edited
Loading

PR Summary

ax.errorbar sends a list of x values to line, but the new datetime64 converter was not being called on a bare list of datetime64 objects (they had to be packed in an ndarray). This now checks the list elements too...

Closes #12271

    import numpy as np
    import matplotlib.dates as mdates

    dt = [np.datetime64('2000-01-01'), np.datetime64('2001-01-01')]
    dn = mdates.date2num(dt)
    assert np.array_equal(dn, [ 730120.,  730486.])

Fails on master, and passes w/ this PR.

PR Checklist

  • Has Pytest style unit tests
  • Code is Flake 8 compliant

@jklymak jklymak added this to the v2.2.4 milestone Sep 25, 2018
@dstansby
Copy link
Member

Hmm, I wonder if converting lists of unitized data is something we need to fix more generally; I had to fix it manually in the Astropy converters too: astropy/astropy#7037

@jklymak
Copy link
Member Author

jklymak commented Sep 25, 2018

I'm not sure we can make all the lists ndarrays. OTOH, we can decide to not make our own functions return lists. The issue here is that this internal helper returns a list, whereas it'd be nice if it returned an array.

matplotlib/lib/matplotlib/axes/_axes.py

Lines 3144 to 3153 in b329c77

def xywhere(xs, ys, mask):
"""
return xs[mask], ys[mask] where mask is True but xs and
ys are not arrays
"""
assert len(xs) == len(ys)
assert len(xs) == len(mask)
xs = [thisx for thisx, b in zip(xs, mask) if b]
ys = [thisy for thisy, b in zip(ys, mask) if b]
return xs, ys

@dstansby
Copy link
Member

Yeah, I was wondering if the unit machinery should be able to cope with lists of units (which as you say are often used internally), independently of particular ConversionInterface implementations .

@jklymak
Copy link
Member Author

jklymak commented Sep 25, 2018

Yeah, I was wondering if the unit machinery should be able to cope with lists of units (which as you say are often used internally), independently of particular ConversionInterface implementations .

I don't know - I don't see an easy way to do that, except right in convert_units, which seems risky to me. Maybe some converters out there do something diffrent w/ a list than an ndarray?

@jklymak jklymak force-pushed the fix-datetime64-not-recognized-in-list branch 2 times, most recently from b4f0492 to 8ac27ac Compare September 26, 2018 01:44
@jklymak
Copy link
Member Author

jklymak commented Sep 26, 2018

@dstansby Thinkng about this more, I think this will be really hard. Suppose we pass a complicated structure to a converter. I don't think we should stop the ConversionInterface from accepting that structure if we can't premassage it into an ndarray. Thats what the converter is supposed to do.

Our current difficulty is that we accept a lot of ways of packing data by default. I'd argue we could be more strict about this. But in the case of #12271 the form that was being passed worked for datetime so I felt it should work for datetime64. Worse, it was being passed to the date ConversionInterface, so the latch for what interface to use was different than the logic to decide how to decode it. So that was a definite bug. But in general, the units registry should direct the right "types" to the right ConversionInterface...

tacaswell
tacaswell reviewed Sep 29, 2018
View reviewed changes
lib/matplotlib/dates.py Outdated
else:
d = np.asarray(d)
if (isinstance(d, np.ndarray) and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first part of this will always be true here, I think we only need the second part of the if statement.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree.... Fixed...

tacaswell
tacaswell reviewed Sep 29, 2018
View reviewed changes
lib/matplotlib/dates.py
if not np.iterable(d):
if (isinstance(d, np.datetime64) or (isinstance(d, np.ndarray) and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that we will ever hit the second half of this conditional as

In [5]: np.iterable(np.arange(5))
Out[5]: True

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

t0 = datetime.datetime(2017, 1, 1, 0, 1, 1)
tnp = np.array(t0, dtype='datetime64[us]')
print(np.iterable(tnp))

returns False (believe it or not)...

@tacaswell tacaswell modified the milestones: v2.2.4, v3.0.x Sep 29, 2018
@tacaswell
Copy link
Member

👍 in principle, left a few comments on the conditionals, but not going to block merging over this.

I only think this needs to be backported to 3.0.x as this is something the user can work-around in userspace (by casting to a numpy array) so this does not pass the critical threshold (ex, import failure, segfault).

@jklymak jklymak force-pushed the fix-datetime64-not-recognized-in-list branch from 8ac27ac to 5038bd7 Compare September 30, 2018 00:45
@jklymak jklymak force-pushed the fix-datetime64-not-recognized-in-list branch from 5038bd7 to 86d3257 Compare September 30, 2018 00:45
@jklymak
Copy link
Member Author

jklymak commented Sep 30, 2018

Thanks @tacaswell

@timhoffm timhoffm merged commit 1c17868 into matplotlib:master Oct 7, 2018
@lumberbot-app
Copy link

lumberbot-app bot commented Oct 7, 2018

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
$ git checkout v3.0.x
$ git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
$ git cherry-pick -m1 1c17868f180d36c8ef91423a0443ca6553b8e1ab
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
$ git commit -am 'Backport PR #12277: FIX: datetime64 now recognized if in a list'
  1. Push to a named branch :
git push YOURFORK v3.0.x:auto-backport-of-pr-12277-on-v3.0.x
  1. Create a PR against branch v3.0.x, I would have named this PR:

"Backport PR #12277 on branch v3.0.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to suggest an improvement.

timhoffm pushed a commit to timhoffm/matplotlib that referenced this pull request Oct 7, 2018
@jklymak @timhoffm
Backport PR matplotlib#12277: FIX: datetime64 now recognized if in a …
aa19374 
…list
@timhoffm timhoffm mentioned this pull request Oct 7, 2018
Merged
dstansby added a commit that referenced this pull request Oct 8, 2018
@dstansby
Merge pull request #12432 from timhoffm/auto-backport-of-pr-12277-on-…
96804d6 
…v3.0.x

Backport PR #12277: FIX: datetime64 now recognized if in a list
@jklymak jklymak deleted the fix-datetime64-not-recognized-in-list branch October 22, 2018 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tacaswell tacaswell tacaswell left review comments

@timhoffm timhoffm timhoffm approved these changes

@dstansby dstansby dstansby approved these changes

Assignees
No one assigned
Labels
topic: date handling topic: units and array ducktypes
Projects
None yet
Milestone
v3.0.1
Development

Successfully merging this pull request may close these issues.

error with errorbar with datetime64
4 participants
@jklymak @dstansby @tacaswell @timhoffm