ENH: support np.datenum64 in dates.py #9779

jklymak · 2017-11-14T08:37:45Z

PR Summary

This allows numpy.datetime64 dates. New and improved!

time = [base - datetime.timedelta(days=x) + datetime.timedelta(seconds=0.900001) for x in range(0, 3)]

timenp = np.array(time, dtype='datetime64[us]')

fig, ax = plt.subplots(tight_layout=True)
ax.plot( time, data)
ax.plot( timenp, data+.1, )

works fine.

PR Checklist

Has Pytest style unit tests
Code is PEP 8 compliant
New features are documented, with examples if plot related
Documentation is sphinx and numpydoc compliant
Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

dstansby · 2017-11-14T14:03:48Z

lib/matplotlib/dates.py

@@ -211,36 +211,58 @@ def _get_rc_timezone():
 WEEKDAYS = (MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY, SATURDAY, SUNDAY)


-def _to_ordinalf(dt):
+def _to_ordinalf(dts):


I think it's probably worth writing a new function called something like _dtime64_to_ordinalf(dts) with the new code to handle np.datetime64s. (for clarity)

dstansby · 2017-11-14T14:10:51Z

xref #9610

efiring · 2017-11-14T16:49:45Z

The normal use case would be with an ndarray of dtype np.datetime64. The conversion of this to matplotlib float datenums should be direct and vectorized--no loop--otherwise the performance will be terrible.

jklymak · 2017-11-14T18:18:27Z

lib/matplotlib/dates.py

@@ -1183,9 +1209,8 @@ def get_locator(self, dmin, dmax):

        locator.set_axis(self.axis)

-        if self.axis is not None:


ooops, I guess I missed a rebase here. Will fix after

Kojoley · 2017-11-14T23:45:56Z

lib/matplotlib/dates.py

+    zerod = False
+    if isinstance(d, np.datetime64):
+        zerod = True
+        d = np.atleast_1d(d)


It runs for me without converting a single item into array and back. This is needed for some older numpy (mine is 1.13.3)?

Hmmm, I didn't even try! You're right..

Kojoley · 2017-11-14T23:51:41Z

lib/matplotlib/dates.py

+        d = np.atleast_1d(d)
+    extra = d - d.astype('datetime64[s]')
+    extra = extra.astype('timedelta64[ns]')
+    t0 = np.datetime64('0001-01-01T00:00:00').astype('datetime64[s]')


t0 = np.datetime64('0001-01-01T00:00:00', 's')

story645 · 2017-11-14T23:55:27Z

lib/matplotlib/tests/test_dates.py

@@ -22,6 +23,52 @@
 import matplotlib.dates as mdates


+def test_date_numpyx():


any reason this isn't done with base/time/timenp being pytest fixtures and the test parameterized or split into two?

Because I can understand this, and I have no idea what a pytest fixture is. Is there an advantage to doing something more obscure than a simple assert?

Only cause you're repeating yourself a lot so it's hard to tell what the specific tests are. Factoring out common features into a fixture means you can make two tests that are uniquely named for the things they're testing.

I'll look into it. OTOH, I find some of the tests aren't very easy to follow. Sometime verbosity is a good thing!

OK, happy to take suggestions, but I find the uses of fixtures in the existing tests makes for pretty obfuscated code.

shrugs can agree on that somewhat, either way it's not something I'd hold the PR up on.

Well, I didn't mean to say I wasn't willing to learn, just that I don't get it... More than happy if you wanted to push a change or give me some pseudo code to get it to work... Thanks!

Sorry for missing your reply (the embedded ones seem to not pop up as notifications), and will totally try to remember your suggestion for the future. I should have been way clearer on what I wanted and given you an example. :/

story645 · 2017-11-14T23:56:15Z

lib/matplotlib/tests/test_dates.py

+    assert np.array_equal(h.get_ydata(orig=False), hnp.get_ydata(orig=False))
+
+
+def test_date_date2num_numpy():


same here, could this be parameterized with ids?

I sympathise with @jklymak re. pytest, (I still have no idea what a fixture is) but find paramaterising fairly easy to understand (https://docs.pytest.org/en/latest/parametrize.html)

Something like this should work:

@pytest.mark.parametrize('t0', [datetime.datetime(2017, 1, 1, 0, 1, 1), [datetime.datetime(2017, 1, 1, 0, 1, 1), datetime.datetime(2017, 1, 1, 1, 1, 1)], [[datetime.datetime(2017, 1, 1, 0, 1, 1), datetime.datetime(2017, 1, 1, 1, 1, 1)], [datetime.datetime(2017, 1, 1, 2, 1, 1), datetime.datetime(2017, 1, 1, 3, 1, 1)]]]) @pytest.mark.parameterize('dtype', ['datetime64[s]', 'datetime64[us]', 'datetime64[ms]']) def test_date_numpyx(t0, dytpe): time = mdates.date2num(t0) tnp = np.array(t0, dtype=dtype) nptime = mdates.date2num(tnp) assert time == nptime

This will run through every possible combination of t0 and dtype, and avoids copy/pasting the actual testing code.

OK, I agree that parametrize works well here (despite pytests odd way of spelling it) Thanks for the example!

dstansby

👍 Looks good to me, thanks a lot for squashing this long running bug! I'm happy to tidy up the tests if you want @jklymak?

tacaswell · 2017-11-15T20:25:13Z

That was easier than I expected...

tacaswell · 2017-11-15T20:25:58Z

Could someone open up a follow-on PR to add a whats_new about this?

jklymak · 2017-11-15T20:27:32Z

Oops sorry I’ll do so tonight and prob add an example as well.

dstansby added the topic: date handling label Nov 14, 2017

dstansby added this to the v2.2 milestone Nov 14, 2017

dstansby reviewed Nov 14, 2017

View reviewed changes

jklymak commented Nov 14, 2017

View reviewed changes

jklymak force-pushed the add-datenum64 branch 4 times, most recently from ec3ec37 to a32aa1b Compare November 14, 2017 18:48

Kojoley reviewed Nov 14, 2017

View reviewed changes

story645 reviewed Nov 14, 2017

View reviewed changes

jklymak force-pushed the add-datenum64 branch from 3d18de8 to 7126295 Compare November 15, 2017 01:18

dstansby approved these changes Nov 15, 2017

View reviewed changes

jorisvandenbossche mentioned this pull request Nov 15, 2017

Deprecate automatically registering matplotlib units (partial revert of 0.21.0) pandas-dev/pandas#18301

Closed

ENH: support np.datenum64 in dates.py

109acc1

jklymak force-pushed the add-datenum64 branch from 7126295 to 109acc1 Compare November 15, 2017 15:17

story645 approved these changes Nov 15, 2017

View reviewed changes

Kojoley merged commit 3ccaf2d into matplotlib:master Nov 15, 2017

story645 added third-party integration topic: units and array ducktypes labels Nov 15, 2017

jklymak mentioned this pull request Nov 16, 2017

DOC: for datetime64 support #9794

Merged

6 tasks

This was referenced Nov 17, 2017

DatetimeIndex plot converter performance pandas-dev/pandas#17479

Closed

provide converters for datetime64 types #9610

Closed

mwaskom mentioned this pull request Nov 20, 2017

TypeError: invalid type promotion with datetime coordinate mwaskom/seaborn#1325

Closed

jklymak mentioned this pull request Nov 28, 2017

dates.date2num no longer works with numpy.datetime64 #2259

Closed

TomAugspurger mentioned this pull request Nov 29, 2017

API: Restore implicit converter registration pandas-dev/pandas#18307

Merged

2 tasks

jklymak mentioned this pull request Jan 3, 2018

improve sub-second datetime plotting and documentation #10076

Merged

6 tasks

QuLogic modified the milestones: needs sorting, v2.2.0 Feb 12, 2018

timhoffm mentioned this pull request Mar 21, 2018

Cannot plot bar chart with DatetimeIndex #9613

Closed

shoyer mentioned this pull request Apr 30, 2018

Plotting upgrades pydata/xarray#2092

Merged

3 tasks

danieljmr mentioned this pull request Jul 12, 2018

problems with timestamps in matplotlib seems related to bug #9779 #11649

Closed

jklymak mentioned this pull request Jul 20, 2020

FIX: remove dependence on matplotlib.dates.epoch2num pandas-dev/pandas#35361

Closed

5 tasks

Illviljan mentioned this pull request Dec 29, 2021

Remove registration of pandas datetime converter in plotting pydata/xarray#6109

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: support np.datenum64 in dates.py #9779

ENH: support np.datenum64 in dates.py #9779

jklymak commented Nov 14, 2017 •

edited

Loading

dstansby Nov 14, 2017 •

edited

Loading

dstansby commented Nov 14, 2017

efiring commented Nov 14, 2017

jklymak Nov 14, 2017

Kojoley Nov 14, 2017

jklymak Nov 15, 2017

Kojoley Nov 14, 2017

story645 Nov 14, 2017

jklymak Nov 15, 2017

story645 Nov 15, 2017

jklymak Nov 15, 2017

jklymak Nov 15, 2017

story645 Nov 15, 2017

jklymak Nov 15, 2017

story645 Nov 20, 2017

story645 Nov 14, 2017

dstansby Nov 15, 2017 •

edited

Loading

jklymak Nov 15, 2017

dstansby left a comment

tacaswell commented Nov 15, 2017

tacaswell commented Nov 15, 2017

jklymak commented Nov 15, 2017

		@@ -1183,9 +1209,8 @@ def get_locator(self, dmin, dmax):

		locator.set_axis(self.axis)

		if self.axis is not None:

		@@ -22,6 +23,52 @@
		import matplotlib.dates as mdates


		def test_date_numpyx():

		assert np.array_equal(h.get_ydata(orig=False), hnp.get_ydata(orig=False))


		def test_date_date2num_numpy():

ENH: support np.datenum64 in dates.py #9779

ENH: support np.datenum64 in dates.py #9779

Conversation

jklymak commented Nov 14, 2017 • edited Loading

PR Summary

PR Checklist

dstansby Nov 14, 2017 • edited Loading

Choose a reason for hiding this comment

dstansby commented Nov 14, 2017

efiring commented Nov 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dstansby Nov 15, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dstansby left a comment

Choose a reason for hiding this comment

tacaswell commented Nov 15, 2017

tacaswell commented Nov 15, 2017

jklymak commented Nov 15, 2017

jklymak commented Nov 14, 2017 •

edited

Loading

dstansby Nov 14, 2017 •

edited

Loading

dstansby Nov 15, 2017 •

edited

Loading