`dates` classes and functions support `tz` both as string and `tzinfo` #22196

oscargus · 2022-01-11T10:06:31Z

PR Summary

Edit:
There were inconsistent use/documentation of the tz parameters in dates, where the documentation stated string in some cases, although only tzinfo was working. Now, both approaches do work (and there is even an error if the rcParams-provided string is not a valid time zone). See below for discussion.

Original purpose: Added a bunch of tests for dates.py to increase coverage.

In addition: Found an issue with datestr2num where passing a list and a default datetime object gives an error in current main.

import datetime
import matplotlib.dates as mdates

dt = datetime.date(year=2022, month=1, day=10)
mdates.datestr2num(['2022-01', '2022-02'], default=dt)

results in

Traceback (most recent call last):

  File "/tmp/ipykernel_117358/3002628473.py", line 1, in <module>
    mdates.datestr2num(['2022-01', '2022-02'], default=dt)

  File "/home/oscar/dev/matplotlib/lib/matplotlib/dates.py", line 402, in datestr2num
    return date2num(_dateutil_parser_parse_np_vectorized(d))

  File "/usr/lib/python3.10/site-packages/numpy/lib/function_base.py", line 2163, in __call__
    return self._vectorize_call(func=func, args=vargs)

  File "/usr/lib/python3.10/site-packages/numpy/lib/function_base.py", line 2241, in _vectorize_call
    ufunc, otypes = self._get_ufunc_and_otypes(func=func, args=args)

  File "/usr/lib/python3.10/site-packages/numpy/lib/function_base.py", line 2201, in _get_ufunc_and_otypes
    outputs = func(*inputs)

  File "/usr/lib/python3.10/site-packages/dateutil/parser/_parser.py", line 1368, in parse
    return DEFAULTPARSER.parse(timestr, **kwargs)

  File "/usr/lib/python3.10/site-packages/dateutil/parser/_parser.py", line 640, in parse
    res, skipped_tokens = self._parse(timestr, **kwargs)

  File "/usr/lib/python3.10/site-packages/dateutil/parser/_parser.py", line 719, in _parse
    l = _timelex.split(timestr)         # Splits the timestr into tokens

  File "/usr/lib/python3.10/site-packages/dateutil/parser/_parser.py", line 201, in split
    return list(cls(s))

  File "/usr/lib/python3.10/site-packages/dateutil/parser/_parser.py", line 69, in __init__
    raise TypeError('Parser must be a string or character stream, not '

TypeError: Parser must be a string or character stream, not date

Do you use release notes for this type of minor bug fixes?

PR Checklist

Tests and Styling

Has pytest style unit tests (and pytest passes).
Is Flake 8 compliant (install flake8-docstrings and run flake8 --docstring-convention=all).

Documentation

[N/A] New features are documented, with examples if plot related.
[N/A] New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
[N/A] API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).
Documentation is sphinx and numpydoc compliant (the docs should build without error).

lib/matplotlib/tests/test_dates.py

doc/users/next_whats_new/datestr2num_list_with_default_date.rst

oscargus · 2022-01-12T12:01:41Z

I have found some more issues, especially the tz arguments which most of the time should be a tzinfo subclass, but often are str. For example:

matplotlib/lib/matplotlib/dates.py

Lines 499 to 510 in 2f81a8b

    
           def num2date(x, tz=None): 
        
               """ 
        
               Convert Matplotlib dates to `~datetime.datetime` objects. 
        
               Parameters 
        
               ---------- 
        
               x : float or sequence of floats 
        
                   Number of days (fraction part represents hours, minutes, seconds) 
        
                   since the epoch.  See `.get_epoch` for the 
        
                   epoch, which can be changed by :rc:`date.epoch` or `.set_epoch`. 
        
               tz : str, default: :rc:`timezone` 
        
                   Timezone of *x*.

but

In [24]: num2date(12093.0, tz="Iceland")
Traceback (most recent call last):

  File "/tmp/ipykernel_67630/1708974312.py", line 1, in <module>
    num2date(12093.0, tz="Iceland")

  File "/home/oscar/dev/matplotlib/lib/matplotlib/dates.py", line 530, in num2date
    return _from_ordinalf_np_vectorized(x, tz).tolist()

  File "/usr/lib/python3.10/site-packages/numpy/lib/function_base.py", line 2163, in __call__
    return self._vectorize_call(func=func, args=vargs)

  File "/usr/lib/python3.10/site-packages/numpy/lib/function_base.py", line 2246, in _vectorize_call
    outputs = ufunc(*inputs)

  File "/home/oscar/dev/matplotlib/lib/matplotlib/dates.py", line 359, in _from_ordinalf
    dt = dt.astimezone(tz)

TypeError: tzinfo argument must be None or of a tzinfo subclass, not type 'str'

This is somewhat consistent throughout the whole dates.py. Typically it says str, but if none provided a tzinfo object is used.

Two possible solutions:

change documentation to say tzinfo (and check that a str is not provided)
check type and get the tzinfo from the str

Any preferences? I'm leaning towards 2 to not break the documentation (although it didn't work as stated).

oscargus · 2022-01-12T12:08:03Z

Another example:

matplotlib/lib/matplotlib/dates.py

Lines 657 to 658 in 2f81a8b

    
               tz : str, optional 
        
                   Passed to `.dates.date2num`.

Stored as

matplotlib/lib/matplotlib/dates.py

Line 721 in 2f81a8b

self._tz = tz

Used as

matplotlib/lib/matplotlib/dates.py

Line 774 in 2f81a8b

formatter = DateFormatter(self.defaultfmt, self._tz,

But

matplotlib/lib/matplotlib/dates.py

Line 622 in 2f81a8b

tz : `datetime.tzinfo`, default: :rc:`timezone`

jklymak · 2022-01-12T12:40:36Z

OK, first the doc for concise formatter is wrong - it should say as "in num2date", not date2num. num2date takes only datetime.tzinfo objects right now.

num2date calls from_ordinalf which uses tz in datetime.astimezone. So tz must be in whatever formats datetime.astimezone accepts.

So, what does astimezone accept? Well, it accepts tzinfo objects, not strings. If we want to allow strings, then we need to decide on a module to translate strings to tzinfo. The main library can do that, but only >py3.9. dateutil can do that using dateutil.tz.gettz, which we call in a couple of places. So if we want to re-instate string support, we need to carefully define how the string is used.

Given the ambiguity in timezone names and what package can translate them, my tendency would be to tell users to supply tzinfo objects, and point them towards both dateutils, the main library, and pytz, rather than try to parse names and then get a bunch of bug reports that we can't parse the names.

oscargus · 2022-01-12T12:46:15Z

it should say as "in num2date", not date2num.

Yes, I realized that.

In the meanwhile I happened to implement the second approach: support both str and tzinfo. But I can change it to only support tzinfo and change the documentation to reflect that. (Right now I changed it to say both.)

lib/matplotlib/dates.py

oscargus · 2022-01-12T12:55:33Z

Looking at the diff there were three classes/functions that had an incorrect documentation w.r.t tz format:
DateFormatter, ConciseDateFormatter, and num2date.

Just to confirm: it is OK to "break" this and only support tzinfo?

(I can of course do the try/except for the conversion, in that way it is made more flexible in some sense and less controversial to just "change" the argument format, although it didn't work as stated...)

(Anyway, I need to focus on creating an exam, so there will be time to agree on the best solution. Just took a small break...)

jklymak

Suggest being even more explicit about what strings are available. However, I'm not sure if datetutil will resolve in the numpydoc lookup?

lib/matplotlib/dates.py

oscargus · 2022-01-12T15:22:24Z

However, I'm not sure if datetutil will resolve in the numpydoc lookup?

Check For example RRuleLocator in the list of date tickers here: https://matplotlib.org/stable/api/dates_api.html

(I noted that rrulewrapper is not in the documentation, although it is linked there. Should I add it, remove the linking, or just leave it?)

jklymak · 2022-01-12T15:24:58Z

Check For example RRuleLocator in the list of date tickers here: https://matplotlib.org/stable/api/dates_api.html

Great...

(I noted that rrulewrapper is not in the documentation, although it is linked there. Should I add it, remove the linking, or just leave it?)

I think its strange to link it but not have it int he docs. Thanks for looking so carefully!

oscargus · 2022-01-12T16:07:18Z

think its strange to link it but not have it int he docs.

I do not unfortunately understand why it does not show up...

A guess is that everything in __all__ is documented by :members: since rrulewrapper is not there, but rrule (from dateutil) is, so it is documented. But the solution is not obvious to me.

oscargus · 2022-01-12T16:25:48Z

Some more insights, so I changed this and all other missing references in dates.py, so hopefully that doc-page is free from it.

dstansby

Some minor suggestions, but this looks 👍

lib/matplotlib/dates.py

…-argument

tacaswell · 2022-01-27T04:56:36Z

lib/matplotlib/dates.py

@@ -1,6 +1,6 @@
 """
 Matplotlib provides sophisticated date plotting capabilities, standing on the
-shoulders of python :mod:`datetime` and the add-on module :mod:`dateutil`.
+shoulders of python :mod:`datetime` and the add-on module dateutil_.


Could we add the dateutil intersphinx and leave the markup?

I do not fully understand exactly how this works, but it was a way to get a proper link. Maybe there are better ways.

tacaswell · 2022-01-27T05:05:44Z

lib/matplotlib/dates.py

+string.  If you want to use a different timezone, pass the *tz* keyword
+argument of `num2date` to any date tickers or locators you create.  This can
+be either a `datetime.tzinfo` instance or a string with the timezone name that
+can be parsed by `~dateutil.tz.gettz`.


Suggested change

can be parsed by `~dateutil.tz.gettz`.

can be parsed by `dateutil.tz.gettz`.

This is outside of us so leave the full name?

I do not remember the rationale for it, but I see the point. Will revert it (if it works).

tacaswell · 2022-01-27T05:11:46Z

I erred on the side of merging this despite my two minor documentation quibbles.

QuLogic · 2022-01-27T05:11:15Z

lib/matplotlib/dates.py

+string.  If you want to use a different timezone, pass the *tz* keyword
+argument of `num2date` to any date tickers or locators you create.  This can


This sentence no longer says the same thing. Previously it said to pass tz to num2date or the other options. Now it says to pass num2date to the other options.

I do not read it like that. To me it sounds more like "use the same tz argument that you passed to num2date". But I agree that it probably can be made clearer (obviously, as you read it differently).

(Btw, was suggested here: #22196 (comment) )

oscargus · 2022-01-27T10:07:56Z

I can create a follow-up.

oscargus force-pushed the datestest branch 2 times, most recently from 720a681 to 280e7d1 Compare January 11, 2022 11:05

dstansby reviewed Jan 11, 2022

View reviewed changes

lib/matplotlib/tests/test_dates.py Outdated Show resolved Hide resolved

lib/matplotlib/tests/test_dates.py Outdated Show resolved Hide resolved

lib/matplotlib/tests/test_dates.py Outdated Show resolved Hide resolved

QuLogic reviewed Jan 12, 2022

View reviewed changes

doc/users/next_whats_new/datestr2num_list_with_default_date.rst Outdated Show resolved Hide resolved

oscargus force-pushed the datestest branch from 280e7d1 to 98d6c1d Compare January 12, 2022 12:43

oscargus force-pushed the datestest branch from 98d6c1d to da87752 Compare January 12, 2022 12:48

oscargus marked this pull request as draft January 12, 2022 12:49

jklymak reviewed Jan 12, 2022

View reviewed changes

lib/matplotlib/dates.py Outdated Show resolved Hide resolved

oscargus force-pushed the datestest branch from da87752 to 7c95b79 Compare January 12, 2022 14:21

oscargus changed the title ~~Increased coverage for dates and fixed bug~~ dates classes and functions support tz both as string and tzinfo Jan 12, 2022

oscargus force-pushed the datestest branch 2 times, most recently from ad9eac5 to f93514f Compare January 12, 2022 14:33

jklymak reviewed Jan 12, 2022

View reviewed changes

oscargus force-pushed the datestest branch 2 times, most recently from d51b377 to 8e7c2b1 Compare January 12, 2022 15:13

oscargus marked this pull request as ready for review January 12, 2022 15:25

oscargus force-pushed the datestest branch from 8e7c2b1 to afb14cb Compare January 12, 2022 16:00

oscargus force-pushed the datestest branch from b1fd4c1 to a915bfa Compare January 12, 2022 16:25

dstansby approved these changes Jan 12, 2022

View reviewed changes

lib/matplotlib/dates.py Outdated Show resolved Hide resolved

lib/matplotlib/dates.py Outdated Show resolved Hide resolved

Increased coverage for dates and fixed bug

ff79835

oscargus force-pushed the datestest branch from a915bfa to 1827cda Compare January 16, 2022 22:11

oscargus force-pushed the datestest branch from 1827cda to a7282ac Compare January 17, 2022 08:40

jklymak reviewed Jan 17, 2022

View reviewed changes

lib/matplotlib/dates.py Outdated Show resolved Hide resolved

oscargus force-pushed the datestest branch from a7282ac to 2806576 Compare January 17, 2022 09:43

All classes and methods in dates support both string and tzinfo as tz…

1158778

…-argument

oscargus force-pushed the datestest branch from 2806576 to 1158778 Compare January 17, 2022 09:45

jklymak approved these changes Jan 17, 2022

View reviewed changes

tacaswell added this to the v3.6.0 milestone Jan 27, 2022

tacaswell reviewed Jan 27, 2022

View reviewed changes

tacaswell merged commit cc6854c into matplotlib:main Jan 27, 2022

QuLogic reviewed Jan 27, 2022

View reviewed changes

	can be parsed by `~dateutil.tz.gettz`.
	can be parsed by `dateutil.tz.gettz`.

		string. If you want to use a different timezone, pass the tz keyword
		argument of `num2date` to any date tickers or locators you create. This can

Uh oh!

dates classes and functions support tz both as string and tzinfo #22196

dates classes and functions support tz both as string and tzinfo #22196

Uh oh!

Conversation

oscargus commented Jan 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

oscargus commented Jan 12, 2022

Uh oh!

oscargus commented Jan 12, 2022

Uh oh!

jklymak commented Jan 12, 2022

Uh oh!

oscargus commented Jan 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

oscargus commented Jan 12, 2022

Uh oh!

jklymak left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

oscargus commented Jan 12, 2022

Uh oh!

jklymak commented Jan 12, 2022

Uh oh!

oscargus commented Jan 12, 2022

Uh oh!

oscargus commented Jan 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dstansby left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tacaswell Jan 27, 2022

Choose a reason for hiding this comment

Uh oh!

oscargus Jan 27, 2022

Choose a reason for hiding this comment

Uh oh!

tacaswell Jan 27, 2022

Choose a reason for hiding this comment

Uh oh!

oscargus Jan 27, 2022

Choose a reason for hiding this comment

Uh oh!

tacaswell commented Jan 27, 2022

Uh oh!

QuLogic Jan 27, 2022

Choose a reason for hiding this comment

Uh oh!

oscargus Jan 27, 2022

Choose a reason for hiding this comment

Uh oh!

oscargus commented Jan 27, 2022

Uh oh!

Uh oh!

`dates` classes and functions support `tz` both as string and `tzinfo` #22196

`dates` classes and functions support `tz` both as string and `tzinfo` #22196

oscargus commented Jan 11, 2022 •

edited

Loading

oscargus commented Jan 12, 2022 •

edited

Loading

oscargus commented Jan 12, 2022 •

edited

Loading