Allow FuncFormatter to take functions with only one field #17288
Conversation
|
You can parse the format string using https://docs.python.org/3/library/string.html#string.Formatter.parse. |
|
@anntzer um, not sure how to apply your suggestion. Is it for testing? |
|
My point is that by passing the user-provided string to Formatter.parse (and some additional machinery around it) you can determine how many/what arguments it takes. |
lib/matplotlib/ticker.py
Outdated
| self.func = func | ||
| if not isinstance(func, types.BuiltinFunctionType): | ||
| nargs = len(inspect.signature(func).parameters) | ||
| elif (func.__name__ == 'format'): |
There was a problem hiding this comment.
isinstance(getattr(func, "__self__", None), str) and getattr(func, "__name__", "") == "format" just to be safe...
There was a problem hiding this comment.
Little bit unclear here. Is the benefit that this will end up falling through to the warning if either case isn't met rather than triggering an error in the loop or if statement?
There was a problem hiding this comment.
sometimes func won't have a name attribute, but will be otherwise fine, so you don't want to error in that case. sometimes some other object will have a method called format, in which case you don't want to go to this branch.
There was a problem hiding this comment.
but I only end up in this branch if it's a builtin, and I don't think there is any other builtin named format.
https://docs.python.org/3/library/functions.html
also I think functions always have names = even lambdas return as the name?
lib/matplotlib/ticker.py
Outdated
| "and may not work as expected") | ||
|
|
||
| if nargs == 1: | ||
| self.func = lambda x, pos: func(x) |
There was a problem hiding this comment.
that won't be picklable; you need to store nargs as a private attribute and switch over it in __call__.
There was a problem hiding this comment.
will do but slightly confused why - are there more tests I should be writing?
There was a problem hiding this comment.
depends on how much work you want to do, heh.
There was a problem hiding this comment.
🤷 if it's important will do, but would like to know why.
…it concat on message
lib/matplotlib/ticker.py
Outdated
| """ | ||
| def __init__(self, func): | ||
| self.func = func | ||
|
|
||
| if not isinstance(func, types.BuiltinFunctionType): |
There was a problem hiding this comment.
func is part of the public API, which means it can be changed at runtime. So you can't count on the number of arguments being the same when this is actually called. func probably needs to be changed into a property which recalculates the number of arguments in the setter.
lib/matplotlib/ticker.py
Outdated
| """ | ||
| def __init__(self, func): | ||
| self.func = func | ||
|
|
||
| if not isinstance(func, types.BuiltinFunctionType): | ||
| self.nargs = len(inspect.signature(func).parameters) | ||
| elif (isinstance(getattr(func, "__self__"), str) and |
There was a problem hiding this comment.
Why is this needed at all? '{x}'.format(x='eggs', bar='spam') works fine, so you can just always pass both arguments.
There was a problem hiding this comment.
inspect.signature crashes on builtins, so I need to special case .format (used in lots of the polar examples, my guess is still floating in the wild too.) You're right that I probably don't need to count the arguments sent to format, but since FuncFormat only operates on two it's probably worth letting users know that. Should it be a warning instead?
lib/matplotlib/ticker.py
Outdated
| self.nargs = 2 | ||
| cbook._warn_external(f"{func.__name__} is not supported " | ||
| "and may not work as expected") | ||
| if self.nargs not in [1, 2]: |
There was a problem hiding this comment.
I think best practices are to always use a set for inclusion testing (in {1, 2}). The performance benefit isn't a big deal here, but again it is a good habit to follow.
There was a problem hiding this comment.
This is micro-optimization on the nanosecond level, and for the present case not even a benefit
In [2]: %timeit a in [1, 2]
63 ns ± 9.71 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [3]: %timeit a in {1, 2}
73.4 ns ± 8.28 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
In [4]: %timeit a in (1, 2)
61.3 ns ± 7.24 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each)
I think the list is slightly more readable because it's used more often, and because sets and dicts use curly braces so you have to look a bit closer to realize this is a set.
|
The coverage change in the test suite is due to azure not uploading artifacts on PRs. |
|
@tacaswell is there anything for me to do on this PR or does a fix need to go in? |
|
I don't understand all this yet, but I can't help but think there must be a much simpler way to achieve the goal. Use 2 very simple classes, one being the present one, and the second being a single-argument version? Or add a single kwarg to the present constructor to signal 1 argument or 2? Regarding the concern that func can be changed at runtime, that has always been true, and it has always been the responsibility of the person making that change to substitute a new func with the same signature. |
#16715 is a move to hide the machinery all-together. We could keep the existing system which is basically that the input to something like FuncFormatter is I'm against having two classes that practically speaking do the same thing as I think that's going to just be "I have no idea which one to use ever". The complexity here is because Python doesn't support c-style overloading and I think the current usual approach to dealing with it is the switch like dispatch system that this PR is doing. I can agree that maybe this PR doesn't need the string.format parsing but also see how that could be useful to users.
I'm against the user telling us how many arguments are in the function as they're providing us that information in the function itself. I can maybe see making it an optional argument from an optimization point of view, but the expensive type checking doesn't happen in the
But I think them substituting with a new signature so long as it was valid would have still worked as there was no check against it? |
|
I'm not convinced; I would still argue we should be looking for simpler code that is good enough. a Or one could put a try-except in the |
|
two objects While try/catch |
|
I've spent 10 minutes with this, and don't quite understand what it gets us. If it just makes it so the user doesn't have to define a If it doesn't have other benefits, then I'd rather we just explained clearly what |
|
Yes the benefit is that the user doesn't have to type 'pos' but like even explaining 'pos' doesn't help the matter since technically the formatter doesn't need pos; the call signature is The side benefit is by explicitly checking the function signature we can flag and warn users if they're passing in a function that may not work the way they expect it to & we can write a clearer message for functions that straight up won't work (functions with more than two required arguments). |
|
Granted, there are advantages in catching problems with the func at formatter instantiation, but I don't think those advantages are compelling. def __call__(loc, pos=None):
try:
return self.func(loc)
except TypeError:
return self.func(loc, pos)That's pretty simple, it adds only 3 lines to FuncFormatter, and I think it will do the job. I don't see that the try/except is obscuring anything. Do we really need validation of func? I don't think so. Using FuncFormatter directly is a semi-expert operation, and the suggestion above makes it considerably easier to use. It's not a foot-cannon. |
There have been a bunch of PRs lately where we've been giving people clearer/more instructive error messages and warnings. Catching at instantiation would be in-line with that. ETA: Also, if it's in the |
|
On 2020/05/07 12:26 PM, hannah wrote:
There have been a bunch of PRs lately where we've been giving people
clearer/more instructive error messages and warnings. Catching at
instantiation would be in-line with that.
True--but that alone does not make it compelling. Modifying
FuncFormatter as I suggest makes it very simple and straightforward to
understand and to use. Adding better error messages and warnings makes
sense in places where confusion is likely or common, but it doesn't make
sense to do it everywhere.
|
FuncFormatter has the exact same public API regardless of which approach we go with. |
|
Agreed, and I favor the approach with simpler code. But maybe I am missing something. Let's discuss it on the Monday call. |
|
Agree on call on Monday, but the problem with the try catch block is that code like this which is kind of reasonable for folks to write by mistake will get caught by the except: def myformat(x):
return f"{x}"+xSo instead of seeing this message: TypeError Traceback (most recent call last)
<ipython-input-40-10c8bb7d6128> in <module>
----> 1 myformat(1)
<ipython-input-39-00008efbf479> in myformat(x)
1 def myformat(x):
----> 2 return f"{x}"+x
TypeError: can only concatenate str (not "int") to strusers will get this: ---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-41-fed007dcbe0d> in <module>
1 try:
----> 2 myformat(1)
3 except TypeError:
<ipython-input-39-00008efbf479> in myformat(x)
1 def myformat(x):
----> 2 return f"{x}"+x
TypeError: can only concatenate str (not "int") to str
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
<ipython-input-41-fed007dcbe0d> in <module>
2 myformat(1)
3 except TypeError:
----> 4 myformat(1,None)
5
TypeError: myformat() takes 1 positional argument but 2 were givenwhich yes the first one is still there but we've introduced a second level of broken which I think will make debugging harder/more confusing. |
|
Can we use this pattern in the property? try:
sig = inspect.signature(func)
except ValueError:
self._nargs = 2
pass # built in, just keep going
else:
try:
sig.bind(None, None)
self._nargs = 2
except TypeError:
try:
sig.bind(None)
self._nargs = 1
except TypeError:
raise
self._func = func |
|
We settled on
|
…to use strmethodformatter
|
Since this Pull Request has not been updated in 60 days, it has been marked "inactive." This does not mean that it will be closed, though it may be moved to a "Draft" state. This helps maintainers prioritize their reviewing efforts. You can pick the PR back up anytime - please ping us if you need a review or guidance to move the PR forward! If you do not plan on continuing the work, please let us know so that we can either find someone to take the PR over, or close it. |
| try: | ||
| sig = inspect.signature(func) | ||
| except ValueError: | ||
| self._nargs = 2 | ||
| cbook._warn_external("FuncFormatter may not support " | ||
| f"{func.__name__}. Please look at the " | ||
| "other formatters in `matplotlib.ticker`.") | ||
| else: | ||
| try: | ||
| sig.bind(None, None) | ||
| self._nargs = 2 | ||
| except TypeError: | ||
| try: | ||
| sig.bind(None) | ||
| self._nargs = 1 | ||
| except TypeError: | ||
| raise TypeError(f"{func.__name__} must take " | ||
| "at most 2 arguments: " | ||
| "x (required), pos (optional).") |
There was a problem hiding this comment.
Maybe I'm over-simplifying, but can we just try?
| try: | |
| sig = inspect.signature(func) | |
| except ValueError: | |
| self._nargs = 2 | |
| cbook._warn_external("FuncFormatter may not support " | |
| f"{func.__name__}. Please look at the " | |
| "other formatters in `matplotlib.ticker`.") | |
| else: | |
| try: | |
| sig.bind(None, None) | |
| self._nargs = 2 | |
| except TypeError: | |
| try: | |
| sig.bind(None) | |
| self._nargs = 1 | |
| except TypeError: | |
| raise TypeError(f"{func.__name__} must take " | |
| "at most 2 arguments: " | |
| "x (required), pos (optional).") | |
| self._nargs = 2 | |
| # check whether func really suppors two args | |
| try: | |
| func(0, 0) | |
| except TypeError as e: | |
| # Only catch the specific error that a single-arg func would raise: | |
| # TypeError: func() takes 1 positional argument but 2 were given | |
| # every other error is not our business here and will raise on | |
| # actual usage. Since the TypeError will be thrown before any | |
| # internal errors in the function, this approach is a safe way to | |
| # identify single-arg functions without introspection need. | |
| if "1 positional argument" in str(e): | |
| self._nargs = 1 |
The only risk I see is that python changes its exception message, but i) by testing only against "1 positional argument" were more resilent wrt minor changes, and ii) we should add a test that monitors the exception message thrown in such cases.
|
Since this Pull Request has not been updated in 60 days, it has been marked "inactive." This does not mean that it will be closed, though it may be moved to a "Draft" state. This helps maintainers prioritize their reviewing efforts. You can pick the PR back up anytime - please ping us if you need a review or guidance to move the PR forward! If you do not plan on continuing the work, please let us know so that we can either find someone to take the PR over, or close it. |
PR Summary
Jumping off of #16715, the functions used to format ticks often only take the x value and position is left as none. This is also motivated by #17286, because optimally the cursor formatting should be the same as the tick formatting. This PR would hopefully simplify using FuncFormatter by letting users pass in a function that only takes an x value and not having to bother with a pos value they don't use. I also added tests for FuncFormatter
It's admittedly sorta hackier than I like but builtins like "{}".format can't be inspected. Also any advice on cleaning up the commits would be helpful.
I'll update examples, whats_new, and api_changes if there's consensus on this going in.
PR Checklist