Faster categorical tick formatter. #13917
Conversation
Having thousands of categories is most likely a sign that the user
forgot to convert strings to floats or dates, but we may as well not
take forever to generate the incorrect plot so that they can observe the
failure faster.
Right now StrCategoryFormatter constructs the value-to-label dict at
every call to `__call__` which leads to quadratic complexity when
iterating over the ticks. Instead, just do this once in `format_ticks`
(and let `__call__` use that implementation too), for linear complexity.
This speeds up
from pylab import *
cats = [str(x) for x in np.random.rand(4000)] # Bunch of labels.
plt.plot(cats)
plt.gcf().canvas.draw()
from ~25s to ~11s (and the difference gets bigger for more ticks as
we're comparing O(n^2) to O(n) (modulo dict lookup terms in log(n),
probably)).
The other option was to make UnitData maintain both a forward and a
backward mapping in sync but this would require passing the UnitData
instance rather than the mapping to the StrCategoryFormatter constructor
and the API break is just not worth it.
| return r_mapping.get(int(np.round(x)), '') | ||
| return '' if pos is None else self.format_ticks([x])[0] | ||
|
|
||
| def format_ticks(self, values): |
There was a problem hiding this comment.
I'm curious/confused about why/how the dict is only built once if there's still a call to .format_ticks([x]) on every call to call. Is their caching somewhere?
There was a problem hiding this comment.
The tick-labeling code calls format_ticks, not __call__ (__call__ is only used by the mouse-cursor-text (as a fallback because format_data_short is not defined) and left as a backcompat API). (So mouse-cursor-text is still slow, but at least the initial draw is less slow.)
There was a problem hiding this comment.
:/ I have no objection to switching out the data structure holding the mapping to something more reverse friendly
There was a problem hiding this comment.
the deprecation is a bit a pain to handle, I doubt it's worth it, especially considering that nearly all cases where this would be relevant are actually due to user error (not converting strings to floats/dates).
Having thousands of categories is most likely a sign that the user
forgot to convert strings to floats or dates, but we may as well not
take forever to generate the incorrect plot so that they can observe the
failure faster (cf "slow beyond belief" comment in #13910).
Right now StrCategoryFormatter constructs the value-to-label dict at
every call to
__call__which leads to quadratic complexity wheniterating over the ticks. Instead, just do this once in
format_ticks(and let
__call__use that implementation too), for linear complexity.This speeds up
from ~25s to ~11s (and the difference gets bigger for more ticks as
we're comparing O(n^2) to O(n) (modulo dict lookup terms in log(n),
probably)).
The other option was to make UnitData maintain both a forward and a
backward mapping in sync but this would require passing the UnitData
instance rather than the mapping to the StrCategoryFormatter constructor
and the API break is just not worth it.
PR Summary
PR Checklist