Optimize imshow #26335

eendebakpt · 2023-07-17T13:33:25Z

PR summary

In this PR we apply some micro optimizations to improve the imshow performance. Benchmark:

Mean +- std dev: [main] 7.34 ms +- 0.07 ms -> [pr] 6.67 ms +- 0.11 ms: 1.10x faster

Notes:

There is a fast path for the MaskedArray in case the mask is False (e.g. images without a NaN or Inf value)
The np.ma.masked_invalid undoes the shrink_mask from np.ma.masked_where, but matplotlib applies it again. So it is faster to call np.ma.masked_where directly
Another possible optimization is in Normalize.autoscale_None. There both the setting of vmin and vmax triggers _changed. It is faster to first set vmin and vmax (without triggering _changed) and then calling _changed. This would require a new method set_vmin_vmax or some other mechanism to avoid triggering _changed twice.

Benchmark script:

import matplotlib
# print(matplotlib)
import pyperf

setup = """
import matplotlib.pyplot as plt
import numpy as np

n=10
im=np.random.rand(100,100)

def go():
    for ii in range(n):
        plt.figure(1)
        plt.imshow(im)
"""

runner = pyperf.Runner()
runner.timeit(name="mpl", stmt="go()", setup=setup)

PR checklist

[N/A] "closes #0000" is in the body of the PR description to link the related issue
new and changed code is tested
[N/A] Plotting related features are demonstrated in an example
[N/A] New Features and API Changes are noted with a directive and release note
[N/A] Documentation complies with general and docstring guidelines

anntzer · 2023-07-17T14:10:59Z

lib/matplotlib/cm.py

@@ -735,5 +735,5 @@ def _ensure_cmap(cmap):
    cmap_name = cmap if cmap is not None else mpl.rcParams["image.cmap"]
    # use check_in_list to ensure type stability of the exception raised by
    # the internal usage of this (ValueError vs KeyError)
-    _api.check_in_list(sorted(_colormaps), cmap=cmap_name)


This matters for the printed error message when the colormap is invalid. (We could instead ensure that _colormaps is always sorted by sorting it whenever a new item is added, though.)

I see. We could ensure the ordering of _colormaps by adding c._cmaps = {k: c._cmaps[k] for k in sorted(c._cmaps)} at the end of ColormapRegistry.register. Under the assumption that number of colormaps added is reasonable, this will not have any other negative performance impact.

Another option is to replace the code with

if not cmap_name in _colormaps: _api.check_in_list(sorted(_colormaps), cmap=cmap_name)

Making sure that __iter__ on ColormapRegistry is always sorted is probably the best option (rather than mucking with dictionary order we can keep a self._sorted_keys = sorted(self._cmap) attribute and iterate over that in __iter__.

but the if not in... is also fine (as I see you already pushed that).

eendebakpt · 2023-07-17T14:53:46Z

lib/matplotlib/colors.py

+            if A.mask is False or not A.mask.shape:
+                A = A.data


Another way of testing:

Suggested change

if A.mask is False or not A.mask.shape:

A = A.data

try:

no_mask = not bool(A.mask)

except ValueError:

no_mask = False

if no_mask:

A = A.data

Numpy generated a ValueError when testing an array for truth value.

eendebakpt · 2023-07-17T21:34:59Z

Coverage was complaining because the number of lines of code was reduced. I added a few simple tests for strip_math to make ci happy.

tacaswell · 2023-07-17T23:03:28Z

Is stip_math otherwise implicated is this change and does the new test test something that we do not already test?

Some of the codecov failures sort them selves out when all of the CI jobs upload their results and are processed (as some of the tests are platform specific)

eendebakpt · 2023-07-18T07:50:30Z

Is stip_math otherwise implicated is this change and does the new test test something that we do not already test?

Some of the codecov failures sort them selves out when all of the CI jobs upload their results and are processed (as some of the tests are platform specific)

The strip_math is completely unrelated to this PR. Coverage showed the method was not in the unit testing (at least not explicit), so I picked it as an easy way to increase coverage. If you want I can remove the tests again and see what happens to the CI.

jklymak · 2023-07-18T16:39:08Z

Maybe directly testing strip_math is OK, but it points to whatever code calls strip_math as not being tested either. It gets used in LogFormatter, which I'd have thought was thoroughly tested, but maybe not?

EDIT: overall I think it's OK for codecov to decrease percentage in a PR if its due to code removal. It seems confusing to add an unrelated test just to make our tool happy. Maybe consider moving the test improvements to a separate PR?

eendebakpt added 3 commits July 17, 2023 15:20

optimize imshow

ca00835

fix check

6e9fe63

workaround

3991dc6

anntzer reviewed Jul 17, 2023

View reviewed changes

eendebakpt commented Jul 17, 2023

View reviewed changes

eendebakpt added 3 commits July 17, 2023 21:16

ensure error reporting is sorted

ffb8969

add test for strip_math

10a677a

add test for strip_math

c831e85

eendebakpt added 2 commits July 18, 2023 20:37

revert strip_math tests

a2de5ea

remove import

ea53f0d

eendebakpt changed the title ~~Draft: Optimize imshow~~ Optimize imshow Jul 18, 2023

oscargus approved these changes Jul 19, 2023

View reviewed changes

melissawm added the Performance label Jul 20, 2023

jklymak requested a review from anntzer July 27, 2023 18:10

greglucas approved these changes Jul 28, 2023

View reviewed changes

greglucas merged commit 4e988f5 into matplotlib:main Jul 28, 2023

QuLogic added this to the v3.8.0 milestone Jul 28, 2023

ksunden mentioned this pull request Sep 15, 2023

matplotlib 3.8.0 breaks a visualization example for the docs. astropy/astropy#15319

Closed

1 task

greglucas mentioned this pull request Jun 23, 2024

Ignore np.nan values in Normalize.autoscale() #28406

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Optimize imshow #26335

Optimize imshow #26335

Uh oh!

eendebakpt commented Jul 17, 2023 •

edited

Loading

Uh oh!

anntzer Jul 17, 2023

Uh oh!

eendebakpt Jul 17, 2023

Uh oh!

tacaswell Jul 17, 2023

Uh oh!

tacaswell Jul 17, 2023

Uh oh!

eendebakpt Jul 17, 2023

Uh oh!

eendebakpt commented Jul 17, 2023

Uh oh!

tacaswell commented Jul 17, 2023

Uh oh!

eendebakpt commented Jul 18, 2023

Uh oh!

jklymak commented Jul 18, 2023 •

edited

Loading

Uh oh!

Uh oh!

-            if A.mask is False or not A.mask.shape:
-                A = A.data
+            try:
+               no_mask = not bool(A.mask)
+            except ValueError:
+                 no_mask = False
+                if no_mask:
+                     A = A.data

Uh oh!

Optimize imshow #26335

Optimize imshow #26335

Uh oh!

Conversation

eendebakpt commented Jul 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR summary

PR checklist

Uh oh!

anntzer Jul 17, 2023

Choose a reason for hiding this comment

Uh oh!

eendebakpt Jul 17, 2023

Choose a reason for hiding this comment

Uh oh!

tacaswell Jul 17, 2023

Choose a reason for hiding this comment

Uh oh!

tacaswell Jul 17, 2023

Choose a reason for hiding this comment

Uh oh!

eendebakpt Jul 17, 2023

Choose a reason for hiding this comment

Uh oh!

eendebakpt commented Jul 17, 2023

Uh oh!

tacaswell commented Jul 17, 2023

Uh oh!

eendebakpt commented Jul 18, 2023

Uh oh!

jklymak commented Jul 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

eendebakpt commented Jul 17, 2023 •

edited

Loading

jklymak commented Jul 18, 2023 •

edited

Loading