Quadmesh.set_array validates dimensions #20215

jeffreypaul15 · 2021-05-12T17:18:12Z

PR Summary

PR Checklist

Has pytest style unit tests (and pytest passes).
Is Flake 8 compliant (run flake8 on changed files to check).
New features are documented, with examples if plot related.
Documentation is sphinx and numpydoc compliant (the docs should build without error).
Conforms to Matplotlib style conventions (install flake8-docstrings and run flake8 --docstring-convention=all).
New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).

github-actions

Thank you for opening your first PR into Matplotlib!

If you have not heard from us in a while, please feel free to ping @matplotlib/developers or anyone who has commented on the PR. Most of our reviewers are volunteers and sometimes things fall through the cracks.

You can also join us on gitter for real-time discussion.

For details on testing, writing docs, and our review process, please see the developer guide

We strive to be a welcoming and open project. Please follow our Code of Conduct.

jklymak · 2021-05-12T20:14:49Z

This needs to pass the "checks" above, whereas it seems a lot is broken ;-)

jklymak · 2021-05-12T20:57:00Z

Drill down on those tests to see what fails, and then I usually work on them on my home machine until they pass, but that does mean you need to setup py.test locally.

jeffreypaul15 · 2021-05-12T21:00:40Z

Drill down on those tests to see what fails, and then I usually work on them on my home machine until they pass, but that does mean you need to setup py.test locally.

I'm a little confused on how the coverage is decreasing, all of the new code is tested and works fine but the coverage seems to be an issue.

jeffreypaul15 · 2021-05-13T13:11:55Z

@jklymak Does this need a mention in the api_changes?

jklymak · 2021-05-13T14:03:36Z

I'm not sure - did it raise a type error before? If the error type is different that probably merits a API change note

jeffreypaul15 · 2021-05-13T14:07:56Z

I'm not sure - did it raise a type error before? If the error type is different that probably merits a API change note

The changes raise an error when faulty data is passed to set_array so I'm assuming this requires a note.

jklymak · 2021-05-13T14:09:04Z

Sure, but presumably faulty data raised an error before, at least at draw time?

jeffreypaul15 · 2021-05-13T14:15:35Z

Sure, but presumably faulty data raised an error before, at least at draw time?

At draw time yes, but not while using set_array though, so the user could pass array of any shape to set_array.

jklymak · 2021-05-13T14:57:34Z

lib/matplotlib/cm.py

+        if isinstance(self, mpl.collections.QuadMesh):
+            A_ = A.ravel()
+            width, height = self._meshWidth, self._meshHeight
+            if self._shading in ['flat', 'nearest', 'auto']:


Sorry, I should have seen this sooner. This check isn't compatible with flat or auto - please see the docstring for pcolormesh...

I think this function needs to be called when the QuadMesh is initialized to make sure it is compatible with the initialization. I don't know if that means we want to drop extra data for flat or not.

Ah sorry I misunderstood how auto works.

I think this function needs to be called when the QuadMesh is initialized to make sure it is compatible with the initialization. I don't know if that means we want to drop extra data for flat or not.

When quadmesh is initialized the arguments are already validated for faulty data right?

Also, when shading = flat, should set_array only accept data of same shape as X and Y (where extra data is dropped) or should it accept data of where it's dimensions are one less than X and Y, or should it accept both?

So the shading for A [NxM]

nearest: x[M], y[N]
flat: x [M+1], y[N+1] is preferred, but for back-combat x[M], y[N] works by dropping last row and column of A (so it is now [N-1 x M-1]
auto: Chooses nearest if x [M], y[N], or flat if x [M+1], y [N+1]
gouraud: x[M+1], y[N+1].

Ah, okay thanks for explaining it.
One more small question I've got is

matplotlib/lib/matplotlib/axes/_axes.py

Line 5727 in 81f0624

shading = 'flat'

When shading is set to auto and like you've mentioned auto: Chooses nearest if x [M], y[N], or flat if x [M+1], y [N+1].
Shading is set back to being flat when x[M[, y[N] and C is of the same shape, this basically causes Quadmesh._shading to be set to flat regardless of the shading set to auto, it's never nearest when set to auto.

jeffreypaul15 · 2021-05-13T17:03:43Z

@jklymak I've made these changes and it should account for when shading is flat, let me know if this is fine

jklymak · 2021-05-13T17:22:58Z

lib/matplotlib/cm.py

+            A_ = A.ravel()
+            width, height = self._meshWidth, self._meshHeight
+            if self._shading == 'flat':
+                if not (len(A_) == width * height or


I think we now need to check all these, not only that they error, but that they work if the set_array is correct? Like I'm still not sure what happens if you have shading=flat and pass len(A) == width*height after the fact.

Sorry, this might not have really been a good fist issue because of the goofy semantics of this.

Ah, no worries :) before any of the changes made, I had a look at this, when len(A) == (width+1)(height+1) and shading = flat, the same thing happens where the last dimensions are dropped. When len(A) == widthheight, then it's the same thing as setting A with the dimensions of one less than X and Y.
Here, when shading is set to flat, regardless of the shape of C passed to pcolormesh, the height (self._meshHeight) and width (self._meshWidth) are always the same shape as one less than X and Y.
For example :

# shape of xe is (10,) # shape of ye is (14,) # shape of c is (14,10) col = plt.pcolormesh(xe, ye, c, shading='flat') # works with drops the dimensions to get (13, 9) accordingly. # But self.+_meshHeight, self._meshWidth is = (13, 9) # when # shape of xe is (10,) # shape of ye is (14,) # shape of c is (13,9) col = plt.pcolormesh(xe, ye, c, shading='flat') # works as intended without dropping any dimensions. # And self.+_meshHeight, self._meshWidth is = (13, 9), same as that when c is of (14, 9)

So my point is regardless of the shape of A passed to set_array, in the example listed above it should either be (13, 9) or (14, 10) as self._meshHeight and self._meshWidth is already automatically set to (13, 9) if the dimensions are dropped

OK, just checking. I'd forgotten if the dropped row/column happened when it was ingested or when it was drawn...

lib/matplotlib/cm.py

lib/matplotlib/collections.py

timhoffm · 2021-05-14T13:44:15Z

lib/matplotlib/collections.py

+            A_ = A.ravel()
+            width, height = self._meshWidth, self._meshHeight
+            if self._shading == 'flat':
+                if not (len(A_) == width * height or


Um, don't you have to check width and height separately? Otherwise, the dfiffernce between (3, 4) and (4, 3) would not be catched.

Ah, yikes I forgot about that case. My apologies, I'll fix this.

The simplest way seemed to be doing it with a flag, otherwise we'd have to repeat

raise TypeError(f"Dimensions of A {shape} are incompatible with X ({width}) and/or Y ({height})")

multiple times

jklymak · 2021-05-14T18:50:28Z

lib/matplotlib/collections.py

+            width, height = self._meshWidth, self._meshHeight
+
+            if self._shading == 'flat':
+                if len(A.shape) == 1:


Do we allow len(A.shape())==1 arrays to be passed and they get reshaped somehow? If so, I'm not sure why we would block other arrays that fit? I tried to track down what happens but could not quickly...

I was under the assumption that we only allow flattened arrays that fit, isn't that true? I'm not sure I understand what other arrays that fit mean

I was under the assumption that we only allow flattened arrays that fit, isn't that true?

I don't know. That is what I am asking.

jeffreypaul15 · 2021-05-14T21:55:42Z

Actually, before any of the changes were made, pcolormesh calls set_array under the hood and it passes C.ravel() to it. From what I've checked, passing an array A of height*width or width*height would still produce the same plot.

So based on @timhoffm's suggestion, I don't really think we should be checking for height and width separately.
When len(A.shape())==1 it should just match the X and Y dimensions (based on shading) when flattened.

jklymak · 2021-05-14T22:03:07Z

From what I've checked, passing an array A of heightwidth or widthheight would still produce the same plot.

Right, so the new checks you've added would be an API change, which I don't think we want to get into here. So maybe you should go back to what you had before where you just checked that len(A.ravel()) == height*width (etc). Unless @timhoffm wanted an API change (which wouldn't be terrible, but we would need to make sure the API change note reflected that)

jeffreypaul15 · 2021-05-14T22:49:28Z

From what I've checked, passing an array A of height_width or width_height would still produce the same plot.

Right, so the new checks you've added would be an API change, which I don't think we want to get into here. So maybe you should go back to what you had before where you just checked that len(A.ravel()) == height*width (etc). Unless @timhoffm wanted an API change (which wouldn't be terrible, but we would need to make sure the API change note reflected that)

Yeah, makes sense. I'll revert back to those changes. Thanks for explaining it :)

jklymak · 2021-05-14T23:02:53Z

I'd wait for Tim to comment. He is the API boss but he may not have realized this is a change.

jeffreypaul15 · 2021-05-14T23:03:56Z

I'd wait for Tim to comment. He is the API boss but he may not have realized this is a change.

Yes, will do

timhoffm · 2021-05-15T23:37:00Z

In the light of #18472 (comment) and my reply below that I propose the following:

For all inputs: raise if the total number of elements does not match.
For 2D inputs: _api.warn_deprecated() if the shape does not match.
For nD (N>=3) inputs _api.warn_deprecated()

While color values are flattened before passing them to the renderer, it does not make sense to accept other shapes of matching total number of elements or even higher-dimensional data.

Whether we should deprecate 1D inputs as well should be discussed separately. This would violate the Liskov substitution principle. We'd have to weigh that against a cleaner more narrow API.

timhoffm · 2021-05-31T19:53:13Z

@jeffreypaul15 I'm sorry, I currently don't have bandwidth to look into this. Will take a couple of days for a response.

jeffreypaul15 · 2021-06-01T12:52:12Z

@jeffreypaul15 I'm sorry, I currently don't have bandwidth to look into this. Will take a couple of days for a response.

No worries

QuLogic · 2021-06-02T00:15:45Z

I think your other PRs are in? You should rebase to remove those changes from this one.

timhoffm

Almost good to go. Only some comment/documentation to do.

lib/matplotlib/collections.py

timhoffm · 2021-06-06T16:04:34Z

lib/matplotlib/collections.py

@@ -2047,6 +2046,40 @@ def set_paths(self):
        self._paths = self._convert_mesh_to_paths(self._coordinates)
        self.stale = True

+    def set_array(self, A):


This is public API, so it needs a docstring.

Ah, will do.

I'm not really sure how to structure this docstring, please make any changes if required

Well, it would have inherited a docstring from ScalarMappable, I think.

matplotlib/lib/matplotlib/cm.py

Lines 363 to 373 in 63261cc

"""

Set the value array from array-like *A*.

Parameters

----------

A : array-like or None

The values that are mapped to colors.

The base class `.ScalarMappable` does not make any assumptions on

the dimensionality and shape of the value array *A*.

"""

Though some of that needs to be overridden anyway, so it's good to write one here.

lib/matplotlib/collections.py

timhoffm

Sorry, my docstring suggestions apparently had unneeded spaces 🙄 , which cause flake8 to complain.

You can directly merge the corrections from the github interface. We'll squash-merge this PR anyway.

lib/matplotlib/collections.py

QuLogic · 2021-06-07T21:20:47Z

lib/matplotlib/collections.py

@@ -2047,6 +2046,40 @@ def set_paths(self):
        self._paths = self._convert_mesh_to_paths(self._coordinates)
        self.stale = True

+    def set_array(self, A):


Well, it would have inherited a docstring from ScalarMappable, I think.

matplotlib/lib/matplotlib/cm.py

Lines 363 to 373 in 63261cc

"""

Set the value array from array-like *A*.

Parameters

----------

A : array-like or None

The values that are mapped to colors.

The base class `.ScalarMappable` does not make any assumptions on

the dimensionality and shape of the value array *A*.

"""

Though some of that needs to be overridden anyway, so it's good to write one here.

The requested change has been addressed.

jklymak · 2021-06-10T14:14:24Z

This is great @jeffreypaul15 Did you want to rebase to just one commit? We can squash it from our end if you prefer...

jklymak · 2021-06-10T14:15:53Z

https://www.internalpointers.com/post/squash-commits-into-one-git is as good a guide as any to how to do this...

jeffreypaul15 · 2021-06-10T14:18:04Z

https://www.internalpointers.com/post/squash-commits-into-one-git is as good a guide as any to how to do this...

Thanks, but I've got a history of messing things up when squashing. I'll have a look at this and in the meantime you can squash it, thanks for reviewing all of this repeatedly 😄

jklymak · 2021-06-10T14:27:39Z

.. I pinged you on gitter

jklymak · 2021-06-10T16:22:26Z

Congratulation @jeffreypaul15 Sorry that was such a tough one!

jeffreypaul15 · 2021-06-10T16:31:39Z

Congratulation @jeffreypaul15 Sorry that was such a tough one!

Haha glad that it's done. Thanks for the help and multiple reviews

github-actions bot reviewed May 12, 2021

View reviewed changes

jklymak added the status: needs revision label May 12, 2021

jklymak marked this pull request as draft May 12, 2021 20:15

jklymak marked this pull request as ready for review May 13, 2021 14:03

jklymak removed the status: needs revision label May 13, 2021

jklymak reviewed May 13, 2021

View reviewed changes

jklymak added this to the v3.5.0 milestone May 13, 2021

jeffreypaul15 force-pushed the mpl-17508 branch from ae3de41 to 2204dd1 Compare May 13, 2021 16:43

jklymak reviewed May 13, 2021

View reviewed changes

jklymak approved these changes May 13, 2021

View reviewed changes

jklymak added status: needs review API: argument checking labels May 13, 2021

timhoffm reviewed May 14, 2021

View reviewed changes

lib/matplotlib/cm.py Outdated Show resolved Hide resolved

lib/matplotlib/cm.py Outdated Show resolved Hide resolved

lib/matplotlib/cm.py Outdated Show resolved Hide resolved

lib/matplotlib/cm.py Outdated Show resolved Hide resolved

timhoffm reviewed May 14, 2021

View reviewed changes

jklymak reviewed May 14, 2021

View reviewed changes

QuLogic added the status: waiting for other PR label May 29, 2021

timhoffm reviewed Jun 6, 2021

View reviewed changes

timhoffm removed status: needs revision status: waiting for other PR labels Jun 6, 2021

timhoffm reviewed Jun 6, 2021

View reviewed changes

lib/matplotlib/collections.py Outdated Show resolved Hide resolved

lib/matplotlib/collections.py Outdated Show resolved Hide resolved

timhoffm approved these changes Jun 7, 2021

View reviewed changes

lib/matplotlib/collections.py Outdated Show resolved Hide resolved

lib/matplotlib/collections.py Outdated Show resolved Hide resolved

lib/matplotlib/collections.py Outdated Show resolved Hide resolved

QuLogic reviewed Jun 7, 2021

View reviewed changes

jklymak approved these changes Jun 10, 2021

View reviewed changes

jklymak added the status: needs rebase label Jun 10, 2021

jklymak marked this pull request as draft June 10, 2021 14:14

jeffreypaul15 force-pushed the mpl-17508 branch 5 times, most recently from b9bfdb3 to f4d2d61 Compare June 10, 2021 15:11

Quadmesh validates set_array dimensions

d08f8ea

jeffreypaul15 force-pushed the mpl-17508 branch from f4d2d61 to d08f8ea Compare June 10, 2021 15:13

jeffreypaul15 marked this pull request as ready for review June 10, 2021 15:13

jklymak removed the status: needs rebase label Jun 10, 2021

jklymak merged commit ee577c5 into matplotlib:master Jun 10, 2021

	"""
	Set the value array from array-like A.

	Parameters
	----------
	A : array-like or None
	The values that are mapped to colors.

	The base class `.ScalarMappable` does not make any assumptions on
	the dimensionality and shape of the value array A.
	"""

Uh oh!

Quadmesh.set_array validates dimensions #20215

Quadmesh.set_array validates dimensions #20215

Uh oh!

Conversation

jeffreypaul15 commented May 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Checklist

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

jklymak commented May 12, 2021

Uh oh!

jklymak commented May 12, 2021

Uh oh!

jeffreypaul15 commented May 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jeffreypaul15 commented May 13, 2021

Uh oh!

jklymak commented May 13, 2021

Uh oh!

jeffreypaul15 commented May 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jklymak commented May 13, 2021

Uh oh!

jeffreypaul15 commented May 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffreypaul15 May 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffreypaul15 May 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffreypaul15 commented May 13, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffreypaul15 May 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffreypaul15 May 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffreypaul15 commented May 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

jeffreypaul15 commented May 12, 2021 •

edited

Loading

jeffreypaul15 commented May 12, 2021 •

edited

Loading

jeffreypaul15 commented May 13, 2021 •

edited

Loading

jeffreypaul15 commented May 13, 2021 •

edited

Loading

jeffreypaul15 May 13, 2021 •

edited

Loading

jeffreypaul15 May 13, 2021 •

edited

Loading

jeffreypaul15 May 14, 2021 •

edited

Loading

jeffreypaul15 May 14, 2021 •

edited

Loading

jeffreypaul15 commented May 14, 2021 •

edited

Loading

timhoffm commented May 15, 2021 •

edited

Loading