-
-
Notifications
You must be signed in to change notification settings - Fork 7.9k
ENH: Adding callbacks to Norms for update signals #19553
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I'll block on this feature, and suggest #4387 be closed.
In general I am against magic callbacks like this. I think an artist should reflect the state of the artist at creation time, or when it is explicitly acted upon, not at draw time, and possibly based on actions applied to other artists. If you need to keep two artists limits in sync, you should explicitly do so, and not rely on a black box callback.
I appreciate there was a push to make the library cache more things until draw time, and maybe if the whole library had been designed with that goal from the start that would make sense. But as-is, implicit state is simply confusing and leads to unexpected behaviour.
Of course happy to unblock if there is a real strong case for why this is needed, and can't be done at the user level.
I think I agree with you...? This isn't caching until draw-time, it is just creating an update mechanism that will automatically trigger the re-drawing chain (norm -> scalar-mappable -> artist). Someone still has to explicitly modify the norm. This was motivated by me wanting to have interactivity with the colorbars and panning/zooming on that axis, see here: #19515 Basically, I was trying to bring things a little more in-sync throughout the pipeline of the scalar-mappable updates to not force a user to call |
I see. But that feature should also be discussed I guess? Are you suggesting if I zoom on a colorbar, that should change the mappable somehow? How? And why is that a useful thing to do? I guess I could imagine the converse, where you set the limits of the colorbar to change vmin and vmax, but that would require a text box or something, rather than zooming. But here, you are pushing it further, and not only changing the mappable that belongs to the colorbar, but changing the norm and wanting the changes to propagate to other mappables that may have the "same" norm. I guess I'm arguing we should be doing exactly the opposite, and making the norms a copy when added to the mappable, and hence a change to one mappable does not affect the others. But maybe I haven't though that through and many folks are making a single norm, and sharing it, and then expecting to manipulate the norm and have it show up on the next draw. I guess the PR would benefit from a bit more context of what the current state is, and why this is an improvement? |
@jklymak The current state is that this: fig, ax = plt.subplots()
norm = plt.Normalize(-2, 2)
cmap = copy.copy(plt.get_cmap('BuGn'))
ax.imshow(data, norm=norm, cmap=cmap)
norm.vmax = 0.1
norm.vmin = -0.1
cmap.set_over('red') plots with a norm from 0.1 to -0.1, and a colormap with an "over" color of red. I'm not a fan of having the API make a copy of an object. I don't know of anywhere else in the API that we do that (e.g. |
OK, sure, but without running it, are you sure what this does? import matplotlib.pyplot as plt
import numpy as np
import copy
fig, axs = plt.subplots(1,2)
norm = plt.Normalize(-2, 2)
data = np.linspace(-1, 1, 900).reshape(30, 30)
cmap = copy.copy(plt.get_cmap('BuGn'))
axs[0].imshow(data, norm=norm, cmap=cmap)
norm.vmax = 0.1
norm.vmin = -0.1
cmap.set_over('red')
im = axs[1].imshow(data, norm=norm, cmap=cmap)
fig.colorbar(im, ax=axs)
im.set_clim(-0.3, 0.3)
plt.show() |
I will say what I think should happen without running it. Everything is using the same norm, so everything should be in-sync with the final updates that were made before the If you run this in an interactive session, currently when you modify the norm I'll try to add some more context to the original PR summary |
OK, thanks @greglucas and @dopplershift - I thought about this some and agree with both of you. This sort of behaviour can be a bit mysterious - I wouldn't expect the user to know that the clim on an object is linked up to a global norm instance, but I agree with you that this is how it is, and that such behaviour can have uses.... |
As the author of the linked issue I am (still) in favor of this :) The problem this is trying to solve is that we do have a number of caches (for performance reasons) that need to be invalidated when we change the norm. If you do it one way (through
This is adding an explicit coupling (via the callback) to get rid of an implicit coupling (with hysteresis and dependencies on exactly where draws are!). I have not run the code on this PR, but I think it will fix this. We will need to check than |
@jklymak, just to be clear there is no global Norm here. This is a specific norm object and you can share that object between images to keep them in sync. But, if you put @tacaswell, I got rid of the 3rd |
No I understand, but there is a top-level norm = plt.LogNorm()
im1 = ax1.imshow(X1, norm=norm)
im2 = ax2.inshow(X2, norm=norm)
im2.set_clim(1e-3, 1e5) and be surprised that this affects |
That makes sense, but that is the case regardless of this PR (both images are modified on current master), and I think not too bad to explain to someone not to re-use an object like that but rather just create separate |
@@ -43,6 +44,9 @@ def test_imagegrid_cbar_mode_edge(): | |||
# "second" ones. To achieve this, clear out the axes first. | |||
for ax in grid: | |||
ax.cax.cla() | |||
# We don't want to share the norms between the colorbar axes | |||
# so create a deepcopy for each image | |||
ax.images[0].norm = copy.deepcopy(ax.images[0].norm) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did we used to rely on a race condition in drawing to make this test work?
I'm also not seeing in the above code how they ended up coupled.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This one took me a long time to figure out that this one-line fix is all that is needed. If I remove the deepcopy, the scales on the colorbar axes seem to stick around or not get updated appropriately? There are some extra ticks floating around and some spines don't get fully removed. I think there is some state that isn't fully getting cleared in axes_grid
with the way the colorbars are handled there.
I don't think this is the most ideal test with this comment about clearing and overwriting portions of the axes.
# In each row/column, the "first" colorbars must be overwritten by the
# "second" ones. To achieve this, clear out the axes first.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test bit me again... I just added some new code to connect/disconnect the callbacks when changing the norm on a ScalarMappable that already exists. Now, I'm able to remove the ax.cax.cla()
call, but I need a brand new ScalarMappable for only this test. I think there is something with the colorbars trying to modify old state here, which is why creating a new instance that has no sharing is what we need.
I still don't 100% follow this test code and haven't taken the time to really investigate it in depth and determine whether this test could be rewritten, or if something else in colorbars should be updated.
cb = ax.cax.colorbar( | ||
ax.images[0], | ||
sm, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think @anntzer worked on this last, trying to make these colorbar-like objects the same as main-library colorbars.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This still feels like there is something different about axes_grid
that I am missing and we shouldn't need to deepcopy the norm on the mappable... My guess is there is possibly a locator or something similar attached to ax.cax
that needs to be cleared/reset on the ax.cax.cla()
call above, but I haven't been able to find it while debugging and have spent quite a lot of time on this.
Is there a way to step through a debugger session while keeping the plot window open to see how things move around while stepping into functions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not that I know of, but I never use a debugger (probably a bad idea).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am very confused as well. I think what is happening is that on the first pass of colorbar addition to the cax, some locators/formatters get updated and set due to the new callback. Then on the second addition of the colorbar, that state is still hanging around. Rather than deepcopying, I switched the test to remove the callback from the colorbar.
The difference with/without the final commit is that the second colorbar axis has ticklabels showing up. I'm not exactly sure where this state creeps in, so this is a hard one to trace down.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, I figured out this problem. This is basically due to the interaction between two points.
First, as discussed during the call, cla() doesn't call Colorbar.remove(), which means that the callback of the first image doesn't get deleted, which it should. Making cla() actually go through each and every artist to call cla() on it may be too expensive, but at least perhaps we can check if there's any colorbar on the axes and properly remove() them.
Still, even knowing that, it was unclear why the first image would overwrite the second colorbar. The reason is because the sequence of events was the following:
- instantiate first image, create associated colorbar.
- instantiate second image, create associated colorbar.
- actually walk the draw tree and draw the images... which has the effect of calling _ImageBase._make_image, which contains the following snippet:
# we have re-set the vmin/vmax to account for small errors
# that may have moved input values in/out of range
s_vmin, s_vmax = vrange
if isinstance(self.norm, mcolors.LogNorm) and s_vmin <= 0:
# Don't give 0 or negative values to LogNorm
s_vmin = np.finfo(scaled_dtype).eps
with cbook._setattr_cm(self.norm,
vmin=s_vmin,
vmax=s_vmax,
):
output = self.norm(resampled_masked)
so here s_vmin and s_vmax were recomputed to take interpolation into account. Unfortunately, this means they may be slightly different due to the rescaling and unscaling, which can be checked by printing s_vmin - A.min(), s_vmax - A.max()
. One then sees that there's indeed a tiny (1e-16) difference for the first image's norm, which means it'll say, oh, I need to redraw the colorbar; but there's no difference(!) for the second image's norm, so it doesn't redraw itself. A quickfix to confirm that this works is that forcing s_vmin = A.min(); s_vmax = A.max()
here fixes the issue (but would cause other problems with over/under interaction with resampling; I suspect that the "correct" fix is to temporarily disable emission of callbacks by self.norm
just for the call to _setattr_cm
.
That was nasty :p
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That looks like it did the trick. I disconnected from the callback before this and reconnected after it. I wonder if more fundamentally there should be something within the context manager to remove the callbacks on enter/exit...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess we could add a disabling_callbacks()
context manager on CallbackRegistry?
with self.norm.callbacks.disabling_callbacks(), cbook._setattr_cm(self.norm, ...): ...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, that looks pretty clean to me. I'm not sure how much more use it would have in more than this one situation though, so is it worth adding or not...
Lets keep it as a separate issue and implement it later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rebased and updated this with the blocked callback feature now.
f3064f5
to
b968fc5
Compare
@tacaswell I'll try and pop this back on your PR review queue... |
5cf5015
to
e97155b
Compare
@@ -1017,8 +1017,8 @@ def test_imshow_bool(): | |||
def test_full_invalid(): | |||
fig, ax = plt.subplots() | |||
ax.imshow(np.full((10, 10), np.nan)) | |||
with pytest.warns(UserWarning): | |||
fig.canvas.draw() | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this not warning anymore?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The previous warning was:
matplotlib/lib/matplotlib/image.py:442: UserWarning: Warning: converting a masked element to nan.
vmid = np.float64(self.norm.vmin) + dv / 2
If I print out the previous im.norm.vmin, im.norm.vmax
, they were both np.ma.masked
, which stems from passing in a fully masked array (np.max(np.ma.array([1, 2], mask=[True, True]))
returns masked
), so here we are calling np.float64(np.ma.masked)
. I don't think we want masked
as vmin/vmax, so this perhaps even helps here... The vmin/vmax turn into (0, 0)
now.
Note that the image doesn't change at all, just no warning about norm vmin/vmax anymore.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is fun second-order behavior change!
- In Don't convert vmin, vmax to floats. #6700 we stop passing vmin/vmax through
float
on init because it stripped precision of float128 - this broke yt that was expecting this to strip off units so we got Don't convert vmin, vmax to floats. #6700 to add
_sanitize_extrema
which gently passes things through float - this got updated in MAINT: numpy deprecates asscalar in 1.16 #12478 due to changes in numpy to deprecate np.asscalar
- this PR now passes all setting of vmin/vmax through
_sanitize_extrema
(instead of previously just on init) np.ma.masked.item() == 0
butfloat(np.ma.masked) is np.nan
Previously if you did
im = plt.imshow(np.full((10, 10), np.nan), vmin=np.ma.masked, vmax=np.ma.masked)
assert im.norm.vmin == 0
assert im.norm.vmax == 0
but if you did
im = plt.imshow(np.full((10, 10), np.nan)
assert im.norm.vmin is np.ma.masked
assert im.norm.vmax is np.ma.masked
(==
is nan-like with np.ma.masked
).
I would argue that this is a bug, in the sense that explicitly passing the np.max(data)
and letting us auto-computer the max gave different results.
We also say that if vmin == vmax the result of __call__
will be 0.
This will lead to a subtle behavior change in the case where:
- the user passed us all nan
- let us infer the vmin/vmax
- later updates the array with non-nan values
Previously the user would get all "bad" color, now they will get the "0" color. Given the other difference in behavior of explicitly passing vmin/vmax, the agreement with "if vmin/vmax match -> go to 0", this feeling like a very cornery corner case, and the results already being useless from a communications point of view, I think we can leave this as-is and do not need a further API change note.
I probably need to spend more time with this to understand what is going on...
If we go with a callback registry, it should be named |
Also fixes #17052 which I just found. |
e3b2932
to
af3b419
Compare
This adds a callback registry to Norm instances that can be connected to by other objects to be notified when the Norm is updated. This is particularly relevant for ScalarMappables to be notified when the vmin/vmax are changed on the Norm. Quadcontourset overrides ScalarMappable's `changed()` function, which meant that autoscaling would get called with the wrong data too early. Therefore, we force an autoscale with the proper data earlier in the `Quadcontourset.changed()` function. The Quadcontourset.changed() method assumes some attributes to be there. If we called changed from a parent class, the object may not have been initialized with those attributes yet, so skip that portion of the update. Remove the ScalarMappable callback from axes_grid as the state isn't fully cleared when updating the axes.
af3b419
to
88ca5c2
Compare
…553-on-v3.5.x Backport PR #19553 on branch v3.5.x (ENH: Adding callbacks to Norms for update signals)
ENH: Adding callbacks to Norms for update signals
ENH: Adding callbacks to Norms for update signals
PR Summary
This adds a callback registry to Norm instances that can be connected to by other objects to be notified when the Norm is updated. This is particularly relevant for ScalarMappables to be notified when the vmin/vmax are changed on the Norm. With this update, the axis is now immediately registered as stale in the following example.
Closes #4387
Fixes #17052
PR Checklist
pytest
passes).flake8
on changed files to check).flake8-docstrings
and runflake8 --docstring-convention=all
).doc/users/next_whats_new/
(follow instructions in README.rst there).doc/api/next_api_changes/
(follow instructions in README.rst there).