[Bug]: .contains for some artists does not check for visibility #23875

rdgraham · 2022-09-11T22:04:10Z

Bug summary

I recently managed to find the source of a long-standing bug in my application that was quite tricky to find.

It turned out to be a consequence of the contains method of a line returning true even when the line was set to not visible at the time. Although from what I can see in the documentation there is nothing prohibiting this, I think almost everyone would want contains to return false for objects which have been plotted and set to invisible.

Moreover, a non-exhaustive look through the code shows that there are class-specific checks in place for visibility for some objects, but not others. Specifically, the following do not have checks:

lines.Line2D
patches.Patch
quiver.QuiverKey

But these ones do (though not explicitly documented.)

collections.Collection
text.Text
image.BboxImage

It seems like a fix might put the check for visibility in the _default_contains method of Artist. This is called by the child implementations, but as of now seems to provide only a bail-out if the event is outside the figure canvas. Although I guess maybe it was done that way if there are some specific exceptions where you would actually want contains to return true (maybe bounding boxes or more abstract things not plotted directly).

Code for reproduction

from matplotlib import pyplot as plt

class Demo:

    def __init__(self):

        fig, ax = plt.subplots()

        self.lines = []
        self.lines.append(ax.axhline(1))
        self.lines.append(ax.axhline(-1))
        self.lines[0].set_visible(False)

        self.vis_points = ax.scatter([-0.5, 0.5], [0, 0])
        self.invis_points = ax.scatter([0], [0])
        self.invis_points.set_visible(False)
        self.points = [self.vis_points, self.invis_points]

        fig.canvas.mpl_connect('button_press_event', self.clicked)

        plt.show()

    def clicked(self, event):
        at_obj = lambda x: x.contains(event)[0]

        if any(at_obj(l) for l in self.lines):
            print(f"Clicked on a line at {event.ydata:0.3f}")
        elif any(at_obj(p) for p in self.points):
            print(f"Clicked point at {event.ydata:0.3f}")
        else:
            print('Clicked elsewhere')

if __name__ == '__main__':
    demo = Demo()

Actual outcome

It is possible to click anywhere near the invisible line at y=1 and see click registered message.
However, clicking at [0,0], the location of an invisible scatter point does not show a point clicked message (expected behavior).

Expected outcome

Clicking anywhere y=1 should not generate a line click message --- just as clicking at 0,0 does not generate a point clicked message.

Additional information

No response

Operating system

Arch linux, Linux mint, Windows

Matplotlib Version

3.5.0

Matplotlib Backend

qt

Python version

3.9.9

Jupyter version

No response

Installation

No response

The text was updated successfully, but these errors were encountered:

tacaswell · 2022-09-12T04:16:41Z

Definitely agree that is something we need to fix, however I am not convinced that putting the check in contains is the right place. The call chain to get to calling contains is through pick, and specifically the pick method on the parent. I can see both sides "should contains consider visibility?" and ways it can go confusing if it happens to have the same color as the background vs not being drawn vs happening to alpha=0... which is probably why we have a mix because different reasonable people have come to the opposite conclusion at different times and we should unify that (I'm inclined to think that it should go towards "contains does not care if the artist is visible). Getting that sorted may be a bigger project and any per-artist configuration has a high probability of going out of sync again.

Instead, I think a better place to fix this is in

matplotlib/lib/matplotlib/artist.py

Lines 513 to 525 in 9f17f3f

    
           # Pick children 
        
           for a in self.get_children(): 
        
               # make sure the event happened in the same Axes 
        
               ax = getattr(a, 'axes', None) 
        
               if (mouseevent.inaxes is None or ax is None 
        
                       or mouseevent.inaxes == ax): 
        
                   # we need to check if mouseevent.inaxes is None 
        
                   # because some objects associated with an Axes (e.g., a 
        
                   # tick label) can be outside the bounding box of the 
        
                   # Axes and inaxes will be None 
        
                   # also check that ax is None so that it traverse objects 
        
                   # which do no have an axes property but children might 
        
                   a.pick(mouseevent)

where we can add a filter to only recurse to visible children. This way we fix it for all classes of artists in the case where we really care here (picking) and avoid the discussion of what contains should do.

rdgraham · 2022-09-12T06:04:30Z

Thanks for the quick reply.

From my limited understanding, the pick_event mechanism might be the 'new' nicer way of implementing these interactive elements, but one can also still use .contains directly and gain a little more control. I have a lot of code which I don't think I could easily change to using the picking mechanism -- for one example, I need to detect specifically the first motion after a click and hold. Indeed I notice the documentation example on event handeling shows both ways.

So seems to me it would be better to unify this in the contains methods directly (or indeed default_contains if there are not other issues with that).

The case of having something set as visible but in practice invisible due to the color/alpha is one interesting thing I didn't think of.
Sounds difficult and I don't think it is strictly necessary to handle that one. If the documentation just clearly states that .contains returns true for any event which occurred within the object if it is currently set to visible that should be clear enough. Though maybe a footnote about transparency might be helpful.

tacaswell · 2022-09-12T15:02:56Z

Oh, 🐑 I did not read carefully enough and thought you were already using pick_event. I think the bug here is that there is inconsistency in the behavior but I'm still not sure which way it should go.

I see the argument contains should check visibility (as when used in an interactive context if you do not see it, it should not be "clickable"). However as you point out there are some cases where you might want to know if the event is in a not-drawn artist.

If contains uniformly did not check visibility, then it is up to the client to do the filtering. Effectively changing your at_obj to:

at_obj = lambda x: x.get_visible() and x.contains(event)[0]

which is clear about your intentions and I do not think too much more verbose. Another advantage of doing this is that it will work with all versions of Matplotlib independent of where contains lands!

On the other hand, if contains always set checked visibility the work around is to set the alpha to 0. This is a bit odd and for Agg has no real consequence (other than some CPU time), but for vector backends it will put the (invisible) element into the output. The various layout tools (tightlayout and constrained layout) ignore artists where obj.set_visible(False) but I am pretty sure will take into account artists where obj.set_alpha(0). Both of those are possibly disruptive, but are to be fair corner cases.

Adding a flag to contains is also an option, but I think a poor one. It would be fewer characters, but not actually any simpler than having the client call obj.get_visible()

When thinking about API choices I tend to consider what the choice will prevent at least as strongly as what it will enable (and the cost to the user of working around an incorrect assumption or the library developers part). On balance, I am still leaning towards contains should not check visibility and leave that decision/check to the caller.

pick_event has been in the library for at least a decade so I do not think it is really "new" anymore ;)

timhoffm · 2022-09-12T15:06:40Z

On balance, I am still leaning towards contains should not check visibility and leave that decision/check to the caller.

Agreed.

jklymak · 2022-09-12T15:55:52Z

Artists all have a pickable state separate from visible. We could either use that, or add another state.

rdgraham · 2022-09-13T06:20:36Z

Ok well from the discussion it seems like it makes sense from a design point of view if .contains consistently does not check for visibility, though I think only in the case that:

It is very clear in the docstrings
Example code such (as the event handling draggable rectangle example) is updated to include the check for visibility. Although that example doesn't change visibility anywhere specifically, code written based on the example may well do, and I think almost everyone would want draggable elements to only respond when visible.

However, I would caution that if there is an API change that makes point collections, text items etc now no stop checking for visibility then there is a potential to break existing code, and in a way that potentially seems intermittent and is hard to debug. So I think if going that route there should at least in the interim be a warning printed whenever the new behavior occurs, i.e. when a text item's .contains triggers on an invisible item.

I have to say that while in hindsight the issue is quite simple, in my case finding the bug in my application is probably somewhere among the most difficult bugs I have fixed. Mostly because it occurred so infrequently, only when a user happened to click in exactly the right small area. And it just didn't occur to me that it would be possible to select some types of invisible items, given testing with (more numerous) hidden scatter points not having any issue.

tacaswell · 2022-09-13T15:23:16Z

Artists all have a pickable state separate from visible. We could either use that, or add another state.

The pickable state can also be a callable so toggling it can be a bit annoying. I'm also not convinced that more state would be clearer.

@rdgraham Sorry if this did not go quite the way you hoped initially! I think one of the things that came out of this discussion is that we may have never actually had a discussion about this which is how we got to the inconsistent state.

Definitely agree that when we make a decision one way or the other we should make it clear in the documentation.

I also very much agree that the changing the behavior of .contains is a backwards incompatible change that will need to go through our normal deprecation cycle. I think another weak argument in favor of .contains not considering visibility internally is that it is easier to explain the deprecation

This used to give False, in the future it will give True, silence this warning by doing obj.get_visible() and obj.contains(....) and get False on all versions of Matplotilb

seems easier to explain and document than

This used to give True, in the future it will give False. Set the visibility to True and set the alpha to 0 to get the same behavior.

It seems the worst bugs are not glaring problems, but mostly innocuous / locally sensible behaviors that interact in pathological ways!

tacaswell added this to the v3.7.0 milestone Sep 12, 2022

QuLogic modified the milestones: v3.7.0, future releases Jan 25, 2023

QuLogic added the status: needs comment/discussion needs consensus on next step label Jan 25, 2023

timhoffm mentioned this issue Oct 30, 2024

Compute area of path #16859

Open

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: .contains for some artists does not check for visibility #23875

[Bug]: .contains for some artists does not check for visibility #23875

rdgraham commented Sep 11, 2022

tacaswell commented Sep 12, 2022

rdgraham commented Sep 12, 2022

tacaswell commented Sep 12, 2022

timhoffm commented Sep 12, 2022

jklymak commented Sep 12, 2022

rdgraham commented Sep 13, 2022

tacaswell commented Sep 13, 2022

[Bug]: .contains for some artists does not check for visibility #23875

[Bug]: .contains for some artists does not check for visibility #23875

Comments

rdgraham commented Sep 11, 2022

Bug summary

Code for reproduction

Actual outcome

Expected outcome

Additional information

Operating system

Matplotlib Version

Matplotlib Backend

Python version

Jupyter version

Installation

tacaswell commented Sep 12, 2022

rdgraham commented Sep 12, 2022

tacaswell commented Sep 12, 2022

timhoffm commented Sep 12, 2022

jklymak commented Sep 12, 2022

rdgraham commented Sep 13, 2022

tacaswell commented Sep 13, 2022