I think that when the zoom button is on, I would rather have scroll = xy zoom, x+scroll = x zoom, y+scroll = y zoom. This may be nonstandard wrt. most UIs, but most UIs are optimized (I'd say) for scrolling vertically, which is not our use case.
(And then it's unclear whether anything should zoom when the pan button is on.)

It appears to zoom from the centre? Zoom should be around the mouse cursor. See, e.g., scroll zoom in Inkscape, or GIMP.

With multi-touch pads and fancy mice it is also pretty common to get both vertical and horizontal scroll, is there interesting interaction with that that should folded in?

also xref to https://gist.github.com/tacaswell/3144287 which was found and used by a number of people found, used, and reported issues against (in the comments). @mapfiable also pointed out that https://matplotlib.org/stable/api/_as_gen/matplotlib.axis.Axis.zoom.html#matplotlib.axis.Axis.zoom exists)

Thanks all for the input. From your responses I infer that we want to add scrolling interactions. Of course, lot's of details are still to be fleshed out.

@anntzer I hear your thoughts on button-to-interaction bindings. I've checked also more graphics-like UIs like Gimp, Inkscape, LibreOffice Impress, PowerPoint, Paint.Net. They all use the bindings that I've proposed above. So it seems a quasi-standard.

@QuLogic Sure, zooming will center on the cursor. I just was too lazy to figure out the maths for this proof-of-concept. 😄

@tacaswell Support for multi-touch and fancy mice would be a nice addition, but I'll make that a second priority. - I suppose we'll have to figure out how the individual UI frameworks handle these (if they can) and need to extend our event system for that.

A somewhat radical idea: Remove the Zoom and Pan tool buttons

Checking other programs, most of them do not have an explicit tool state for these operations, so do we need one?

Proposal

• Remove the Zoom and Pan buttons and instead add a single button that enables/disables interactions with the plot.

  I believe we still want the option to enable/disable manipulations of the plot. When enabled the following interactions are active:

• Mouse scroll behaviors as proposed above

• Click&Drag: Pan

• Ctrl + Click&Drag: Zoom Rectangle (Using the ctrl modifier is in analogy to Ctrl+Scroll for zooming)
  This is a change to the existing behavior, up to now

  • The Zoom rectangle had its own tool

  • Ctrl + Click&Drag is documented as

    Preserve aspect ratio | hold CONTROL when panning/zooming with mouse

    but implementations are different:

    • Zoom Mode: Ctrl has no effect
    • Pan Mode: Ctrl seems to restict panning to a diagonal (which doesn't make much sense).

    anyway, nobody has complained on this, so I'm not too worried changing it.

• additional x and y modifiers are kept.

• The colorbar interactions would be integrated similarly, and we don't have to argue if they are technically zoom/pan operations and should be enabled via the buttons. 😄

The big advantage is that users don't have to switch tools between zoom and pan, which can be tedious. Also these explicit tool concepts start to blur: e.g. should ctrl+scroll only work if the zoom tool is activated? This would seem overly restrictive.

1. I think most graphic guis have a little magnifying glass for zoom.
2. In my usage zoom is far more common than pan, so I'd argue click-drag should be zoom-in. Ctrl-click-drag is already zoom out, so perhaps shift-click-drag for pan. I'd have to look at some other programs for what pan is, but I'm 90% certain it's usually shift. The point is the less used option should require the modifier key.

We should also remember that there is more than one mouse button!

I tend to use the zoom tool only when I know exactly what data region I want to look at, and spend far more time with the pan tool turned on (using the left button to pan around and the right mouse button to zoom).

we still want the option to enable/disable manipulations of the plot

Certainly; users may have implemented other mouse-interaction interfaces which may otherwise interfere. (For example, I often bind "scroll" to "advance/go back along the time dimension of a movie".)

users don't have to switch tools between zoom and pan, which can be tedious

I tend to just 'o' and 'p' for that... I think "usability" here really depends on what keyboard shortcuts you have committed to your muscle memory; for example I would think that just pressing o/p once for zoom rect is much less tedious than having to keep the finger on ctrl (and I have never managed to remember the effect of ctrl/alt/... modifiers in inkscape, for example).

Hi, not that my opinion has any weight, but because @tacaswell mentioned me in a comment, I ended up subscribed to this post, and I thought maybe I can give you some valuable input:

General

I really like @timhoffm's suggestion to remove the zoom and pan tool buttons. They always felt clunky to me and seemed to be remnants from an era when zooming and panning (z&p) were still something novel. I would argue that nowadays, in the age of touchscreens, google maps and modern image viewers, z&p is an absolute default that everybody expects to be able to do without thinking. So I would even go a step further than @timhoffm and say that you should be able to z&p without having to press a button to toggle interactivity.
I understand the concern that the user might accidentally change the FOV, but I believe that 1) today's users have a good feel for z&p and should be able to easily correct for it (thought it's probably still annoying if it happens), and 2) that this would happen only very rarely. Think about e.g. how often you accidentally change the FOV when you use google maps or an image viewer, and about how big of a deal it is. After all, there is also always the home / default FOV button. Thus, instead of having to turn on z&p, I would suggest it to be active by default, and having a button to turn it off / lock the FOV. This could be useful e.g. if you need a very specific section that you want to save.
Additionally, I would propose a field where you would be able to enter the exact center of your FOV, as well as the FOV's dimensions, e.g. in the form of total width & height, or the positions of two opposite corners that span the FOV. I always missed this kind of precision.
Having the additional mouse wheel functionality à la Gimp, Inkscape, LibreOffice Impress etc. would be a nice extra, but I guess it would be overkill in 95% of use cases (I might be strongly extrapolating from my bubble here though). I think the mpl viewer is much closer related to, or at least perceived as, a simple image viewer, or google maps. In most cases, people only use it to look at (and save) static figures that don't have any intrinsic interactivity. I would argue for making z&p as simple and intuitive as possible.

Implementation

I have no idea how you will ultimately implement z&p, but I just wanted to comment on the popular zoom factory that @tacaswell came up with and that was subsequently slightly improved by other users. I adaped this approach for my own purposes and am very happy with it. For panning I am using something similar to what a user proposed here. There is also this approach which you may already be aware of.
The Axis.zoom method that I mentioned does not zoom centered on the cursor but centered on the center of the FOV. There is also a depreciation warning when using it (though you're probably aware of it):

MatplotlibDeprecationWarning: 
The zoom function was deprecated in Matplotlib 3.3 and will be removed two minor releases later.
  self.ax.get_xaxis().zoom(scale_factor)

In addition to Axis.zoom there is also Axis.pan, but when I tried it out, I never really managed to get it to work.

I remember that when I first implemented the z&p methods two years ago or so, I ran into some problems because they are manually changing the xlims and ylims of the axis. For some reason, I'm now unable to reproduce the issue for the life of me, but I believe it had something to do with autoscale to stop working once you manually changed the limits (or something similar). I definitely remember that there was a warning that popped up telling me as much. Is that something that has been fixed in the meantime? Or am I misremembering this completely? I only managed to find this bug report, and the typical issues you run into when adding artists to the axis or change the data of artists.

Lastly, I wanted to mention some performance issues that can occur during z&p. Maybe there is a smart way to address these in the implementation of z&p?

@mapfiable Thank you for your detailed comment! As we can already see in the few posts above there are varying opinions on different aspects of this, so it's valuable to get a broader view on this before making a specific design decision.

Since you're thinking about other behaviours, do note that there's an implementation of gesture support on touchscreens at #8041, which could translate to touchpad gestures.

I also would agree to have Google Maps behavior on mouse and touch control mode.

Simply scrolling means zooming, click and drag means panning. No keys required here.
I‘m almost always disappointed when the mouse wheel results in scrolling up and down in 2D canvasses.

In touch mode, two fingers for zooming, one finger for panning. I always feels weird when I need two fingers for panning.

just my 50 cents 🙂

I also would agree to have Google Maps behavior on mouse and touch control mode.

Simply scrolling means zooming, click and drag means panning. No keys required here.

Zooming absolutely needs a rectangle selector! scrolling=zoom works on google maps because the aspect ratio is always the same, that is not the case for most data plots. I wouldn't oppose a scrolling short cut, but click-and-drag should be the rectangle selector if we are going modeless.

The GUI interactions currently in Matplotlib have been around since the 80s. Maybe they could be improved, but I'd be pretty conservative about big changes.

Zooming absolutely needs a rectangle selector! scrolling=zoom works on google maps because the aspect ratio is always the same, that is not the case for most data plots.

Do you mean that in case of plots there can be a need to change the aspect ratio while for Google Maps that is not the case? Because if you simply want to zoom in and out of a plot, you typically don't want the aspect ratio to change. I agree that having a tool to change the aspect ratio can be very useful, and I'm not arguing against that, but I wouldn't make it the default. If you want to keep the rectangle selector I would suggest to toggle it by pressing and holding ctrl for example.

And just to throw another idea in the ring about changing the aspect ratio: what if you could control whether you want to zoom only vertically or horizontally by pressing and holding either ctrl or alt (e.g.), therby changing the aspect ratio?

if you simply want to zoom in and out of a plot, you typically don't want the aspect ratio to change

I would say this is extremely use-case dependent.

if you simply want to zoom in and out of a plot, you typically don't want the aspect ratio to change

I would say this is extremely use-case dependent.

Strongly agree - typically plots do not have a natural aspect ratio (think plot or scatter), and you want zoom to zoom independently in each co-oridinate. Again, maps and images are notable exceptions...

Because if you simply want to zoom in and out of a plot, you typically don't want the aspect ratio to change.

I agree. I‘m always annoyed when the rectangular zoom changes the aspect ratio. Never found that “feature” useful tbh. But YMMV.

I for one can live with different modes where the beginners mode should be the Google Maps mode and the traditional ones can remain as are.

I would also approach this issue with regards to adoption, acceptance and user-friendliness.

Maybe, one could explain what use-cases would require the “conservative” zooming modes?

and you want zoom to zoom independently in each co-oridinate

Here I would definitely love to see @mapfiable ’s idea where you can scroll and by doing so zoom one coordinate independently from the other.

@mapfiable maybe also consider that some scroll wheels do have horizontal scrollability.
When you look at 3D design programs such as inventor they can also integrate 3D mice which can handle more dimensions. But I guess this is a lot more work than required for something that would make the majority happy. 🙂

Maybe, one could explain what use-cases would require the “conservative” zooming modes?

Google Maps and CAD packages are very atypical of scientific plotting, and are indeed fixed-aspect ratio plots. However, except for the imshow examples, the overwhelming majority of our examples in the gallery do not have fixed aspect ratio, and benefit from being able to zoom on the axes with x and y limits chosen independently.

I think the choice of using mouse scroll to pan vertically is pretty arbitrary. It makes sense on text documents because that's frequently how they're laid out--text is wrapped to fit horizontally and you pan vertically to read more. I think for matplotlib's plots it doesn't feel natural to me. I think this jumps out in particular when you want to go to Axes3D (Edit: Also, I'm not sure about vertical scrolling panning only a single axes for a plot with multiple subplots).

To me, it seems completely reasonable to have mousewheel by itself zoom preserving aspect, and you need to press some modifier to limit zooming to a particular axis. I'm not coming at this necessarily from a position of "fixed aspect plots are normal", but more from a "principle of least surprise".

it seems completely reasonable to have mousewheel by itself zoom preserving aspect,

I agree with that - I mean if you are just scrolling a wheel, you don't have any indicator of aspect, so preserving the current aspect is the least-surprising thing to do.

To me, it seems completely reasonable to have mousewheel by itself zoom preserving aspect, and you need to press some modifier to limit zooming to a particular axis. I'm not coming at this necessarily from a position of "fixed aspect plots are normal", but more from a "principle of least surprise".

That would also be my understanding. I think that the combo "zooming with mouse wheel and panning with left click" (z&p) on the one hand and using a rectangle selector on the other (for simplicity I'll call this cropping in the following) simply have different mindsets / goals behind them.

• Z&p is more casual for when you simply want to get closer (or further away!) from something but don't care so much about the exact value ranges (I'll just call this FOV). It's much faster and fluent than cropping and a bit more direct / responsive, whereas cropping feels clunkier (imo). Zooming also has the option to quickly zoom out (again), while there is no corresponding option with cropping (except maybe for an undo method). I'm actually curious if something analogous could be designed for cropping. I was wrong, as @anntzer pointed out, there actually is a way:

  you can just crop with the right mouse button, which zooms out with the effect that the initial extents go to the extents of the rectangle you just drew.

  Again, I think zooming should keep the aspect ratio, e.g. if xlim = (0, 100); ylim = (0, 10) and the zoom factor is 10%, then after zooming in, your FOV reduced to xlim = (0, 90); ylim = (0, 9). This to me is the most intuitive. One question then would be e.g. if the zoom speed can increase when the mouse wheel is turned quickly (I believe this is the default in many modern applications but I really don't know; at least it feels natural to me). Another question might be if the zoom factor is always applied to the current or the default FOV, but I think using the current FOV is standard.
  Caveats that I see with z&p are the questions of how to deal with polar plots and probably more importantly log-scale axis. I'm not really sure what the intuitive response in these cases would be. At least with log-scale axis there is no ambiguity with cropping.

• Cropping is more precise but requires more thought. It is very useful when you have a pretty good idea about the FOV you would like to see, because it can get you much closer to it than z&p. Especially in scientific applications this can be very valuable where you might want to change the aspect ratio because it can help you to make better sense of the data. However, it can be very annoying if you "miss-click", i.e. if you accidentally crop the wrong FOV and have to go back (especially after a few cropping iterations).

Coming back to my previous suggestion to have a button you can use to toggle between the two modes. Instead of using a key for that it might be even more user-friendly / intuitive to simply use the right mouse button for cropping (and keeping left mouse button and mouse wheel for z&p as usual). Another idea would be to simply be able to use a keyword to pass your preferences to the figure / viewer instance. E.g. you could define whether you would like cropping to be the default and using ctrl to toggle z&p or the other way around. In addition, you could have a checkbox for that in the menu(bar).

Lastly, I wanted to mention two additional ideas that I've also seen in several other professional / scientific viewers:

• having an additional smaller plot window that is always zoomed in, say, 10% of the main plot, and is centered on the cursor, follwing it around. This can often eliminate the need for (extensive) zooming.
• having an overview "map", that is also a smaller plot which simply shows the outline of the default FOV and the current FOV as rectangle inside / overlapping with it. This can be useful to always see exactly where you are in the plot. You could even use it to quickly pan around in the plot by grabbing the small rectangle-representation of the current FOV and dragging it around on the map.

there is no corresponding option with cropping (except maybe for an undo method). I'm actually curious if something analogous could be designed for cropping.

Actually you can just crop with the right mouse button, which zooms out with the effect that the initial extents go to the extents of the rectangle you just drew...

there is no corresponding option with cropping (except maybe for an undo method). I'm actually curious if something analogous could be designed for cropping.

Actually you can just crop with the right mouse button, which zooms out with the effect that the initial extents go to the extents of the rectangle you just drew...

Oh wow, sorry, I didn't know that! My bad.

• I think zooming should keep the aspect ratio, e.g. if xlim = (0, 100); ylim = (0, 10) and the zoom factor is 10%, then after zooming in, your FOV reduced to xlim = (0, 90); ylim = (0, 9).

Additionally, I would also consider the position of the mouse pointer. In such a way that a user can zoom in on exactly the position where he or she points at (again like Google Maps).

So in the example above, the pointer was at (0, 0). Would it have been in the middle of the FOV, the resulting limits were (5, 95) and (0.5, 9.5) respectively.

Additionally, I would also consider the position of the mouse pointer.

Yes, I agree. Sorry I didn't make that clear.

I forgot to mention another benefit of cropping: you usually need fewer steps to get the FOV you had in mind (in most cases probably only one). This means you also need to redraw the figure less often, which can save a lot of time. With z&p, it can be nice to have look at all those intermediate steps between your starting point and where you want to go. But with longer drawing times these unnecessary steps can become annoying very quickly. E.g. the project I'm working on can have very many artists so drawing times can take up to 15secs some times. If I then try to zoom in but am unsure if something is happening due to the delayed reaction time, I can end up zooming in too much or getting lost so that I have to zoom out again (which takes time yet again). I guess this generally doesn't happen with cropping.

I was about to open an issue about it, but thought I may just as well put it here as there is a good bit of discussion already.

CAD and EDA tools quite often support "mouse gestures" using the middle mouse button. I find myself trying these out in quite a few programs and, well, it doesn't work. :-)

I think parts of this could be doable here though, maybe not as part of this PR, but I still want to mention it here as it may affect it.

There are probably eight different gestures to select from that are rather easy to detect: (horizontal, vertical and the two different diagonal) x direction. Of course we should try and adapt to some common "standard" if that is the case, not having bottom right to top left to be "zoom to fit" will be quite confusing to me, but probably this can also be made to be configurable.

Independent of this discussion: I hope that this PR can evolve and be merged to 3.6. Not having to click the buttons would be great!

Gestures are of course worth considering as an extension. However to me that would be second priority and we need to carefully check what common conventions are.

As for 3.6, I'm currently not having the time to go into this topic. If anybody want to pick this up, you are welcome to do so.

Minimal proposal: Scroll up/down without modifiers = zoom in/out to the cursor, keeping aspect ratio.

Direction: Consistent with zooming in maps, and browsers (while they additionally need a modifier, the up=magnification connection mathes)

Modifiers and panning can be discussed and added later.