The biggest con is that [...] the axes object we create is new so that cb.ax? is no longer cax. In fact cax == cb.ax.parent_axes.

Can we turn this around: make a new parent axes and inject cax as the inset?

Can we turn this around: make a new parent axes and inject cax as the inset?

I don't know how we would make it so cax.set_position and cax.set_xticks would both work, because in this implementation they are different axes.

I see, but this is mostly set_position vs. Everything else? In which case it might still be better to have everything else work?

My thinking is that the usual pattern is to not specify cax

cb = fig.colorbar(im, ax=axs[0, 1])  # Note cb.ax is type ColorAxes that we control, cb.parent_axes is the usual outline axes
cb.ax.set_yticks()

cb.ax.set_position()  # note this passes through to the parent axes, so work as before.

So cb.ax behaves exactly as before (if I haven't missed something).

Currently we do not support most tick methods on colorbar axes: e.g.

cbax = fig.add_axes([0.9, 0.1, 0.07, 0.8])
cb = fig.colorbar(im, cax=cbax)
cbax.set_yticks(np.arange(-2, 2, 0.2))

returns a UserWarning: UserWarning: Use the colorbar set_ticks() method instead. We can however, access the xaxis and yaxis properties of the parent, so what we are breaking is the relatively low-level:

cbax.yaxis.set_major_locator(mticker.FixedLocator(np.arange(-2, 2, 0.1)))

which will be a no-op.

Of course the alternate works fine:

cb.ax.yaxis.set_major_locator(mticker.FixedLocator(np.arange(-2, 2, 0.1)))

Maybe we could make an Axis subclass that just warned when the y or xaxis is accessed and no-ops for everything.

One thing we could do to be super clear is destroy cax so no methods work on it and it is removed from the axes stack. EDIT: no we can't, because it is a kwarg and hence immutable.

I don't see a clean way to cripple the parent axes (cax in cb = colorbar(im, cax=cax)) or point it to the compound axes that we store at cb.ax. We can't just change the pointer because it is an immutable argument to a kwarg. So, if the user uses a pattern as above, cax is not a good thing for the user to operate on.

While that is awkward, the benefit is that it allows a lot of things to work for cb.ax that previously did not work at all (and of course did not work for cax either).

• we can set all the locators
• we can change the scale.
• we can set the ticks with the usual methods and not lock them out.
• we can have both axes be labeled if the user wants.
  This will all allow other non-linear norms other than logarithmic, and more user-customizable locators.

Personally I feel that the slight API annoyance (you must use cb.ax and not cax if you want this thing to work as a colorbar) is a significant improvement over the existing situation, particularly as I expect most users are not positioning colorbars using cax=cax. (well, probably many are, but they probably are only doing so based on out-of-date stackoverflow recipes).

... about 30 minutes after the above, I came up with a reasonable way to do this in the latest commit:

cb = fig.colorbar(im, cax=cax)
assert cb.ax == cax

will return False, but they are functionally the same because assert cb.ax.__dict__ == cax.__dict__ returns True. So now anything the user may want to do to cb.ax should also work on their terribly modified cax.

This is acceptable - cax is only used to position the colorbar. In the current PR, if it is passed, we copy its position and its _axes_locator.

Your last commit looks like a lot of extra code and keeping track of things to make it "functionally" the same without being exactly the same. I'm wondering if you can put something in the docstring to make it clear that you're overwriting the Axes passed in with a ColorbarAxes instead of adding a bunch of code to try and maintain half of the status quo.

I guess I'd argue the new code is pretty straight forward though - basically, if the user passed the axes we need to do some extra bookkeeping, otherwise don't bother.

I suppose its possible that the user could manually pass us a gridspec colorbar, but at that point they are tying then selves in knots.

Another option here is whether you need to create an entirely new ColorbarAxes or not? Could you just add an attribute to the Colorbar itself to store the inset axes instead (cb.inner_ax)? I think that might simplify some areas to not have to access nested Axes attributes (cb.ax.inner_ax). That might allow you to keep cb.ax == cax == cb.ax.parent_axes (now removed). That is just a quick thought I had, so I might be missing something fundamental too.

The whole point of all this is to make cb.ax behave like a normal axes, rather than the mostly-broken thing it is now. So the methods we would normally call on an axes shouldn't be buried in cb.ax.inner_ax. (i.e. set_xscale...) Worse, we need to lock out those methods on cb.ax.parent because we don't want the user mucking with the outer axes by accident.

Where you have:

        inner_ax = outer_ax.inset_axes([0, 0, 1, 1])
        self.__dict__ = inner_ax.__dict__

        self.outer_ax = outer_ax
        self.inner_ax = inner_ax
        self.outer_ax.xaxis.set_visible(False)
        self.outer_ax.yaxis.set_visible(False)
        self.outer_ax.set_facecolor('none')
        self.outer_ax.tick_params = self.inner_ax.tick_params
        self.outer_ax.set_xticks = self.inner_ax.set_xticks
        self.outer_ax.set_yticks = self.inner_ax.set_yticks
        for attr in ["get_position", "set_position", "set_aspect"]:
            setattr(self, attr, getattr(self.outer_ax, attr))

I was wondering if you could create a new method instead that gets called on initialization to create your inner_ax? I think that you could just put your dispatching methods onto the new attribute axes, rather than needing a ColorbarAxes to do that work for you. Something like this:

    def _add_insest_axes(self):
         # self.ax is outer_ax

        inner_ax = self.ax.inset_axes([0, 0, 1, 1])
        self.__dict__ = inner_ax.__dict__
        self.inner_ax = inner_ax
        self.ax.xaxis.set_visible(False)
        self.ax.yaxis.set_visible(False)
        self.ax.set_facecolor('none')
        self.ax.tick_params = self.inner_ax.tick_params
        self.ax.set_xticks = self.inner_ax.set_xticks
        self.ax.set_yticks = self.inner_ax.set_yticks
        for attr in ["get_position", "set_position", "set_aspect"]:
            setattr(self, attr, getattr(self.ax, attr))

I was wondering if you could create a new method instead that gets called on initialization to create your inner_ax? I think that you could just put your dispatching methods onto the new attribute axes, rather than needing a ColorbarAxes to do that work for you. Something like this:

I guess stylistically I think that delineating it as a different type of axes would be helpful, and it may sprout its own methods.

Technically, your suggestions above won't work as written because we will have lost the outer-axes altogether, and we need to keep it and its methods, just make them basically private. i.e. we need something to continue to hold those methods.

As discussed, this needs tests of the new monkeying around with the parent....

Having spent a bit of time looking at the side idea (discussed during the call) of using an axis wrapper to fix the locator clipping issue (and solely that -- I'm not claiming it fixes anything else), it turns out one of the issues with that is that we don't use locator return values "as is": there's an additional clip-to-viewlims step which is performed by Axis._update_ticks (that's why Axis.get_majorticklocs can sometimes (for normal axes) return an additional tick out of bound on either side of the viewlims, which is another issue which has been discussed elsewhere...), so that would also need to know about the axis wrapper faked lims, and various tests based on cbar.ax.yaxis.get_majorticklocs() need to be rewritten as the ticklocs are not the actually drawn one anymore...

I may revisit this later, but probably will stash the idea for now. (See also polar._AxisWrapper, from where I stole the design.)

I'm wondering if you want to default userax to True instead of False..

I'm a little torn about this. Users really not really supposed to do this, and if they do, they should expect the axes to get subsumed. It was a problem because axes_grid calls Colorbar directly and when I made this default to True, it of course didn't work. I've made it False but I think its questionable what the default should be.

I somewhat agree with you on the default choice being questionable, but since you've gone through the whole process of trying to make the original cax passed in point to the new ColorbarAxes it seems to me like the default should be to try and monkey-patch that original axis to keep the original object alive. My guess is that if you pass cax into axes_grid, you lose that reference now, although axes_grid is a little bit different and I'm not sure that makes sense since it is really trying to create cax for you in the first place...

Would it be worth trying to deprecate the cax object being kept alive? You could put in the docstring that cax is only used for placement of a new ColorbarAxes, but should not be kept as a reference due to a new object being created.
If you want to have a reference to the axes you should do this instead:

cb = fig.colorbar(cax=cax)
cax = cb.ax

(FYI I can't resolve any of my previous comments, so feel free to resolve them yourself)

We discussed this on the dev call this week. Deprecating cax was definitely possible, and maybe what we will want to do. However in the meantime we would still need to do something with it, and having it do the wrong thing would be confusing. So something like the current setup is still necessary. Or poisoning the original cax.

The subtlety here is that cax can have a _locatable_axes method on it, and that should still work. Actually I should probably add a test that this still works.

Interestingly this example doesn't properly take into account the colorbar in the constrained layout. This will take a bit of investigation:

import matplotlib.pyplot as plt
import numpy as np

fig, ax = plt.subplots(constrained_layout=True)
pc = ax.imshow(np.arange(100).reshape(10, 10))
cax = ax.inset_axes([1.02, 0.1, 0.03, 0.8])
# cax.set_in_layout(True)
fig.colorbar(pc, cax=cax)

plt.show()

Thanks a lot @greglucas I've squashed the whole thing - I probably should get better at curating commits, but I usually mash them all together as I fix things.

Good work on this, @jklymak!

@jklymak this has caused some issues downstream in astropy (astropy/astropy#11800) with custom sub-classes of Normalize not locating the colorbar ticks properly. Off the top of your head, do you know if there's anything extra someone making a Normalize sub-class needs to now add to the new class? I couldn't find anything in the docs that went with this PR, but sorry if I missed something.

I think we need a color-bar-con-fab in the next month or so (presumably by taking over one of our standing meetings).

I mean this is why we have pre-releases...

But, I will discuss on your astropy issue, because it looks like there is some confusion on astropy's end. But happy to open a new issue here.