See the discussion in #2831.

I think the issue is that you are specifying the bounding box locations in display units. The transformation from data -> display takes into account things like the final dpi (which is one of the main reasons that the transform system exists).

In particular #2831 (comment) has a work-around that should solve your problem short of a complete overhaul of how the rendering backends work.

@tacaswell Thanks for the suggestion. The figure dpi and the savefig dpi were different and that fixed the png issue. (I have never understood why the figure dpi and savefig dpi do not default to the same values.) I tried to incorporate leejjoon's #2831 (comment) to fix the pdf problem, but I am not sure how to do it without an annotation. Doesn't the TransformedBbox that I used above do the same thing anyways?

The problem I am seeing might be the same as #2831, but I believe there is something different here. Check out the following example,

import matplotlib.pyplot as plt
import numpy as np
from matplotlib.image import BboxImage
from matplotlib.transforms import Bbox, TransformedBbox
import matplotlib as mpl
mpl.rcParams['savefig.dpi'] = mpl.rcParams['figure.dpi']

im = np.random.rand(10, 10) * 255.0

#Set up figure
fig = plt.figure()
ax = fig.add_axes([0.1,0.1,0.8,0.8])
ax.set_xlim(-2,24)
ax.set_ylim(0,10)

#Create an image that transforms automatically
bbox1 = Bbox.from_extents([0,0,10,10])
trans_bbox1 = TransformedBbox(bbox1, ax.transData)
bbox_image1 = BboxImage(trans_bbox1, data = im)
ax.add_image(bbox_image1)
#Save the result
fig.savefig('pdf_test_with_1_image.pdf')

#Add another image that transforms automatically
bbox2 = Bbox.from_extents([12,0,22,10])
trans_bbox2 = TransformedBbox(bbox2, ax.transData)
bbox_image2 = BboxImage(trans_bbox2, data = im)
ax.add_image(bbox_image2)
#Save the result
fig.savefig('pdf_test_with_2_images.pdf')

The pdf with only one image looks correct:

When I add a second image, the figure window looks like this
,
but when I save it to a pdf, it looks all wrong:

The first image gets moved, and the second image is gone. Something happens in the pdf backend when I have two Bbox images that does not occur when I have one Bbox image.

@tacaswell can I ask you to take a look at my post above? I think my issue is not quite a duplicate of issue #2831. When I use TransformedBbox with one image, the PDF looks fine. When I use TransformedBbox with two images, the PDF gets messed up.

I modified your example a little bit. I flipped the second image: bbox_image2 = BboxImage(trans_bbox2, data = im[::-1]). When running this example, we see that the second image is getting placed on top of the first. So, the second image is using the first image's transform (in the file, that is). I can also verify that this problem occurs when saving the results as 'eps' files.

@WeatherGod, thanks for confirming the bug. It appears that, no matter where I place the two images, they both get moved to the lower left corner of the axes when the figure is saved as a pdf.

For example, here is the matplotlib figure window:

and here is an image of the pdf file:

Oh, now that's interesting! It is as if none of the transform data is being
utilized!
On Jan 20, 2015 8:50 PM, "Benjamin Reedlunn" [email protected]
wrote:

@WeatherGod https://github.com/WeatherGod, thanks for confirming the
bug. It appears that, no matter where I place the two images, they both get
moved to the lower left corner of the axes when the figure is saved as a
pdf.

For example, here is the matplotlib figure window:
[image: fig_window]
https://cloud.githubusercontent.com/assets/4099759/5829823/f495047a-a0d4-11e4-929a-b5878ca7a7f4.png
and here is an image of the pdf file:
[image: pdf_test_with_2_images]
https://cloud.githubusercontent.com/assets/4099759/5829829/0a685f18-a0d5-11e4-9f78-efae0a45c7f5.png

—
Reply to this email directly or view it on GitHub
#3918 (comment)
.

attn @jenshnielsen

After an evening of digging, I believe I found the issue. In the draw method of axes/._base (lines 2029-2033), we have:

# add images to dsu if the backend support compositing.
# otherwise, does the manaul compositing  without adding images to dsu.
if len(self.images) <= 1 or renderer.option_image_nocomposite():
    dsu.extend([(im.zorder, im) for im in self.images])
    _do_composite = False
else:
    _do_composite = True

When the _do_composite flag is triggered then it combines all images on a set of axes into a single composite image. The reference position of the composite image is naturally at the lower left corner of the axes. _AxesBase then hands this reference position to the pdf (and presumably ps) backend, which then places the images at the lower left corner of the axes.

The _do_composite flag is not triggered if there is only one image, so that is why a single image does not get moved when I save as a pdf. I also confirmed that if I change the if statement to

if len(self.images) <= 100 or renderer.option_image_nocomposite():

then my two images also do not get moved when I save as a pdf.

So now, the question is, what should I do to fix this? I tried adding code to the pdf backend to make renderer.option_image_nocomposite() = True, but I get an error from MixedModeRenderer, because it has assert not vector_renderer.option_image_nocomposite(). This perplexes me. I never want my images to be combined into a single image. In fact, that is why I started playing around with BboxImage in the first place. I want to be able to open the pdf in Adobe Illustrator or Inkscape afterwards and move each image independently of the others. Why does the MixedModeRenderer prohibit this?

@mdboom, I noticed that you put originally put in assert not vector_renderer.option_image_nocomposite() to the MixedModeRenderer class back in 2007. Do you happen to remember why that is a requirement? I would prefer to not combine images when I save as a pdf.

My guess would be that this is for efficiency. The reason that the mixed-mode rendering exists (as I understand it) is so that you don't try to write a 1kx1k image into a 1inx1in region (that will eventually be rendered at 100dpi on the screen) as 10e6 square patches. By the same token, you don't want to render two large bit maps to be inserted into the vector document which will ultimately overlap. This works correctly if you use imshow + extent correct?

Another possible reason is that at least the ps backend does not deal with alpha correctly so if you have multiple images with alpha which overlap you need to composite them in AGG.

Why do you want to move the images around later? What use-case is that covering that can't be covered by mpl?

Thanks for the reply, tacaswell. The reasons you list for mixed-mode rendering make sense.

I like to move the images around later, because I frequently will save a sequence of, say 5, vertically stacked images, and then I will realize 3 months later that I want to horizontally stack them, stagger them, overlap them, or something else. Usually I am in a hurry to put together a presentation or a writeup for something. Rather than dig up the old code, and rewrite it to do staggered images, I would much rather just work in Adobe Illustrator to manipulate the images on the fly. I find the GUI is the way to go for quick, single time, operations. If find myself doing the same GUI operations over and over again, then I will go to the effort to code it up.

What about if I left option_image_nocomposite() = False by default, but I created a rcParam that allows the user to set option_image_nocomposite() = True? Following what is done in the SVG backend, I could add the following to the RenderPdf class

def option_image_nocomposite(self):
    """
    if pdf.image_nocomposite is True, compositing multiple images into one is prohibited
    """
    return rcParams['pdf.image_nocomposite']

Fair enough. This is just one more use-case where having proper serialization for figures (so you can just load the figure back up (data and all?!?) poke at the objects and re-save).

I am not super familiar with the mixed mode code (I read it a year ago, but it has exited the FIFO buffer of my memory), wouldn't the rcparam option still run afoul of the mixed mode assert? Why doesn't the SVG code run afoul of it?

The smash-down-to-one-image approach also leads to some really annoying z-order issues (if you have overlapping image, line, image in that order, pdf renders them as image, image, line) so it probably should be made configurable how the mixed-mode rendering deals with compositing (and make it smart enough to only smash down artists that are adjacent in the layer). That also side step this bug, fix an annoying long-standing bug, and solve your use-case.

The SVG backend does not run into trouble with the mixed mode assert because it does not use the MixedModeRenderer when option_image_nocomposite() = True. For the pdf backend, I would still like to use the MixedModeRenderer, even when option_image_nocomposite() = True, because it appears to do more than just smash-images-down-to-one.

According to the MixedModeRenderer class docstring, here is what it does:

"""
A helper class to implement a renderer that switches between
vector and raster drawing.  An example may be a PDF writer, where
most things are drawn with PDF vector commands, but some very
complex objects, such as quad meshes, are rasterised and then
output as images.
"""

I am thrilled it converts things like quad meshes to raster images, because Matlab did not, which was a pain in the rear.

As far as I can tell, the smashing-images-down-to-one actually seems to happen inside axes/._base right after the _do_composite flag gets defined. The downsampling may still happen in the stop_rasterizing method inside MixedModeRenderer.

To answer your question, yes, the rcParam option would still run afoul of the mixed mode assert. I would have to delete the assert, but I think that MixedModeRenderer would still do valuable things without it.

Also, if rcParams['pdf.image_nocomposite'] = False then bbox images would still get moved when images get smashed-down-to-one. I would still need to fix that issue too.

Well, I spent the evening trying to figure out why the if _do_composite: block (lines 2052-2084) in matplotlib/axes/._base.py moves BboxImages to the lower left corner of the axes, but it does not move AxesImages. I made very little headway. My one observation is, BboxImage and AxesImage both have a make_image method, but they are different. Perhaps AxesImage.make_image is doing something BboxImage.make_image should be doing. I just can't figure out what.

Don't feel discouraged! I have lost track of how many evenings I have spent failing to understand various bits of the code base. If you think you can clean anything you have been reading up a bit, please have at it (just make sure the test coverage is pretty good before you start).

I suspect that everywhere the code is assuming that someplace else will deal with the transform on the artist which means this probably going to be a very subtle bug which will require adding one or two lines to fix 😈 .

This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!

This now works, however I have not bisected back to when it was fixed.

@tacaswell how do you know this issue is resolved? When I try to run the code in my Dec 13, 2014 post, I get the following error: TypeError: 'image' must be an instance of matplotlib.image.AxesImage, not a matplotlib.image.BboxImage. I tried swapping out BboxImage for AxesImage, but that gave a different error.

I ran this with 3.6 when that was just a warning. Change ax.add_image -> ax.add_artist.

Thanks, tacaswell. Running the code in my Dec 13, 2014 post, except with ax.add_image -> ax.add_artist, allows me to confirm the issue is fixed with matplotlib 3.7. Yay! The first image moves ever so slightly when I save two images in the same set of axes, but I think it is negligible for most purposes.