Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Allow users to decide whether a vector graphics backend combines multiple images into a single image #4061

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Mar 6, 2015

Conversation

breedlun
Copy link
Contributor

@breedlun breedlun commented Feb 2, 2015

Currently, if multiple images are on a set of axes, matplotlib will combine them into a single image when the figure window is saved. This is fine when the figure is saved as a raster image, such as a PNG, but it may not be what the user wants when the figure is saved as a vector graphics file, such as a PDF.

Personally, I like to be able to open a PDF in Adobe Illustrator or Inkscape after I have saved the figure and move each image independently of the others. I frequently will save a sequence of, say 5, vertically stacked images, and then I will realize 3 months later that I want to horizontally stack them, stagger them, overlap them, or something else. Usually I am in a hurry to put together a presentation or a writeup for something. Rather than dig up the old code, and rewrite it to do staggered images, I would much rather just work in Adobe Illustrator to manipulate the images on the fly. I find the GUI is the way to go for quick, single time, operations. If find myself doing the same GUI operations over and over again, then I will go to the effort to code it up.

To accommodate this workflow, I propose to add the rcParams 'pdf.combine_images', 'svg.combine_images', and 'ps.combine_images' to permit users to decide whether they want the vector graphics backend to combine all images within a set of axes into a single image. I also propose to change 'renderer.option_nocomposite' to the clearer name 'renderer.option_combine_images'.

Note: As a side benefit, this PR will give users a viable workaround for issue 3918.

@tacaswell tacaswell added this to the v1.5.x milestone Feb 2, 2015
@breedlun
Copy link
Contributor Author

breedlun commented Feb 2, 2015

Did the Travis CI build fail because I did not rebuild the documentation? If so, should I just execute python make.py html from within the doc/ directory, and commit the changes?

@tacaswell
Copy link
Member

No, definitely do not do that. The issues seems to be a large number of coding standards violations:


======================================================================
FAIL: matplotlib.tests.test_coding_standards.test_pep8_conformance_installed_files
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_coding_standards.py", line 254, in test_pep8_conformance_installed_files
    expected_bad_files=expected_bad_files)
  File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_coding_standards.py", line 140, in assert_pep8_conformance
    assert_equal(result.total_errors, 0, msg)
AssertionError: Found code syntax errors (and warnings):
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_pdf.py:91:1: E302 expected 2 blank lines, found 1
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_pdf.py:93:75: W291 trailing whitespace
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_pdf.py:94:74: W291 trailing whitespace
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_pdf.py:98:77: W291 trailing whitespace
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_pdf.py:106:24: E251 unexpected spaces around keyword / parameter equals
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_pdf.py:106:26: E251 unexpected spaces around keyword / parameter equals
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_pdf.py:107:30: E251 unexpected spaces around keyword / parameter equals
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_pdf.py:107:32: E251 unexpected spaces around keyword / parameter equals
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_ps.py:89:1: E302 expected 2 blank lines, found 1
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_ps.py:91:75: W291 trailing whitespace
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_ps.py:92:74: W291 trailing whitespace
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_ps.py:96:77: W291 trailing whitespace
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_ps.py:104:24: E251 unexpected spaces around keyword / parameter equals
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_ps.py:104:26: E251 unexpected spaces around keyword / parameter equals
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_ps.py:105:30: E251 unexpected spaces around keyword / parameter equals
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_ps.py:105:32: E251 unexpected spaces around keyword / parameter equals
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_svg.py:56:1: W293 blank line contains whitespace
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_svg.py:57:1: E302 expected 2 blank lines, found 1
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_svg.py:59:75: W291 trailing whitespace
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_svg.py:60:74: W291 trailing whitespace
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_svg.py:64:77: W291 trailing whitespace
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_svg.py:72:24: E251 unexpected spaces around keyword / parameter equals
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_svg.py:72:26: E251 unexpected spaces around keyword / parameter equals
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_svg.py:73:30: E251 unexpected spaces around keyword / parameter equals
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_svg.py:73:32: E251 unexpected spaces around keyword / parameter equals
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/tests/test_backend_svg.py:76:1: E302 expected 2 blank lines, found 1
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/backends/backend_pdf.py:1558:1: W293 blank line contains whitespace
/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/matplotlib-1.5.dev1-py2.7-linux-x86_64.egg/matplotlib/backends/backend_pdf.py:1561:76: W291 trailing whitespace

----------------------------------------------------------------------
Ran 2 tests in 20.972s

ax.set_xlim(0, 3)
ax.imshow(Z, extent = [0, 1, 0, 1])
ax.imshow(Z[::-1], extent = [2, 3, 0, 1])
plt.rcParams['pdf.combine_images'] = False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could save the plot to an io.BytesIO buffer and then make sure there are more than one image command in the file. Then we've verified this actually works.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. How do I incorporate checking for more than one image command into the testing architecture? As it is now, I just added the image_comparison decorator to the test function. I don't see a similar decorator to check the contents of a text file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @mdboom is suggesting to grab the byte stream from the savefig call and use the sting tools to search for image commands.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. So I won't be searching a text file. Instead I will be searching through a very long string. After I have tested whether there are multiple image commands, then what do I do? The image_comparison decorator did all the work for me. Do I just add a cleanup decorator and do an assert?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think that would be enough.

@mdboom
Copy link
Member

mdboom commented Feb 2, 2015

Cool. 👍 to the idea.

One issue for discussion: Does it make sense to have three different rcParams for this (one for each backend), or would one be enough? I usually wish we had fewer rcParams, and I don't see a good reason for turning this on individually.

@breedlun
Copy link
Contributor Author

breedlun commented Feb 3, 2015

I don't see a reason to have one rcParam for each backend either. How about rcParam['vector_backends.combine_images']?

@mdboom
Copy link
Member

mdboom commented Feb 3, 2015

Since we already have an image category in rcParams, maybe use image.combine for the name. @tacaswell: Opinions/thoughts?

@tacaswell
Copy link
Member

Definitely 👍 on using the image namespace. combine seems a bit too generic, maybe image.vector_combine or image.vector_precomposite.

@breedlun
Copy link
Contributor Author

breedlun commented Feb 4, 2015

I think it is important to chose a name that makes it easy to remember what the option does. image.combine does not make it clear that the option only pertains to vector graphics backends. image.vector_combine and image.vector_precomposite sound like matplotlib is going to combine vectors. I've tried, but I can't seem to think of anything better that starts with image.

So, I guess I still like vector_backends.combine_images. I do understand the reluctance to make a new category, but I think there might be other options we might want to put under vector_backends. For example, the svg.image_noscale could be turned into vector_backends.scale_images, provided that someone adds the code to make it apply to all vector backends.

Alternatively, I could go back to just having three different rcParams. One for each backend...

@breedlun
Copy link
Contributor Author

breedlun commented Feb 6, 2015

The tests now use io.BytesIO to check if there are two images in the vector graphics file, as @mdboom suggested. The tests work fine in Python 2.6 and 2.7, but fail in Python 3.3 and 3.4. Is there a way to find out what's wrong without installing Python 3, numpy, and matplotlib?

fig.savefig(ps, format="ps")
ps.seek(0)
buff = ps.read()
assert buff.count(' colorimage') == 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The failure appears to be here, and similar locations below and in the svg test. I think ' colorimage' needs to be bytes, not a string. I think you can wrap it with six.b().

@breedlun
Copy link
Contributor Author

breedlun commented Feb 7, 2015

OK. The tests now verify that there are two images in the vector graphics files, and the tests pass on Python 2.6, 2.7, 3.3, and 3.4. I also have fixed the PEP8 violations in the files I was working with, regardless of whether I created the violations or not.

The only thing left to decide is what to call the rcParam option. As I said in my earlier comment, I am in favor of vector_backends.combine_images. However, if y'all disagree, then I will be happy to call it something else. If you would like to me to switch it back to pdf.combine_images, svg.combine_images, and ps.combine_images, then I have no problem with that either.

@mdboom
Copy link
Member

mdboom commented Feb 9, 2015

I think image.precomposite, along with a docstring, is probably the best. I know it only applies to vector backends, so may should include the word "vector", but then I think it actually gets more confusing.

Other than that, this looks good, though unfortunately it needs a rebase.

@breedlun
Copy link
Contributor Author

I compromised and called it image.combine_images. I also updated the docstring in matplotlibrc.template and rebased the PR. If this is acceptable, then I think we are good to go.

@tacaswell tacaswell modified the milestones: proposed next point release, next point release Feb 19, 2015
@breedlun
Copy link
Contributor Author

My PR has gone stale for the second time now. If I rebase, will this be merged or does it need further review?

@tacaswell
Copy link
Member

This fell off my radar so I don't know the current state (and don't have time this second to look at it), but it needs to be rebased before it can be merged either way.

@@ -1,3 +1,11 @@
2015-01-31 Added the rcParam 'vector_backends.combine_images' to permit users
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to update this with the new name

@WeatherGod
Copy link
Member

I really don't see the point in the API change. Sure, it is clearer, but that ship sailed awhile ago. It is a public method. If there are going to be API changes, they need to be noted, and compatibility needs to be provided. As for the name of the rcparam, I am fine with it.

@@ -258,11 +258,11 @@ def test_image_composite_background():
ax.set_axis_bgcolor((1, 0, 0, 0.5))
ax.set_xlim([0, 12])

@image_comparison(baseline_images=['image_composite_alpha'], remove_text=True)
def test_image_composite_alpha():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why rename these?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to make things consistent by using combine_images everywhere instead of image_composite.

@breedlun
Copy link
Contributor Author

Yes, I was trying to simply make things more clear. option_combine_images is easier to understand than option_image_nocomposite. Now, perhaps that ship has sailed as WeatherGod said, but I imagine virtually no users queried option_image_nocomposite because it was not something they could change. So I think it would do very little harm to switch to the clearer combine_images, especially now that the user can change the behavior using rcParams['image.combine_images'].

If you guys agree, then I will note the API change and add a depreciation notice to option_image_nocomposite (my bad for not doing that before). If you guys do not agree because you want to maintain strict backwards compatibility, then I will shut up, switch everything back, and instead of rcParams['image.combine_images'] I will call it rcParams['image.no_composite'].

@breedlun breedlun force-pushed the master branch 2 times, most recently from d3f416d to c254114 Compare February 27, 2015 16:50
@breedlun breedlun force-pushed the master branch 2 times, most recently from b357367 to 3a9a738 Compare February 27, 2015 17:12
@breedlun
Copy link
Contributor Author

Rebased a third time.

So, what's the verdict? Should I depreciate renderer.option_image_nocomposite (since probably no one queries that property) and stay with combine_images for the sake of clarity? Or should I maintain strict backwards compatibility and go back to image_no_composite?

@WeatherGod
Copy link
Member

Let's stick with backwards compatibility. I can easily negate a boolean in
my head. I don't see the need to change so much code just to negate a book.

Don't do what johnny-do-not-do does!
On Feb 27, 2015 12:16 PM, "Benjamin Reedlunn" [email protected]
wrote:

Rebased a third time.

So, what's the verdict? Should I depreciate
renderer.option_image_nocomposite (since probably no one uses that
option) and stay with combine_images for the sake of clarity? Or should I
maintain strict backwards compatibility and go back to the only
no_composite?


Reply to this email directly or view it on GitHub
#4061 (comment)
.

breedlun added 2 commits March 1, 2015 20:59
'ps.combine_images' to permit users to decide whether they want the
vector graphics backend to combine all images within a set of axes
into a single image.  (If images do not get combined, users can open
vector graphics files in Adobe Illustrator or Inkscape and edit each
image individually.)  Also changed 'renderer.option_nocomposite' to
the clearer name 'renderer.option_combine_images'.

Improved combine image tests to actually count the number of images in the vector graphics files.  Also replaced the rcParams 'pdf.combine_images', 'svg.combine_images', and 'ps.combine_images' with 'vector_backends.combine_images'

Corrected PEP8 coding standard violations

Updated 'combine_images' tests for PS and SVG backends to hopefully make them Python 3 compatible.

Fixed some PEP8 violations to leave the code cleaner than I found it.

Fixed PEP8 violations introduced in my effort to fix PEP8 violations in my last commit.

Changed the rcParam option name from 'vector_graphics.combine_images' to 'image.combine_images', and updated the matplotlibrc.template.

Update the backend tests with the new rcParam name 'image.combine_images'

Added the rcParams 'pdf.combine_images', 'svg.combine_images', and
'ps.combine_images' to permit users to decide whether they want the
vector graphics backend to combine all images within a set of axes
into a single image.  (If images do not get combined, users can open
vector graphics files in Adobe Illustrator or Inkscape and edit each
image individually.)  Also changed 'renderer.option_nocomposite' to
the clearer name 'renderer.option_combine_images'.

Improved combine image tests to actually count the number of images in the vector graphics files.  Also replaced the rcParams 'pdf.combine_images', 'svg.combine_images', and 'ps.combine_images' with 'vector_backends.combine_images'

Corrected PEP8 coding standard violations

Updated 'combine_images' tests for PS and SVG backends to hopefully make them Python 3 compatible.

Fixed some PEP8 violations to leave the code cleaner than I found it.

Fixed PEP8 violations introduced in my effort to fix PEP8 violations in my last commit.

Changed the rcParam option name from 'vector_graphics.combine_images' to 'image.combine_images', and updated the matplotlibrc.template.

Update the backend tests with the new rcParam name 'image.combine_images'
Reverted from the 'combine_images' terminology back to the 'composite_image' terminology.  With this terminology, we can keep the method name 'renderer.option_image_nocomposite()', and maintain backwards compatibility.
…a comment in 'test_image.test_image_composite_alpha()' to the 'composite_image' terminology.
@breedlun
Copy link
Contributor Author

breedlun commented Mar 2, 2015

@WeatherGod, thanks for the feedback. I have switched back to the composite_image terminology everywhere. This means that the renderer.option_image_nocomposite() method is back, and the rcParam is now called rcParams['image.composite_image']. When someone gets a moment, I would appreciate a code review, so we can get this thing merged before it needs another rebase.

@WeatherGod
Copy link
Member

Looks good to me. Just need a "whats_new" entry. One question... are changes going to be needed for the PGF backend?

@breedlun
Copy link
Contributor Author

breedlun commented Mar 3, 2015

I added a "whats_new" entry.

I don't think the PGF backend needs any changes. As far as I can tell, PGF is not a vector graphics format, so the PGF backend just saves the entire figure as a rasterized image. There is no point to saving each image individually with that backend.

@tacaswell, I think we are ready to merge this PR.

@@ -249,7 +249,7 @@ class Figure(Artist):

*suppressComposite*
For multiple figure images, the figure will make composite
images depending on the renderer option_image_nocomposite
images depending on the renderer option_combine_images
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change should be reverted (missed un-rename).

@tacaswell
Copy link
Member

For future reference, see https://github.com/matplotlib/matplotlib/tree/master/doc/users/whats_new for adding whats_new entries. By splitting the up into individual files we will hopefully avoid the documentation related rebase-thrashing.

…mposite'. Also moved the whats_new addition into a separate file.
@breedlun
Copy link
Contributor Author

breedlun commented Mar 5, 2015

Nice catch @tacaswell. The comment has been fixed. I also moved the whats_new entry over to the proper individual file.

@WeatherGod
Copy link
Member

Ok, this looks good to me. Thank you for bearing with us through this.

WeatherGod added a commit that referenced this pull request Mar 6, 2015
Allow users to decide whether a vector graphics backend combines multiple images into a single image
@WeatherGod WeatherGod merged commit 4f80cfd into matplotlib:master Mar 6, 2015
@tacaswell
Copy link
Member

Thanks! Glad we got this sorted out and merged.

@breedlun
Copy link
Contributor Author

breedlun commented Mar 7, 2015

Yay! Thank y'all for helping me learn the ropes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants