Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Investigate alternative svg renderers for the test suite #8242

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
anntzer opened this issue Mar 8, 2017 · 8 comments
Closed

Investigate alternative svg renderers for the test suite #8242

anntzer opened this issue Mar 8, 2017 · 8 comments
Labels
topic: testing
Milestone
v2.1

Comments

@anntzer
Copy link
Contributor

anntzer commented Mar 8, 2017

Using rsvg as SVG renderer in the tests is much faster than using inkscape: with the following simple patch:

diff --git a/lib/matplotlib/testing/compare.py b/lib/matplotlib/testing/compare.py
index ab2f26fc0..a90a75901 100644
--- a/lib/matplotlib/testing/compare.py
+++ b/lib/matplotlib/testing/compare.py
@@ -138,9 +138,16 @@ def _update_converter():
         converter['eps'] = make_external_conversion_command(cmd)
 
     if matplotlib.checkdep_inkscape() is not None:
-        def cmd(old, new):
-            return [str('inkscape'), '-z', old, '--export-png', new]
-        converter['svg'] = make_external_conversion_command(cmd)
+        import cairo
+        from gi.repository import Rsvg
+        def convert(old, new):
+            svg = Rsvg.Handle().new_from_file(old)
+            svg.set_dpi(96)
+            surf = cairo.ImageSurface(
+                cairo.FORMAT_ARGB32, svg.props.width, svg.props.height)
+            svg.render_cairo(cairo.Context(surf))
+            surf.write_to_png(new)
+        converter['svg'] = convert
 
 
 #: A dictionary that maps filename extensions to functions which

the entire test suite runs in 594s instead of 787s on my laptop. (There's also a ton of image comparison failures, but that's because my cache (which I did not regenerate in either case) is generated by inkscape and there seems to be pixel-level differences between the two renderers -- but that won't be a problem in practice as all we'll want to do is to compare svgs rendered by the same engine.)

There are also a few constructs that are not correctly rendered by rsvg but perhaps it's a good opportunity to try to generate slightly less demanding svgs... and again in any case we're still comparing the same misrendering by the same engine :-)

The real problem right now is whether there is an easy way to install rsvg on Windows. There is an official .exe installer on Windows (https://wiki.gnome.org/action/show/Projects/PyGObject?action=show&redirect=PyGObject) which I haven't tried; I haven't found a pip package; there are a few user-submitted conda packages (search for pygobject/pygobject3) but none of them seem to work on Py3.

Alternatively, I have tried two other renderers: QtSVG (http://doc.qt.io/qt-5/qtsvg-index.html) and cairosvg (http://cairosvg.org/, which only supports Py3, but could be used as a standalone script that lives in its own environment, similarly to inkscape). QtSVG is a very, very limited SVG renderer so I'd rather not use it (unless we can make our svg output compatible with it); cairosvg seems to freeze(?) on at least one of our constructs.
@tacaswell
Copy link
Member

see #4341 for details about what rsvg gets wrong. That it does'nt actually render svg correctly seems like a sign we shouldn't use it, not a sign that we need to make the tests simpler.

https://github.com/NSLS-II/lightsource2-recipes/blob/master/recipes-tag/pygobject3-3.12.0/meta.yaml is a recipe for pygobject3 that works on py3 (and I am using it's build products in production at my day job).

My guess in that much of the time savings is in not restarting inkscape a zillion times. Is there a way to set inkscape up as a 'server'?

Another approach would be to make it easy to only run the png tests from the CLI (and maybe to put all of the mathtext tests in a 'slow' category).

@tacaswell tacaswell added this to the 2.1 (next point release) milestone Mar 8, 2017
@anntzer
Copy link
Contributor Author

anntzer commented Mar 8, 2017 •
edited
Loading

The issue you mentioned is actually no longer present, see #4341 (comment). Also note that Wikipedia's internal docs (https://en.wikipedia.org/wiki/Wikipedia:SVG_help#Missing_objects_or_black_boxes) suggest that some issues are really due to a difference in the specs between SVG 1.1 and 1.2.

Could that pygobject build be put on conda-forge or some other conda channel? Thanks :-)

There is an inkscape shell mode. #98 indicates a ~32% speedup but there seemed to be some deadlock issues that led to the reversion of that commit.

@tacaswell
Copy link
Member

Ah, I completely missed that rsvg got fixed, great!

No, but you are welcome to do so.

Probably worth re-investigating the deadlocks (as the comments are talking about py2.5 vs py2.7 changes) with inkscape shell mode.

@anntzer
Copy link
Contributor Author

anntzer commented Mar 8, 2017

I think it would be useful to mark the png/ps/svg tests using pytest markers (http://doc.pytest.org/en/latest/example/markers.html) so that one could just do e.g. pytest -m "not svg and not ps" (attn @QuLogic perhaps).

@anntzer anntzer mentioned this issue Mar 9, 2017
Merged
@QuLogic
Copy link
Member

QuLogic commented Mar 12, 2017 •
edited
Loading

You can sort of already do this; the extensions are used for parametrization, so tests get an id that's something like <test name>[<image index>-<image name>-<format>] which you can match using -k. Something like -k png works, or if you want to be a bit more specific -k 'png]'.

-k 'not svg and not pdf' appears to work, but I only checked that it collected without error and didn't see what actually ran.

@QuLogic
Copy link
Member

QuLogic commented Mar 12, 2017

For tests that use a specific backend directly, those are marked by pytest.mark.backend('name'), though there's no guarantee that they might produce a backend-specific file. Unfortunately, based on this SO question, it might not be possible to select those non-default backend tests by specific backend using a builtin method.

@anntzer
Copy link
Contributor Author

anntzer commented Mar 13, 2017

-k 'png]' is good enough in my opinion.

@anntzer
Copy link
Contributor Author

anntzer commented Apr 15, 2017

Closing for now as the inkscape batchmode patch already helps a lot; we can always revisit this once rsvg improves its compliance and pygobject gets packaged at least on c-forge.

@anntzer anntzer closed this as completed Apr 15, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
topic: testing
Projects
None yet
Milestone
v2.1
Development

No branches or pull requests
3 participants
@tacaswell @QuLogic @anntzer