Add basic testing of interactive backends. #8660
Conversation
.travis.yml
Outdated
| # GUI toolkits are pip-installable only for some versions of Python. | ||
| # Moreover, wxPython is only pip-installable as a pre-release for now. | ||
| pip install pyqt5 || true | ||
| pip install --pre wxpython || true |
There was a problem hiding this comment.
But wxpython is in the list above?
There was a problem hiding this comment.
typo, removed (from the first list)
| if [[ $TRAVIS_OS_NAME != 'osx' ]]; then | ||
| export DISPLAY=:99.0 | ||
| sh -e /etc/init.d/xvfb start | ||
| fi |
There was a problem hiding this comment.
Travis docs suggest a sleep here.
There was a problem hiding this comment.
Except that we're going to build matplotlib before running any tests, so xvfb should have more than enough time to start :-)
There was a problem hiding this comment.
Didn't building already happen in install?
There was a problem hiding this comment.
Even then, just test collection should give us enough time.
|
It looks like wxPython is building from source and failing due to missing GStreamer. I was hoping we could use something like pytest-boxed for the individual backend tests, but that plugin seems all or nothing and doesn't appear to have much development. |
|
good catch, let's see whether wxpython's snapshot wheels work... |
|
I dropped wxpython, whose wheels appear not to work on 12.04 (due to a libpng version mismatch), so we're only left with tk and qt5agg for now (but that's already better than nothing IMO). edit: yay to improved coverage (https://codecov.io/gh/matplotlib/matplotlib/pull/8660/changes) |
|
I think appveyor is generally broken with unrelated TeX issues. |
| ("Tkinter", "tkagg"), | ||
| ("tkinter", "tkagg")]: | ||
| try: | ||
| __import__(module_name) |
There was a problem hiding this comment.
https://docs.python.org/3.6/library/functions.html#__import__ <- This suggests to use import lib instead, is there a reason not to here? I have a vague sense of this playing badly with pypi.
Importing multiple GUI frameworks into one process can still cause problems because some of them install something onto PyOS_InputHook, but given that within the context of the test suite we should never land in the repl I think this is ok.
https://docs.python.org/3.6/library/importlib.html#importlib.util.find_spec (for py3) and https://docs.python.org/2.7/library/imp.html#imp.find_module (for py2) might be a better option.
There was a problem hiding this comment.
I would have used importlib.import_module if we were Py3-only, but to be honest I had lost track of what Py2 supports. Anyways, I'll fix it.
I think GUI frameworks "should" only install an inputhook when their mainloop is started, which is not the case here. To be honest I'd rather not have to add version-specific code unless strictly necessary.
|
Nominally 👍 , would like a second opinion on if my concerns about not importing multiple GUIs is worth digging into any more. |
| environ["MPLBACKEND"] = backend | ||
| proc = Popen([sys.executable, "-c", _test_script], env=environ) | ||
| # Empirically, 1s is not enough on Travis. | ||
| assert proc.wait(timeout=5) == 0 |
There was a problem hiding this comment.
Appveyor on python 2.7 doesn't like timeout=5 for some reason - https://ci.appveyor.com/project/matplotlib/matplotlib/build/1.0.1635/job/d70betvulec3rx10#L1592
There was a problem hiding this comment.
because windows + py2 doesn't use subprocess32. I'm a bit surprised that the DISPLAY env variable is set on windows though.
There was a problem hiding this comment.
bah, let's ignore py2. that'll also let me handle @tacaswell's concern re: importing.
There was a problem hiding this comment.
a py2 issue was the source of the instigating bug 😈
I am 👍 on this being a py3 only test though.
|
i think the appveyor failure is spurious... |
|
The only trouble with this now is that locally I see a random window flashing during testing. |
|
Is that really a problem? I do want to trigger a draw through the gui event loop during the test. |
|
Right, I know; it's just a minor irk. |
Just checking that we can open a window and trigger an exit upon draw_event. The idea is to prevent issues such as #8563 (except that it wouldn't have prevented that specific issue as PyQt4 is not pip-installable... I haven't tried apt-installing GUI toolkits as that would only work for some specific python versions anyways).