Testing python 2.6, 2.7, and 3.2 #948

msabramo · 2012-06-13T06:24:52Z

This is a sort of continuation/enhancement of the work in #940.

Result of tox on Ubuntu 11.10 (I installed Python 3.1 from source because I didn't see an apt-get package for it):

GLOB test summary 
  py26: commands succeeded
  py27: commands succeeded
  py31: commands succeeded
  py32: commands succeeded
  congratulations :)

For Travis CI results, see http://travis-ci.org/#!/msabramo/matplotlib/builds/1607970 -- 2.6, 2.7, 3.1, and 3.2 are all passing!

mdboom · 2012-06-13T11:57:55Z

Great. Does this obsolete #940? One last thing I'd like to see before merging this is a short description in the developer documentation about a) how to run the tests with tox locally, b) how to get the results from Travis and c) the basics of how to participate in Travis, understanding that the last should be primarily a link to documentation elsewhere.

pelson · 2012-06-13T14:25:51Z

Looks great. Any draw backs to the service? I definitely think we should give this a trial run on master.

pelson · 2012-06-13T14:27:52Z

.travis.yml

+
+install:
+  - pip install --use-mirrors nose numpy
+  - if [[ $TRAVIS_PYTHON_VERSION == '2.'* ]]; then pip install --use-mirrors PIL; fi


These two lines are my biggest concern. Presumably this is so that the CI server has the appropriate setup, but what happens is PIL & numpy already exist on the server?

Can you easily test with different versions of numpy?

pip install doesn't reinstall packages without the --upgrade option.

I didn't have enough exposure to pip to have confidence, so thanks for the clarification.

Note: For clarification, I read what you said to mean "even if the original install wasn't done by pip, no re-installation will be done".

Yes -- that's my understanding of how it works.

Also, this is for Travis and Travis starts with a completely clean environment every time it runs. Basically you have a fresh new VirtualBox VM sandbox for each build.

msabramo · 2012-06-13T15:29:50Z

IIRC, Travis CI doesn't support GitHub organizations for enabling repos in the Travis UI, so you might run into some trouble following their instructions at http://about.travis-ci.org/docs/user/getting-started/. IIRC, you can get around this by going to the repository in GitHub and going to Admin | Service Hooks, and manually pasting in a personal GitHub account and Travis CI token.

WeatherGod · 2012-06-13T16:36:36Z

doc/devel/coding_guide.rst

+
+Travis CI is already enabled for the main matplotlib GitHub repository:
+https://github.com/matplotlib/matplotlib/ -- for example, see:
+http://travis-ci.org/#!/matplotlib/matplotlib


These won't show up as links.

They do for me with Sphinx==1.1.3 and docutils==0.9

Hmm, must be a new feature. How about for consistency's sake with the rest of the documentation you provided?

Agreed. See ccce78f.

pelson · 2012-06-15T12:34:11Z

doc/devel/coding_guide.rst

+3.2, etc.) and even different Python implementations altogether (e.g.: CPython,
+PyPy, Jython, etc.)
+
+It's a good idea to run ``tox`` in your working directory before submitting a


I'm not sure whether this is good or not. How realistic is it to ask a first-time/infrequent contributor to run the tests over all python versions? Is such a statement going to put people off from contributing all together? Perhaps we could just encourage the use of TravisCI to do the version testing on behalf of the user to keep their lives as simple as possible.

I'd certainly like to make this recommendation, but I think agree with @pelson, that it might be discouraging for simple submissions. How about we just say something like "tox makes it easy to determine if your working copy introduced any regressions before submitting a pull request."

OK, I used this wording in f964e67.

mdboom · 2012-06-18T12:44:29Z

This documentation looks great. Just ran your instructions for running tox in a clean virtualenv and all works great. I think this is going to be very handy. I'll wait 24 hours to see if there's further comments, and barring that, I think this should be merged and have the service hook set up.

mdboom · 2012-06-18T13:14:35Z

One thing that would be nice to add to the documentation about tox.

"Testing all 4 versions of Python (2.6, 2.7, 3.1, and 3.2) requires having four versions of Python installed on your system and on the PATH. Depending on your operating system, you may want to use your package manager (such as apt-get, yum or MacPorts) to do this, or use pythonbrew <https://github.com/utahta/pythonbrew>_"

msabramo · 2012-06-21T07:35:31Z

Is this merge-ready now?

pelson · 2012-06-21T07:46:29Z

.travis.yml

+  - python setup.py install
+
+script:
+  - mkdir ../foo


Would be nice if we could have a better folder name. Presumably this is done to have somewhere to store the result images?

IIRC, that foo folder is simply so that we're not in the source directory because then Python 3 will import modules from there which have not been transformed by 2to3. Will definitely take ideas for better folder names or a better approach.

So pwd is the root of the repository, but if your in the root of the repository, when importing matplotlib via python3, you would actually be importing the source rather than the built version?
Perhaps it would be better to run python setup.py build which will make a clean build/lib* directory, or is it advised by travis to actually install the code that is being tested?

At the very least, this mkdir should have a comment to explain the reasoning, and perhaps rename the folder to test_results (I don't have much feeling for a good folder name either).

pelson · 2012-06-21T07:51:32Z

Whilst I haven't run this branch myself, the changes look sound, so it has my +1.

mdboom · 2012-06-21T14:10:31Z

I'm going to go ahead and merge this. Thanks for this, I think it is very handy.

msabramo · 2012-06-27T07:23:02Z

Looks like you haven't gotten around to merging this yet?

Testing python 2.6, 2.7, and 3.2

msabramo · 2012-06-28T00:39:13Z

Did you enable http://travis-ci.org/#!/matplotlib/matplotlib ?

pelson · 2012-06-28T06:51:15Z

Don't think I have permissions to do this. I was looking for a https://github.com/matplotlib/matplotlib/admin/hooks as per the guide at http://about.travis-ci.org/docs/user/how-to-setup-and-trigger-the-hook-manually/.

@mdboom: do you have admin privs for the mpl repo or organization?

msabramo · 2012-07-10T00:52:59Z

Did http://travis-ci.org/#!/matplotlib/matplotlib ever get set up?

pelson · 2012-07-16T19:52:54Z

@jdh2358: John, as I know you will have suitable permissions, please can you enable the travis hook on the matplotlib repository. Thanks!

pelson · 2012-08-03T22:45:15Z

@mdboom: We still need to set this up (it would have caught the tests failing). Are there other people with admin rights other than yourself?

mdboom · 2012-08-29T14:43:37Z

I'm looking into this now. I have the service hook set up, and I have flipped the switch on the matplotlib/matplotlib repo on Travis, but I'm still not having much luck getting the build to trigger. Now travis-ci.org seems to be non-responsive.

efiring · 2012-08-29T18:24:32Z

On 2012/08/29 4:43 AM, Michael Droettboom wrote:

I'm looking into this now. I have the service hook set up, and I have
flipped the switch on the matplotlib/matplotlib repo on Travis, but I'm
still not having much luck getting the build to trigger. Now
|travis-ci.org| seems to be non-responsive.

—
Reply to this email directly or view it on GitHub
#948 (comment).

It is building, but not successfully. Among other problems, it looks
like it is starting by trying to build numpy, and going through an
excruciatingly long process.

msabramo · 2012-08-30T14:18:04Z

Numpy takes a long time to build -- should probably have it install a pre-built numpy with apt-get or easy_install. Let me know if you need help.

mdboom · 2012-08-30T14:39:42Z

Can apt-get run on the travis buildbot? easy_install on a Linux box installs Numpy from source anyway, so that provides no real savings. (I know next to nothing about Travis).

msabramo · 2012-08-30T15:10:03Z

Yeah, there are definitely .travis.ymls that invoke apt-get and it's officially supported (try searching http://about.travis-ci.org/docs/user/build-configuration/ for "apt-get"). The question is whether the apt-get repo has a new enough version of numpy to use matplotlib and test everything that you want to test. If not, I guess you might be able to find an alternative repo or create your own. Here's an example .travis.yml that uses apt-get to install numpy and also uses easy_install to install nibabel. (it also does this weird trick where it pretends to be an erlang project rather than python so that they can create a custom python environment by hand -- you may or may not need to go that far -- I forget the exact details of when it's needed):

https://github.com/matthew-brett/nipy/blob/master/.travis.yml

pwuertz · 2012-08-31T09:22:54Z

I found a way to bypass the numpy compilation process. I set up a travis worker box and built binary dists of numpy for our testing environments (2.6, 2.7, 3.1, 3.2). The install section of travis.yml can now download and unzip numpy instead of compiling it.

install:
  - PY_VER=`python -c "import sys;print('%d.%d'%sys.version_info[:2])"`
  - wget https://github.com/downloads/pwuertz/travis-tests/travis_py${PY_VER}_numpy.tar.gz
  - sudo tar -C / -xf travis_py${PY_VER}_numpy.tar.gz

Setup and the travis testing results:
https://github.com/pwuertz/travis-tests
http://travis-ci.org/#!/pwuertz/travis-tests

pelson · 2012-08-31T09:59:57Z

Its a nice solution, but it sounds a bit flakey. I'm assuming we can't just use a pre-built versions of numpy from official sources? This is bound to be a common problem for other projects depending on numpy. Maybe its worth spinning up a whole repo which has pre-built travis compatible tar.gz files (matplotlib also being one for use with upstream projects) which has a stable process for creating these tar files in the first place (via a .travis-ci.yml file perhaps).

pwuertz · 2012-08-31T10:10:38Z

No, there are no pre-built versions for linux. I think ideally one would just use a travis worker to build all dependencies required for testing matplotlib, build binary dists taylored to the travis environment and upload these somewhere. Travis announced support for building artifacts which could aid this process, but right now there is no official way of getting data out of the worker processes.

pwuertz · 2012-08-31T11:38:36Z

Ok, I just talked to the guys in the irc chat. There is currently no way of securely uploading data from a worker process. So setting up a VM like I did for doing the work is apparently the only way at the moment.

msabramo mentioned this pull request Jun 13, 2012

Add support for tox (http://tox.testrun.org/) and Travis CI (http://travis-ci.org/) #940

Closed

pelson reviewed Jun 13, 2012
View reviewed changes

Add support for tox (http://tox.testrun.org/)

faa78e9

Add .travis.yml for Travis CI (http://travis-ci.org/)

94e61b0

WeatherGod reviewed Jun 13, 2012
View reviewed changes

Add sections to devel/coding_guide.rst on using Tox and Travis CI

ccce78f

pelson reviewed Jun 15, 2012
View reviewed changes

Tweaks to devel/coding_guide.rst suggested by @mdboom

f964e67

pelson reviewed Jun 21, 2012
View reviewed changes

mdboom added a commit that referenced this pull request Jun 27, 2012

Merge pull request #948 from msabramo/testing_py3

83471aa

Testing python 2.6, 2.7, and 3.2

mdboom merged commit 83471aa into matplotlib:master Jun 27, 2012

Uh oh!

Testing python 2.6, 2.7, and 3.2 #948

Testing python 2.6, 2.7, and 3.2 #948

Uh oh!

Conversation

msabramo commented Jun 13, 2012

Uh oh!

mdboom commented Jun 13, 2012

Uh oh!

pelson commented Jun 13, 2012

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msabramo commented Jun 13, 2012

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mdboom commented Jun 18, 2012

Uh oh!

mdboom commented Jun 18, 2012

Uh oh!

msabramo commented Jun 21, 2012

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pelson commented Jun 21, 2012

Uh oh!

mdboom commented Jun 21, 2012

Uh oh!

msabramo commented Jun 27, 2012

Uh oh!

msabramo commented Jun 28, 2012

Uh oh!

pelson commented Jun 28, 2012

Uh oh!

msabramo commented Jul 10, 2012

Uh oh!

pelson commented Jul 16, 2012

Uh oh!

pelson commented Aug 3, 2012

Uh oh!

mdboom commented Aug 29, 2012

Uh oh!

efiring commented Aug 29, 2012

Uh oh!

msabramo commented Aug 30, 2012

Uh oh!

mdboom commented Aug 30, 2012

Uh oh!

msabramo commented Aug 30, 2012

Uh oh!

pwuertz commented Aug 31, 2012

Uh oh!

pelson commented Aug 31, 2012

Uh oh!

pwuertz commented Aug 31, 2012