Build docs with Circle CI #8893
Conversation
|
I've updated this PR to use CircleCI 2.0, though I did not finish adding the cache. The reason is that 2.0 builds are much faster than 1.0. It was previously about 30 minutes (slower than Travis), and now is <20 minutes (faster than Travis). Uploading build artifacts takes longer than all installation steps combined, so I think the cache will probably not be a huge benefit. We could investigate custom docker images for the additional apt packages if that seems like it's a problem. There's no longer any need to mess with parallelism as each environment is automatically run as a separate build (with old parallelism, each build step is executed in lock-step, which made builds even slower; Python 3.5 is faster than 2.7 by a bit.) |
|
My 2 cents: I think this is an awesome idea, thanks @QuLogic :-) I'll look through in a little bit, will put some thoughts here.
...more to come maybe :-) |
|
so AFAIK there's nothing in here that's going to break MPL...I think we should merge this (after the conflict is resolved) and see if it starts behaving the way we expect it to behave on other PRs etc. Does that make sense? |
This helps to provide a reasonable version number.
It's not available on jessie, the distro on which CircleCI's docker image is based.
The docker image does not appear to be running as root.
Add 'doc-' prefix so that it will be clear on the GitHub status list what these builds are doing.
Otherwise, random files might be left over, such as the old gallery.html that was removed when switching to sphinx-gallery.
|
I've added upload of the sphinx-gallery artifacts as well. What it's shown me is that CircleCI's recursive artifact upload is super super slow. So I wonder what's more convenient: 1) being able to browse built docs online, or 2) getting builds done quicker but having to download a full tarball to view the docs? Because recursive uploading currently takes almost 6 minutes (30% of our build time) whereas that tarball was created and uploaded in 1s. I did open a bug report with CircleCI, so hopefully the problem will be moot soon enough. In any case, this would still need an admin to toggle the switch (as you can see CircleCI is not leaving a status check here.) |
|
I think I turned circle on at the Matplotlib layer (https://circleci.com/gh/matplotlib/matplotlib/24 ?) and I can see it in the web-hooks. Power cycled this PR to see if it would get picked up. If not, I am 👍 on squash merging this PR as-is and seeing what happen. |
|
https://circleci.com/docs/1.0/fork-pr-builds/ <- key piece of information I think I have done this... |
|
Yeah I'm +1 on merging too (though I lack the ability to do so still) Probably worth revisiting the conversation about the time it takes etc in a few months to determine whether it's worth tweaking the setup. Though I could see it being quite useful for contributions to the docs, which often get bogged down in things like getting sphinx-gallery set up w/ docs building just so you can demo the example locally |
@choldgraf had mentioned this (in a PR or on gitter), and I mentioned this on gitter some time ago, but the response was less enthusiastic at that point. I think support has grown a bit for this move, so here's the work I've got so far to migrate.
The docs are built on Python 3
by default, and if a second container is enabled, it builds docs onand Python 2. If you are logged in, you can see the results by clicking the Artifacts tab and navigating to the index.html. If you are not logged in, I've added anechoat the end that should point to the built artifacts.This would need an admin to a) enable CircleCI,
b) enable the second container,c) set up the encrypted variables to upload devdocs (if we still want that).Note though that the deploy step is untested.This is currently building on CircleCI 1.0 infrastructure (which it appears most Python projects use), but I am investigating CircleCI 2.0. It requires a complete change in workflow, so I may not complete that soon.