Migration to sphinx-gallery #8247

NelleV · 2017-03-09T02:02:03Z

This work is still in progress.

Right now, the documentation builds two gallery. The old one, which stays identical as on current master and a sphinx-gallery one. The sphinx-gallery one needs specific examples formatting, hence only includes so far a subset of our example.

I'll slowly update this PR with progress and build this on a local website to show how it looks. Once we decide it is "good enough", the switch can easily be done through a URL switch.

Here is the sphinx-gallery build

tacaswell

I still have reservations about needing to re-name all of the examples, but the benefits out weigh the cost.

phobson · 2017-03-24T04:32:04Z

doc-requirements.txt

@@ -12,3 +12,4 @@ ipython
 mock
 colorspacious
 pillow
+sphinx-gallery


we need to add graphviz to this list

unfortunately, we can't as this file is meant to be used by pip, and graphviz is not pip installable.

QuLogic · 2017-03-24T22:46:58Z

ci/travis/test_script.sh

@@ -30,7 +30,7 @@ if [[ $BUILD_DOCS == false ]]; then
  fi
 else
  cd doc
-  python make.py html -n 2
+  python make.py html


Does parallel build not work?

No, but ironically, the parallel build doesn't change the run time :D
We can easily activate the flags in sphinx-gallery to make it parallel-safe, but considering there is no gain at all, I haven't created the PR yet.
Actually making the code parallel requires more work, but should be doable.

I I have definitely seen good speed up on building the docs (as they are now) but have found you need to use ~2x the number of workers compared to the number of core.

QuLogic · 2017-03-24T22:49:04Z

doc/conf.py

@@ -54,6 +55,12 @@
                      "numpydoc to build the documentation.")

 try:
+    import sphinx_gallery
+except ImportError:
+    raise ImportError("No module named sphinx-gallery - you need to install "


The module is called sphinx_gallery.

QuLogic · 2017-03-24T22:49:21Z

doc/conf.py

+    import sphinx_gallery
+except ImportError:
+    raise ImportError("No module named sphinx-gallery - you need to install "
+                      "numpydoc to build the documentation.")


Need to install sphinx-gallery, not numpydoc.

QuLogic · 2017-03-24T22:51:04Z

doc/conf.py

+   # path to your examples scripts
+    'examples_dirs' : '../examples',
+    # path where to save gallery generated examples
+    'gallery_dirs'  : 'auto_examples'}


Can we not put it in gallery instead? Or if it were mpl_examples, then some of the lines that were changed would not have been necessary.

Whatever it becomes, I don't like the auto_ prefix; it's an internal concern and doesn't really make any sense to present that in the URL to a user from a semantic point of view.

It's the default and is a convention followed by many (if not all) the projects. I'd rather stick with it for consistency.
We can't rename this to mpl_examples as mpl_examples still exists.

I'm not sure those are compelling reasons. I went back through the blame 7 years and couldn't find any reasoning behind it (that's not to say that there wasn't one.) My reasoning here is that URLs should be discoverable, simple, and at least try to give some semantic meaning.

A user might go to matplotlib.org/gallery or matplotlib.org/examples directly (yes, there's Google, but let's assume they're not using it for some reason), but there's a 99% chance that no-one will spontaneously go to matplotlib.org/auto_examples.

And what does the "auto_" prefix mean? It means that the rst was automatically generated from the example Python code. This is a meaningless distinction to a user. When they see "auto", they see "automatically generated" aka "not hand-written" aka "not very good". I'm not going to claim that all of our examples are 100% clear, but they've all been hand-written in some fashion, and some of them are even quite amazing.

In fact, for that same reasoning, I've just talked myself into retracting the "mpl_examples" suggestion, since the "mpl_" prefix is redundant and meaningless in this context. Also, our existing site uses matplotlib.org/examples/ and matplotlib.org/gallery.html; I don't think we should break that.

For the time being, the folder cannot be neither "examples" nor "mpl_examples", and this will not change in the foreseeable future considering the state of our examples (unless we remove all examples that are not SG compatible).

It's not possible to rename the current examples/mpl_examples to make room for the gallery-based examples? I strongly dislike the auto_examples name as well--I've overridden that name for sphinx-gallery in my own projects.

Will this end up exposed in the URL?

If so, I think we should pick something either a) shorter (ex sg or ae) or b) something more meaningful.

I does show up in the URL, for example this is our gallery at MNE-python. It does have auto-examples in the page, though that's sorta become the convention with sphinx-gallery (same goes for scikit-learn: http://scikit-learn.org/stable/auto_examples/index.html).

I seem to remember conversations with folks that this was a non-trivial thing to change but I could be wrong...it's been working out well enough for us. If you want you could use an .rst file that contains the gallery itself, then the objects that the gallery links to would be in the auto_examples folder.

Nope, it's completely trivial to change: https://unidata.github.io/MetPy/examples/index.html

I'm going to argue one more time NOT to use auto_examples. Whether they're generated automatically or not is immaterial to the user, so we should NOT be bleeding that implementation detail of our documentation build procedure to the outside world.

It's not that trivial, because matplotlib somehow does I don't know what in the plot directive that makes this a pain in the butt to change. Which is why I asked for feedback a month ago.

FWIW I'll give this a shot to change the name to examples_gallery. It doesn't look completely trivial in the sense that there are links hard-coded all over the codebase. There are also some weird errors happening inside the plot directive that I can try to debug. I'll open a separate PR then if I can figure it out.

QuLogic · 2017-03-24T22:51:24Z

doc/conf.py

+    # path where to save gallery generated examples
+    'gallery_dirs'  : 'auto_examples'}
+
+plot_gallery = True


Blank line after.

QuLogic · 2017-03-24T22:52:40Z

doc/conf.py

+# Sphinx gallery configuration
+sphinx_gallery_conf = {
+   # path to your examples scripts
+    'examples_dirs' : '../examples',


Does this mean we can delete the mpl_examples symlink?

Not yet. Our plot directive writes into mpl_examples, and unfortunately we have a bit more work to do before getting rid of this (including more examples to migrate). This will be my next step.

QuLogic · 2017-03-24T23:53:14Z

Is there a way to make sphinx-gallery less verbose? It prints out that it's going to run an example, then that it ran the example and how long it took plus a blank line, then at the end, it prints out how long it took to run all the examples again. That's 4 lines for each example.

NelleV · 2017-03-25T01:00:01Z

I don't think there is a way to make sphinx-gallery less verbose. It also doesn't bother me: it's actually great for debugging.

- Added sphinx-gallery to doc/conf.y - Renamed all conflicting README.txt to README in our example folders

:D

choldgraf · 2017-04-08T16:38:42Z

Looking forward to seeing this get merged into the website! Once it's merged I can spend a little time trying to create more narrative-driven examples out of the ones that are there and generally improving them. Is there anything holding this back right now?

tacaswell · 2017-04-09T01:42:50Z

doc/_templates/index.html

@@ -75,7 +75,8 @@
  <p>Matplotlib tries to make easy things easy and hard things possible.
  You can generate plots, histograms, power spectra, bar charts,
  errorcharts, scatterplots, etc., with just a few lines of code.
-  For a sampling, see the <a href="{{ pathto('users/screenshots') }}">screenshots</a>, <a href="{{ pathto('gallery') }}">thumbnail</a> gallery,  and
+  For a sampling, see the <a href="{{ pathto('users/screenshots')
+  }}">screenshots</a>, <a href="{{ pathto('auto_examples/index') }}">thumbnail</a> gallery,  and


I assume the linebreak does not affect the templating?

No, it works fine

NelleV · 2017-04-10T01:42:18Z

Chris is taking over this work. I don't have time and the energy to continue working on matplotlib's migration to sphinx-gallery.

NelleV added the Documentation label Mar 9, 2017

NelleV mentioned this pull request Mar 11, 2017

DOC moved spines examples sphinx-gallery #8268

Merged

NelleV added the MEP: MEP12 gallery and examples improvements label Mar 11, 2017

NelleV changed the title ~~[WIP] Migration to sphinx-gallery~~ [MRG] Migration to sphinx-gallery Mar 23, 2017

tacaswell approved these changes Mar 23, 2017

View reviewed changes

tacaswell changed the title ~~[MRG] Migration to sphinx-gallery~~ [MRG+1] Migration to sphinx-gallery Mar 23, 2017

tacaswell approved these changes Mar 23, 2017

View reviewed changes

phobson reviewed Mar 24, 2017

View reviewed changes

phobson approved these changes Mar 24, 2017

View reviewed changes

QuLogic added this to the 2.1 (next point release) milestone Mar 24, 2017

QuLogic reviewed Mar 24, 2017

View reviewed changes

QuLogic mentioned this pull request Mar 25, 2017

Issue #1888: added in the \dfrac macro for displaystyle fractions #8151

Merged

NelleV mentioned this pull request Apr 4, 2017

Merged the fill_demo figures and changed the axes #8407

Closed

NelleV added 15 commits April 6, 2017 16:35

DOC starting the migration to sphinx

b918016

- Added sphinx-gallery to doc/conf.y - Renamed all conflicting README.txt to README in our example folders

DOC Set up sphinx-gallery in parallel of our own gallery

b110b1f

DOC added sphinx-gallery as a requirement

ba6d268

DOC Added readme files to include examples in SG

c908f98

DOC SG doesn't support parallel builds yet

c61c856

DOC lines, bars and markers is now full SG

eaaf182

DOC examples/statistics is fully SG

4f92501

DOC pie and polar charts is now full SG

40eab47

DOC subplots, axes and figures are now full SG

9738124

DOC added ticks and spines to sphinx-gallery

8f8d316

DOC image, contours and fields are fully SG compatible

55a5840

DOC showcase are fully SG compatible

9bf91a8

DOC text, labels and annotations are fully SG compatible

f50ab00

DOC tick and spines is now full SG compatible

7c79b12

DOC shapes and collections is now fully SG compatible

1ed25aa

NelleV added 6 commits April 6, 2017 16:36

DOC mplot3d is fully SG-compatible

8bc4597

DOC The gallery is now built with sphinx-gallery

d5583f8

:D

DOC all plots in the gallery now produce images

8f5f185

DOC API is now fully SG-compatible

9d805a1

DOC linking to auto_examples when possible

4a13444

DOC FIX nitpicks in the conf file

8b764e7

tacaswell reviewed Apr 9, 2017

View reviewed changes

NelleV closed this Apr 10, 2017

choldgraf mentioned this pull request Apr 10, 2017

Migration to sphinx-gallery #8456

Merged

dstansby added the status: duplicate label Apr 12, 2017

QuLogic changed the title ~~[MRG+1] Migration to sphinx-gallery~~ Migration to sphinx-gallery Apr 30, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migration to sphinx-gallery #8247

Migration to sphinx-gallery #8247

NelleV commented Mar 9, 2017 •

edited

Loading

tacaswell left a comment

phobson Mar 24, 2017

NelleV Mar 24, 2017

QuLogic Mar 24, 2017

NelleV Mar 25, 2017

tacaswell Apr 9, 2017

QuLogic Mar 24, 2017

QuLogic Mar 24, 2017

QuLogic Mar 24, 2017

NelleV Mar 25, 2017

QuLogic Mar 26, 2017

NelleV Mar 26, 2017

dopplershift Mar 27, 2017

tacaswell Apr 9, 2017

choldgraf Apr 9, 2017

dopplershift Apr 9, 2017

NelleV Apr 10, 2017

choldgraf Apr 10, 2017

QuLogic Mar 24, 2017

QuLogic Mar 24, 2017

NelleV Mar 25, 2017

QuLogic commented Mar 24, 2017

NelleV commented Mar 25, 2017

choldgraf commented Apr 8, 2017

tacaswell Apr 9, 2017

NelleV Apr 9, 2017

NelleV commented Apr 10, 2017

Migration to sphinx-gallery #8247

Migration to sphinx-gallery #8247

Conversation

NelleV commented Mar 9, 2017 • edited Loading

tacaswell left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

QuLogic commented Mar 24, 2017

NelleV commented Mar 25, 2017

choldgraf commented Apr 8, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

NelleV commented Apr 10, 2017

NelleV commented Mar 9, 2017 •

edited

Loading