Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Migration to sphinx-gallery #8247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 21 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
b918016
DOC starting the migration to sphinx
NelleV Mar 8, 2017
b110b1f
DOC Set up sphinx-gallery in parallel of our own gallery
NelleV Mar 9, 2017
ba6d268
DOC added sphinx-gallery as a requirement
NelleV Mar 11, 2017
c908f98
DOC Added readme files to include examples in SG
NelleV Mar 11, 2017
c61c856
DOC SG doesn't support parallel builds yet
NelleV Mar 11, 2017
eaaf182
DOC lines, bars and markers is now full SG
NelleV Mar 11, 2017
4f92501
DOC examples/statistics is fully SG
NelleV Mar 11, 2017
40eab47
DOC pie and polar charts is now full SG
NelleV Mar 11, 2017
9738124
DOC subplots, axes and figures are now full SG
NelleV Mar 11, 2017
8f8d316
DOC added ticks and spines to sphinx-gallery
NelleV Mar 12, 2017
55a5840
DOC image, contours and fields are fully SG compatible
NelleV Mar 16, 2017
9bf91a8
DOC showcase are fully SG compatible
NelleV Mar 16, 2017
f50ab00
DOC text, labels and annotations are fully SG compatible
NelleV Mar 16, 2017
7c79b12
DOC tick and spines is now full SG compatible
NelleV Mar 16, 2017
1ed25aa
DOC shapes and collections is now fully SG compatible
NelleV Mar 17, 2017
8bc4597
DOC mplot3d is fully SG-compatible
NelleV Mar 19, 2017
d5583f8
DOC The gallery is now built with sphinx-gallery
NelleV Mar 19, 2017
8f5f185
DOC all plots in the gallery now produce images
NelleV Mar 19, 2017
9d805a1
DOC API is now fully SG-compatible
NelleV Mar 19, 2017
4a13444
DOC linking to auto_examples when possible
NelleV Mar 19, 2017
8b764e7
DOC FIX nitpicks in the conf file
NelleV Mar 25, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
build
# sphinx build directory
doc/_build
doc/auto_examples
# setup.py dist directory
dist
# Egg metadata
Expand All @@ -58,7 +59,6 @@ lib/matplotlib/mpl-data/matplotlibrc
# Documentation generated files #
#################################
doc/examples
doc/_templates/gallery.html
doc/users/installing.rst
doc/_static/matplotlibrc
doc/pyplots/tex_demo.png
Expand Down
1 change: 1 addition & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ install:
pip install -ve .

script: source ci/travis/test_script.sh

before_cache:
- rm -rf $HOME/.cache/matplotlib/tex.cache
- rm -rf $HOME/.cache/matplotlib/test_cache
Expand Down
2 changes: 1 addition & 1 deletion ci/travis/test_script.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ if [[ $BUILD_DOCS == false ]]; then
fi
else
cd doc
python make.py html -n 2
python make.py html
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does parallel build not work?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, but ironically, the parallel build doesn't change the run time :D
We can easily activate the flags in sphinx-gallery to make it parallel-safe, but considering there is no gain at all, I haven't created the PR yet.
Actually making the code parallel requires more work, but should be doable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I I have definitely seen good speed up on building the docs (as they are now) but have found you need to use ~2x the number of workers compared to the number of core.

# We don't build the LaTeX docs here, so linkchecker will complain
touch build/html/Matplotlib.pdf
# Linkchecker only works with python 2.7 for the time being
Expand Down
1 change: 1 addition & 0 deletions doc-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,4 @@ ipython
mock
colorspacious
pillow
sphinx-gallery
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to add graphviz to this list

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately, we can't as this file is meant to be used by pip, and graphviz is not pip installable.

3 changes: 2 additions & 1 deletion doc/README.txt
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ All of these dependencies *except graphviz* can be installed through pip::

or all of them via conda and pip::

conda install sphinx numpydoc ipython mock graphviz pillow
conda install sphinx numpydoc ipython mock graphviz pillow \
sphinx-gallery
pip install colorspacious

To build the HTML documentation, type ``python make.py html`` in this
Expand Down
5 changes: 3 additions & 2 deletions doc/_templates/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@ <h1>Introduction</h1>
<p>Matplotlib tries to make easy things easy and hard things possible.
You can generate plots, histograms, power spectra, bar charts,
errorcharts, scatterplots, etc., with just a few lines of code.
For a sampling, see the <a href="{{ pathto('users/screenshots') }}">screenshots</a>, <a href="{{ pathto('gallery') }}">thumbnail</a> gallery, and
For a sampling, see the <a href="{{ pathto('users/screenshots')
}}">screenshots</a>, <a href="{{ pathto('auto_examples/index') }}">thumbnail</a> gallery, and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the linebreak does not affect the templating?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it works fine

<a href="{{ pathto('examples/index') }}">examples</a> directory</p>

<p>For simple plotting the <tt>pyplot</tt> module provides a
Expand All @@ -99,7 +100,7 @@ <h1>Documentation</h1>
</script>

<p>Trying to learn how to do a particular kind of plot? Check out
the <a href="{{ pathto('gallery') }}">gallery</a>, <a href="{{ pathto('examples/index') }}">examples</a>,
the <a href="{{ pathto('auto_examples/index') }}">gallery</a>, <a href="{{ pathto('examples/index') }}">examples</a>,
or the <a href="{{ pathto('api/pyplot_summary') }}">list of plotting
commands</a>.</p>

Expand Down
2 changes: 1 addition & 1 deletion doc/_templates/layout.html
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ <h3>{{ _('Navigation') }}</h3>

<li><a href="{{ pathto('index') }}">home</a>|&nbsp;</li>
<li><a href="{{ pathto('examples/index') }}">examples</a>|&nbsp;</li>
<li><a href="{{ pathto('gallery') }}">gallery</a>|&nbsp;</li>
<li><a href="{{ pathto('auto_examples/index') }}">gallery</a>|&nbsp;</li>
<li><a href="{{ pathto('api/pyplot_summary') }}">pyplot</a>|&nbsp;</li>
<li><a href="{{ pathto('contents') }}">docs</a> &raquo;</li>

Expand Down
22 changes: 19 additions & 3 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,10 @@
extensions = ['matplotlib.sphinxext.mathmpl', 'sphinxext.math_symbol_table',
'sphinx.ext.autodoc', 'matplotlib.sphinxext.only_directives',
'sphinx.ext.doctest', 'sphinx.ext.autosummary',
'matplotlib.sphinxext.plot_directive',
'sphinx.ext.inheritance_diagram',
'sphinxext.gen_gallery', 'sphinxext.gen_rst',
'sphinx_gallery.gen_gallery',
'sphinxext.gen_rst',
'matplotlib.sphinxext.plot_directive',
'sphinxext.github',
'numpydoc']

Expand All @@ -53,6 +54,12 @@
raise ImportError("No module named numpydoc - you need to install "
"numpydoc to build the documentation.")

try:
import sphinx_gallery
except ImportError:
raise ImportError("No module named sphinx_gallery - you need to install "
"sphinx_gallery to build the documentation.")

try:
import colorspacious
except ImportError:
Expand Down Expand Up @@ -96,6 +103,16 @@

autodoc_docstring_signature = True


# Sphinx gallery configuration
sphinx_gallery_conf = {
# path to your examples scripts
'examples_dirs' : '../examples',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean we can delete the mpl_examples symlink?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not yet. Our plot directive writes into mpl_examples, and unfortunately we have a bit more work to do before getting rid of this (including more examples to migrate). This will be my next step.

# path where to save gallery generated examples
'gallery_dirs' : 'auto_examples'}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we not put it in gallery instead? Or if it were mpl_examples, then some of the lines that were changed would not have been necessary.

Whatever it becomes, I don't like the auto_ prefix; it's an internal concern and doesn't really make any sense to present that in the URL to a user from a semantic point of view.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the default and is a convention followed by many (if not all) the projects. I'd rather stick with it for consistency.
We can't rename this to mpl_examples as mpl_examples still exists.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure those are compelling reasons. I went back through the blame 7 years and couldn't find any reasoning behind it (that's not to say that there wasn't one.) My reasoning here is that URLs should be discoverable, simple, and at least try to give some semantic meaning.

A user might go to matplotlib.org/gallery or matplotlib.org/examples directly (yes, there's Google, but let's assume they're not using it for some reason), but there's a 99% chance that no-one will spontaneously go to matplotlib.org/auto_examples.

And what does the "auto_" prefix mean? It means that the rst was automatically generated from the example Python code. This is a meaningless distinction to a user. When they see "auto", they see "automatically generated" aka "not hand-written" aka "not very good". I'm not going to claim that all of our examples are 100% clear, but they've all been hand-written in some fashion, and some of them are even quite amazing.

In fact, for that same reasoning, I've just talked myself into retracting the "mpl_examples" suggestion, since the "mpl_" prefix is redundant and meaningless in this context. Also, our existing site uses matplotlib.org/examples/ and matplotlib.org/gallery.html; I don't think we should break that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the time being, the folder cannot be neither "examples" nor "mpl_examples", and this will not change in the foreseeable future considering the state of our examples (unless we remove all examples that are not SG compatible).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not possible to rename the current examples/mpl_examples to make room for the gallery-based examples? I strongly dislike the auto_examples name as well--I've overridden that name for sphinx-gallery in my own projects.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this end up exposed in the URL?

If so, I think we should pick something either a) shorter (ex sg or ae) or b) something more meaningful.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I does show up in the URL, for example this is our gallery at MNE-python. It does have auto-examples in the page, though that's sorta become the convention with sphinx-gallery (same goes for scikit-learn: http://scikit-learn.org/stable/auto_examples/index.html).

I seem to remember conversations with folks that this was a non-trivial thing to change but I could be wrong...it's been working out well enough for us. If you want you could use an .rst file that contains the gallery itself, then the objects that the gallery links to would be in the auto_examples folder.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, it's completely trivial to change: https://unidata.github.io/MetPy/examples/index.html

I'm going to argue one more time NOT to use auto_examples. Whether they're generated automatically or not is immaterial to the user, so we should NOT be bleeding that implementation detail of our documentation build procedure to the outside world.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not that trivial, because matplotlib somehow does I don't know what in the plot directive that makes this a pain in the butt to change. Which is why I asked for feedback a month ago.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I'll give this a shot to change the name to examples_gallery. It doesn't look completely trivial in the sense that there are links hard-coded all over the codebase. There are also some weird errors happening inside the plot directive that I can try to debug. I'll open a separate PR then if I can figure it out.


plot_gallery = True
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blank line after.


# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

Expand Down Expand Up @@ -234,7 +251,6 @@
# Additional templates that should be rendered to pages, maps page names to
# template names.
html_additional_pages = {'index': 'index.html',
'gallery':'gallery.html',
'citing': 'citing.html'}

# If false, no module index is generated.
Expand Down
2 changes: 1 addition & 1 deletion doc/devel/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -442,7 +442,7 @@ We have hundreds of examples in subdirectories of
:file:`matplotlib/examples`, and these are automatically generated
when the website is built to show up both in the `examples
<../examples/index.html>`_ and `gallery
<../gallery.html>`_ sections of the website.
<../auto_examples/index.html>`_ sections of the website.

Any sample data that the example uses should be kept small and
distributed with Matplotlib in the
Expand Down
2 changes: 1 addition & 1 deletion doc/mpl_toolkits/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ plotting (scatter, surf, line, mesh) tools. Not the fastest or feature
complete 3D library out there, but ships with Matplotlib and thus may be a
lighter weight solution for some use cases.

.. plot:: mpl_examples/mplot3d/contourf3d_demo2.py
.. plot:: auto_examples/mplot3d/plot_contourf3d_2.py

.. _toolkit_axes_grid1:

Expand Down
38 changes: 19 additions & 19 deletions doc/mpl_toolkits/mplot3d/tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,41 +36,41 @@ Line plots
====================
.. automethod:: Axes3D.plot

.. plot:: mpl_examples/mplot3d/lines3d_demo.py
.. plot:: auto_examples/mplot3d/plot_lines3d.py

.. _scatter3d:

Scatter plots
=============
.. automethod:: Axes3D.scatter

.. plot:: mpl_examples/mplot3d/scatter3d_demo.py
.. plot:: auto_examples/mplot3d/plot_scatter3d.py

.. _wireframe:

Wireframe plots
===============
.. automethod:: Axes3D.plot_wireframe

.. plot:: mpl_examples/mplot3d/wire3d_demo.py
.. plot:: auto_examples/mplot3d/plot_wire3d.py

.. _surface:

Surface plots
=============
.. automethod:: Axes3D.plot_surface

.. plot:: mpl_examples/mplot3d/surface3d_demo.py
.. plot:: mpl_examples/mplot3d/surface3d_demo2.py
.. plot:: mpl_examples/mplot3d/surface3d_demo3.py
.. plot:: auto_examples/mplot3d/plot_surface3d.py
.. plot:: auto_examples/mplot3d/plot_surface3d_2.py
.. plot:: auto_examples/mplot3d/plot_surface3d_3.py

.. _trisurface:

Tri-Surface plots
=================
.. automethod:: Axes3D.plot_trisurf

.. plot:: mpl_examples/mplot3d/trisurf3d_demo.py
.. plot:: auto_examples/mplot3d/plot_trisurf3d.py


.. _contour3d:
Expand All @@ -79,18 +79,18 @@ Contour plots
=============
.. automethod:: Axes3D.contour

.. plot:: mpl_examples/mplot3d/contour3d_demo.py
.. plot:: mpl_examples/mplot3d/contour3d_demo2.py
.. plot:: mpl_examples/mplot3d/contour3d_demo3.py
.. plot:: auto_examples/mplot3d/plot_contour3d.py
.. plot:: auto_examples/mplot3d/plot_contour3d_2.py
.. plot:: auto_examples/mplot3d/plot_contour3d_3.py

.. _contourf3d:

Filled contour plots
====================
.. automethod:: Axes3D.contourf

.. plot:: mpl_examples/mplot3d/contourf3d_demo.py
.. plot:: mpl_examples/mplot3d/contourf3d_demo2.py
.. plot:: auto_examples/mplot3d/plot_contourf3d.py
.. plot:: auto_examples/mplot3d/plot_contourf3d_2.py

.. versionadded:: 1.1.0
The feature demoed in the second contourf3d example was enabled as a
Expand All @@ -102,37 +102,37 @@ Polygon plots
====================
.. automethod:: Axes3D.add_collection3d

.. plot:: mpl_examples/mplot3d/polys3d_demo.py
.. plot:: auto_examples/mplot3d/plot_polys3d.py

.. _bar3d:

Bar plots
====================
.. automethod:: Axes3D.bar

.. plot:: mpl_examples/mplot3d/bars3d_demo.py
.. plot:: auto_examples/mplot3d/plot_bars3d.py

.. _quiver3d:

Quiver
====================
.. automethod:: Axes3D.quiver

.. plot:: mpl_examples/mplot3d/quiver3d_demo.py
.. plot:: auto_examples/mplot3d/plot_quiver3d.py

.. _2dcollections3d:

2D plots in 3D
====================
.. plot:: mpl_examples/mplot3d/2dcollections3d_demo.py
.. plot:: auto_examples/mplot3d/plot_2dcollections3d.py

.. _text3d:

Text
====================
.. automethod:: Axes3D.text

.. plot:: mpl_examples/mplot3d/text3d_demo.py
.. plot:: auto_examples/mplot3d/plot_text3d.py

.. _3dsubplots:

Expand All @@ -146,5 +146,5 @@ in the same figure.
Subplotting 3D plots was added in v1.0.0. Earlier version can not
do this.

.. plot:: mpl_examples/mplot3d/subplot3d_demo.py
.. plot:: mpl_examples/mplot3d/mixed_subplots_demo.py
.. plot:: auto_examples/mplot3d/plot_subplot3d.py
.. plot:: auto_examples/mplot3d/plot_mixed_subplots.py
Loading