Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Migration to sphinx-gallery #8456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 29 commits into from
Apr 11, 2017
Merged

Conversation

choldgraf
Copy link
Contributor

This is a riff off of #8247 and all of the hard work that @NelleV put into getting the documentation building off of sphinx-gallery. As of the latest discussion, @dopplershift @QuLogic and @tacaswell suggested that it'd be better to use a more "human readable" folder for the examples. This PR is the same as Nelle's but it calls the generated examples "examples_gallery". So a URL might be:

matplotlib.org/examples_gallery/api/plot_font_family_rc.html

Do folks think that this is a decent name for the folder? Maybe a little long, but it's straightforward. @dopplershift was right in that it wasn't too tough to change the name, so I could go with something different if that's preferred.

@choldgraf choldgraf changed the title Sphinx gallery [mrg+1] Migration to sphinx gallery Apr 10, 2017
@choldgraf choldgraf changed the title [mrg+1] Migration to sphinx gallery [MRG+1] Migration to sphinx-gallery Apr 10, 2017
@anntzer
Copy link
Contributor

anntzer commented Apr 10, 2017

My vote is also for something simpler, i.e. either examples or gallery.

@choldgraf
Copy link
Contributor Author

choldgraf commented Apr 10, 2017

/examples/ is already taken. There's already 'matplotlib.org/gallery.html' but no gallery folder. The problem with gallery is that then we'd have both a matplotlib.org/gallery.html and a matplotlib.org/gallery/index.html file.

Most important is that we just agree on something and move forward.

@NelleV
Copy link
Member

NelleV commented Apr 10, 2017

How about auto_examples? :D

@anntzer
Copy link
Contributor

anntzer commented Apr 10, 2017

let's just have a vote.
+1 for gallery

@QuLogic
Copy link
Member

QuLogic commented Apr 10, 2017

I thought gallery.html was deleted since this PR stops using gen_gallery.py?

@choldgraf
Copy link
Contributor Author

choldgraf commented Apr 11, 2017

so your docs built fine using that command?

edit: just saw your "docs still building" bit, nvm!

@anntzer
Copy link
Contributor

anntzer commented Apr 11, 2017

you probably need to git clean -xdf too, old files sticking around cause build failures ("warning: document isn't included in any doctree")

@choldgraf
Copy link
Contributor Author

well...I can try to spend some time to do this tomorrow. If we know that those commands will work then I can just do it...otherwise it'll take some time and I've already burned my evening trying to get this to work w/ python :-/

@anntzer
Copy link
Contributor

anntzer commented Apr 11, 2017

Also throw in a

sed -i 's/-plot_/-/g' **/*.rst

I also did

-   # path to your examples scripts
-    'examples_dirs' : '../examples',
-    # path where to save gallery generated examples
-    'gallery_dirs'  : 'gallery'}
+    'examples_dirs': '../examples',
+    'filename_pattern': '\.py$',
+    'gallery_dirs': 'gallery'}

to conf.py but not sure setting filename_pattern was actually needed.

Now everything works.

Excluding the folders without a README, there is a grand total of ONE example for which s-g puts a placeholder (agg_oo.py).

And you need to .gitignore doc/modules.

    sed -i 's/\bplot_\([[:alnum:]_]*\.py\)/\1/g' **/*.py **/*.rst
    sed -i 's/-plot_/-/g' **/*.rst
    git checkout -- doc/users/github_stats.rst doc/users/changelog.rst
@choldgraf
Copy link
Contributor Author

@anntzer, thanks for the help! I just tried running your commands and I'm running into errors - is this something that will only work on linux? Since you have already done the renaming etc on your machine, want to just push that to your fork and open a PR against my branch? Then I can merge and we'll have it in here.

@anntzer
Copy link
Contributor

anntzer commented Apr 11, 2017

done

@choldgraf
Copy link
Contributor Author

Thanks @anntzer

let's see if the tests still pass!

@choldgraf
Copy link
Contributor Author

It looks like the doc builds aren't passing - there seem to be links that point to examples that start with plot_.

E.g.: https://travis-ci.org/matplotlib/matplotlib/jobs/221050520#L2282

I went through the travis build and found the missing links (they were all in the users folder embedded in tutorials etc). Just pushed these changes so let's see if that fixes it.

@choldgraf
Copy link
Contributor Author

It looks like the doc builds are passing now. I only changed a few links so I wonder if the other travis failures (which were passing before) are travis problems and not code problems. @anntzer or @dopplershift could you restart those failing builds?

@anntzer
Copy link
Contributor

anntzer commented Apr 11, 2017

This is a well known flaky test and we do not block on it. Assuming Appveyor and Travis/OSX pass we should be good to go.

@choldgraf
Copy link
Contributor Author

It looks like only that first travis test failed, all the rest passed!

(BTW, thank you again for making that PR on my branch...maybe it was easy for you but this would have taken me probably another several hours to figure out :-/ )

@anntzer
Copy link
Contributor

anntzer commented Apr 11, 2017

The failure can safely be ignored. Unless someone speaks up in the coming ~24hr, I will merge.

I think the lesson here is that what is hard for some may be easy for others, and that we are all willing to help. I honestly had been unable to see why such a big fuss was being made about the rename. A comment as simple as "we don't only need to rename a bunch of files [this is what I thought that was all about], but also to update the links everywhere -- that's a lot of work for me" would probably have caught my attention some time ago; it literally took me less than half an hour to fix that (most of which was spent running the doc builds, so I could actually do other stuff in the meantime -- say 10min actually writing code and skimming through diffs).

@choldgraf
Copy link
Contributor Author

This is a good point - I think it's always good to remember (especially with newer contributors) that things that sound easy to a dev will not sound easy to a contributor, particularly if they've already put in a lot of time into the PR.

And I think it's also important to make it clear if/when devs are willing to work with people on the PR code etc...there's definitely a power differential between the devs and a contributor, and in my experience they won't feel comfortable spontaneously asking devs to do some coding for them.

Finally I think it's also useful to have posts similar to the one you just made. Where somebody says something along the lines of "OK everybody I'm giving 24 hours for comment and unless there's a big concern, I'm going to merge". It sucks when the author is in PR limbo where it seems like everybody has approved, but nobody is pressing the green button (I am sure we've all been there :-) ). It's useful to give explicit decision points to ensure that things keep moving forward.

Either way, I appreciate people taking the time to give their input on the PR!! We're all volunteers and I think it's awesome that such a useful package as matplotlib exists and continues to improve.

@dstansby
Copy link
Member

Just built this locally, and it looks great!

I think there are some little tweaks that still need to be made, and as always lots of cleaning to be done with examples, but that should all be much easier now sphinx-gallery is up and running. Thanks to all who worked on this!

@dstansby dstansby merged commit a7e006a into matplotlib:master Apr 11, 2017
@dstansby dstansby changed the title [MRG+1] Migration to sphinx-gallery Migration to sphinx-gallery Apr 11, 2017
@dstansby dstansby added this to the 2.1 (next point release) milestone Apr 11, 2017
@dstansby
Copy link
Member

Putting this in 2.1, since it's a fairly big change. Am happy for someone else to backport though if they think it should go in earlier.

@tacaswell
Copy link
Member

pros of backporting:

  • will get new docs out faster
  • reduce odds of conflicts in the examples between master and 2.0.x

cons:

  • will be a pain to do as all of the other example related PRs will likely have to be tracked down and backported for this to backport cleanly

Leaning towards not.

@QuLogic
Copy link
Member

QuLogic commented Apr 11, 2017

We have, up to this point, never* backported a documentation PR that deletes (moving counts, too) examples.

* except by accident or extreme necessity

@tacaswell
Copy link
Member

That is a solid point, sold on on not backporting.

@choldgraf
Copy link
Contributor Author

that's great news, thanks all! I think the next step should be to try and get the remaining examples ported to sphinx-gallery compatibility. This could be done one of two ways:

  1. Manually, which means going through each example, specifying a title, and specifying a top-level description.
  2. Semi-automatically, which could mean just generating a title built automatically from the file name, and then adding a blank line after the title (per this issue that @anntzer opened which means we don't have to actually have a description for every example)

If people are OK with approach 2, then I could open a PR that tries to do this on one folder so we can see how well it works...what do people think about this? Is it worth giving it a shot to see if we can get something that looks good without manually going through like 500 examples?

@anntzer
Copy link
Contributor

anntzer commented Apr 12, 2017

I would favor the automatic approach for the "examples" gallery. They can always be moved later to the "gallery" gallery after manual curation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants