Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Is it possible to use sphinx-gallery with a separate source directory? #115

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
emmanuelle opened this issue Mar 28, 2016 · 8 comments
Closed

Comments

@emmanuelle
Copy link
Contributor

First of all, many thanks to the developers for this great project!

I'm currently trying to use sphinx-gallery for the examples gallery of scikit-image (http://scikit-image.org/docs/dev/auto_examples/), so that scikit-image can benefit from the developments made in sphinx-gallery in the future.

However, I'm facing some problems with relative paths (that I detail below), and I'm wondering if some developers/users have already successfully used sphinx-gallery for projects that have a separate source directory (when you answer "yes" to the following question in sphinx-quickstart
" > Separate source and build directories (y/n) [n]: y")

I tried to reproduce the problem with a simple example, generated with sphinx-quickstart. I have this doc directory with the following structure

.>>> tree
├── build
├── make.bat
├── Makefile
└── source
    ├── conf.py
    ├── conf.py~
    ├── examples
    │   ├── plot_adapt_rgb.py
    │   └── README.txt
    ├── index.rst
    ├── _static
    └── _templates

And when I run make html, the following structure is generated (clipping some lines)

├── build
│   ├── doctrees
│   │   ├── auto_examples
│   │   │   ├── index.doctree
│   │   │   └── plot_adapt_rgb.doctree
│   │   ├── environment.pickle
│   │   └── index.doctree
│   └── html
│       ├── auto_examples
│       │   ├── index.html
│       │   └── plot_adapt_rgb.html
│       ├── _downloads
│       │   ├── plot_adapt_rgb.ipynb
│       │   └── plot_adapt_rgb.py
│       ├── genindex.html
│       ├── index.html
│       ├── objects.inv
│       ├── search.html
│       ├── searchindex.js
│       ├── _sources
│       │   ├── auto_examples
│       │   │   ├── index.txt
│       │   │   └── plot_adapt_rgb.txt
│       │   └── index.txt
│       
├── make.bat
├── Makefile
└── source
    ├── auto_examples
    │   ├── images
    │   │   ├── sphx_glr_plot_adapt_rgb_001.png
    │   │   ├── sphx_glr_plot_adapt_rgb_002.png
    │   │   └── thumb
    │   │       └── sphx_glr_plot_adapt_rgb_thumb.png
    │   ├── index.rst
    │   ├── plot_adapt_rgb_codeobj.pickle
    │   ├── plot_adapt_rgb.ipynb
    │   ├── plot_adapt_rgb.py
    │   ├── plot_adapt_rgb.py.md5
    │   └── plot_adapt_rgb.rst
    ├── conf.py
    ├── conf.py~
    ├── examples
    │   ├── plot_adapt_rgb.py
    │   └── README.txt
    ├── index.rst
    ├── _static
    └── _templates

Images don't get copied because the path in plot_adapt_rgb.rst does not exist

      .. image:: /../doc/source/auto_examples/images/sphx_glr_plot_adapt_rgb_001.png
            :scale: 47

(sphinx complains about it during the build).

I think the problem comes from this line of code in gen_gallery.py

gallery_dir = os.path.relpath(gallery_dir, app.builder.srcdir)

in https://github.com/sphinx-gallery/sphinx-gallery/blob/master/sphinx_gallery/gen_gallery.py#L85

This issue seems to be related to #50. In #50 it was said that relative paths were not a priority at the moment. However, it seems that a large fraction of major scientific Python packages use a separate source directory: numpy, scipy, mayavi, pandas, ipython have a source directory, while matplotlib or scikit-learn don't.

@emmanuelle
Copy link
Contributor Author

If there is no quick workaround that I could use on my side, I'd be happy to give a hand for a PR solving this issue, but I might need some help as I can't see an obvious solution.

@Titan-C
Copy link
Member

Titan-C commented Mar 28, 2016

  • How are you configuring sphinx-gallery in this setup of yours, and from which folder do you launch the build?
  • This is a tricky problem as app.builder.srcdir despite sphinx claiming is the location of conf.py it actually seems to be the directory sphinx-build is called. So dealing with relative and absolute paths is quite complex. Then absolute paths in the rst files have as root the source folder and not the filesystem root, complicating again dealing with paths.

@emmanuelle
Copy link
Contributor Author

Thanks for the quick answer!

For the configuration of sphinx-gallery, I have in source/conf.py:

sphinx_gallery_conf = {
    # path to your examples scripts
    'examples_dirs' : 'doc/source/examples',
    # path where to save gallery generated examples
    'gallery_dirs'  : 'doc/source/auto_examples',
    }

(I first tried without doc but then it couldn't find the examples dir, once again because of os.path.relpath(examples_dir, app.builder.srcdir)). And of course I added sphinx_gallery.gen_gallery to the list of extensions.

I launch the build from doc (the directory where the Makefile is), with make html. doc is the directory from which I executed the tree commands above, which contains all the architecture listed above.

@emmanuelle
Copy link
Contributor Author

I think I found a workaround, with a minor modification of gen_gallery.generate_gallery_rst: adding

os.chdir(app.builder.srcdir)

allows to get the right paths with os.path.relpath. I checked on minimal examples that this change makes possible to build the gallery for the two types of folder organization (with or without a source directory).

Should I submit a PR with this change, or could it be a problem to make this modification (of course, one can go back to the initial directory at the end of the function)?

@emmanuelle
Copy link
Contributor Author

Coming back to this issue: any opinion on whether the workaround I suggest is the way to go, or whether another approach should be used?

@Titan-C
Copy link
Member

Titan-C commented Apr 28, 2016

doing the os.chdir(app.builder.srcdir) looks harmless for now. I'm not entirely sure how to work on this with Sphinx as is know for now it behaves different to their specification. I also see Sphinx has released a new version recently. It might be relevant to test it with it too, things usually break with Sphinx updates.

@emmanuelle
Copy link
Contributor Author

See #119.

@lesteve lesteve closed this as completed May 12, 2016
@emmanuelle
Copy link
Contributor Author

It turns out that it is necessary to be located in the source directory for other tasks as well, as in doc_resolv._embed_code_links. It would be possible to add the same workaround (cd into source directory) as in #119, or maybe it would be preferable to go to the source directory from the beginning, that is in gen_gallery.setup.

What do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants