Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ENH: allow rst files to pass through #1071

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 14, 2023
Merged

ENH: allow rst files to pass through #1071

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c8b44fc
ENH: allow bare index.rst, and rst files to pass through
jklymak Feb 8, 2023
55d69c6
Rearrange the index file building logic to remove repeated code
jklymak Feb 9, 2023
755c265
MNT: refactor format toctree
jklymak Feb 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 50 additions & 8 deletions doc/configuration.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ file:
- ``nested_sections`` (:ref:`nested_sections`)
- ``api_usage_ignore`` (:ref:`api_usage_ignore`)
- ``show_api_usage`` (:ref:`show_api_usage`)
- ``copyfile_regex`` (:ref:`manual_passthrough`)

Some options can also be set or overridden on a file-by-file basis:

Expand Down Expand Up @@ -409,14 +410,14 @@ point to the directory containing ``searchindex.js``, such as
If you wish to do the same for ordinary RST documentation,
see :ref:`plain_rst`.

If you are using inheritance for your documented code and you are experience
wrong links in the sense that the links point to the base class of an object
instead of the child, the option ``prefer_full_module`` might solve your issue.
Have also a look at `the GitHub
If you are using inheritance for your documented code and you are experience
wrong links in the sense that the links point to the base class of an object
instead of the child, the option ``prefer_full_module`` might solve your issue.
Have also a look at `the GitHub
issue <https://github.com/sphinx-gallery/sphinx-gallery/issues/947>`__
implementing this option for more context.

To make this work in your documentation you need to include to the
To make this work in your documentation you need to include to the
configuration
dictionary within your Sphinx ``conf.py`` file::

Expand All @@ -429,8 +430,8 @@ dictionary within your Sphinx ``conf.py`` file::
]
}

In the above examples all modules matching the string ``'yourmodule.*+\d{4}'``
would use the full module name when creating the links. All other use the
In the above examples all modules matching the string ``'yourmodule.*+\d{4}'``
would use the full module name when creating the links. All other use the
(default) way of linking.

.. _references_to_examples:
Expand Down Expand Up @@ -1865,6 +1866,47 @@ each subsection, and a specific toctree for each subsection.
In particular, sidebars generated using these toctrees might not reflect the
actual section / folder structure.

.. _manual_passthrough:

Manually passing files
======================

By default, Sphinx-Gallery creates all the files that are written in the
sphinx-build directory, either by generating rst and images from a ``*.py``
in the gallery-source, or from creating ``index.rst`` from ``README.txt``
in the gallery-source. However, sometimes it is desirable to pass files
from the gallery-source to the sphinx-build. For example, you may want
to pass an image that a gallery refers to, but does not generate itself.
You may also want to pass raw rst from the gallery-source to the
sphinx-build, because that material fits in thematically with your gallery,
but is easier to write as rst. To accomodate this, you may set
``copyfile_regex`` in ``sphinx_gallery_conf``. The following copies
across rst files.

.. code-block:: python

sphinx_gallery_conf = {
...
'copyfile_regex': r'.*\.rst',
}

Note that if you copy across files rst files, for instance, it is your
responsibility to ensure that they are in a sphinx ``toctree`` somewhere
in your document. You can, of course, add a ``toctree`` to your
``README.txt``.

Manually passing ``index.rst``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

You can bypass Sphinx-Gallery automatically creating an ``index.rst`` from a
``README.txt`` in a gallery directory or subdirectory. If your
``copyfile_regex`` includes ``index.rst``, and you have an ``index.rst`` in the
gallery-source instead of the README, Sphinx-Gallery will use that instead of
the index it automatically makes. If you do this, you are responsible for
adding your own Sphinx ``toctree`` in that index (or elsewhere in your Sphinx
documentation) that includes any gallery items or other files in that
directory.

.. _show_api_usage:

Showing API Usage
Expand All @@ -1884,7 +1926,7 @@ are used in. This could be helpful for making a map of which examples to
look at if you want to learn about a particular module. Setting
``show_api_usage`` to ``False`` will not make any graphs or documentation
about API usage. Note, ``graphviz`` is required for making the unused and
used API entry graphs.
used API entry graphs.

.. _api_usage_ignore:

Expand Down
203 changes: 114 additions & 89 deletions sphinx_gallery/gen_gallery.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ def __call__(self, gallery_conf, script_vars):
'prefer_full_module': [],
'api_usage_ignore': '.*__.*__',
'show_api_usage': False,
'copyfile_regex': '',
}

logger = sphinx.util.logging.getLogger('sphinx-gallery')
Expand Down Expand Up @@ -397,7 +398,7 @@ def call_memory(func):
return gallery_conf


def get_subsections(srcdir, examples_dir, gallery_conf):
def get_subsections(srcdir, examples_dir, gallery_conf, check_for_index=True):
"""Return the list of subsections of a gallery.

Parameters
Expand All @@ -408,16 +409,25 @@ def get_subsections(srcdir, examples_dir, gallery_conf):
path to the examples directory relative to conf.py
gallery_conf : dict
The gallery configuration.
check_for_index : bool
only return subfolders with a ReadMe, default True

Returns
-------
out : list
sorted list of gallery subsection folder names
"""
sortkey = gallery_conf['subsection_order']
subfolders = [subfolder for subfolder in os.listdir(examples_dir)
if _get_readme(os.path.join(examples_dir, subfolder),
gallery_conf, raise_error=False) is not None]
subfolders = [subfolder for subfolder in os.listdir(examples_dir)]
if check_for_index:
subfolders = [subfolder for subfolder in subfolders
if _get_readme(os.path.join(examples_dir, subfolder),
gallery_conf, raise_error=False)
is not None]
else:
# just make sure its a directory
subfolders = [subfolder for subfolder in subfolders
if os.path.isdir(os.path.join(examples_dir, subfolder))]
base_examples_dir_path = os.path.relpath(examples_dir, srcdir)
subfolders_with_path = [os.path.join(base_examples_dir_path, item)
for item in subfolders]
Expand Down Expand Up @@ -447,6 +457,24 @@ def _prepare_sphx_glr_dirs(gallery_conf, srcdir):
return list(zip(examples_dirs, gallery_dirs))


def _format_toctree(items, includehidden=False):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I refactored this out because the lack of indentation was making it hard to follow.

"""Format a toc tree"""

st = """
.. toctree::
:hidden:
"""
if includehidden:
st += """
:includehidden:
"""
st += """

%s\n""" % "\n ".join(items)

return st


def generate_gallery_rst(app):
"""Generate the Main examples gallery reStructuredText

Expand Down Expand Up @@ -481,7 +509,6 @@ def generate_gallery_rst(app):
check_spaces_in_filenames(files)

for examples_dir, gallery_dir in workdirs:

examples_dir_abs_path = os.path.join(app.builder.srcdir, examples_dir)
gallery_dir_abs_path = os.path.join(app.builder.srcdir, gallery_dir)

Expand All @@ -502,49 +529,52 @@ def generate_gallery_rst(app):
include_toctree=False,
)

has_readme = this_content is not None
costs += this_costs
write_computation_times(gallery_conf, gallery_dir_abs_path, this_costs)

# We create an index.rst with all examples
# (this will overwrite the rst file generated by the previous call
# to generate_dir_rst)
index_rst_new = os.path.join(gallery_dir_abs_path, 'index.rst.new')
with codecs.open(index_rst_new, 'w', encoding='utf-8') as fhindex:
# :orphan: to suppress "not included in TOCTREE" sphinx warnings
fhindex.write(":orphan:\n\n" + this_content)

# Write toctree with gallery items from gallery root folder
if len(this_toctree_items) > 0:
this_toctree = """
.. toctree::
:hidden:

%s\n
""" % "\n ".join(this_toctree_items)
fhindex.write(this_toctree)

# list all paths to subsection index files in this array
subsection_index_files = []

for subsection in get_subsections(
app.builder.srcdir, examples_dir_abs_path, gallery_conf):
src_dir = os.path.join(examples_dir_abs_path, subsection)
target_dir = os.path.join(gallery_dir_abs_path, subsection)
subsection_index_files.append(
'/'.join([
'', gallery_dir, subsection, 'index.rst'
]).replace(os.sep, '/') # fwd slashes needed inside rst
)

(
subsection_index_path,
subsection_index_content,
subsection_costs,
subsection_toctree_filenames,
) = generate_dir_rst(
src_dir, target_dir, gallery_conf, seen_backrefs
)

if this_content:
# :orphan: to suppress "not included in TOCTREE" sphinx warnings
indexst = ":orphan:\n\n" + this_content
else:
# we are not going to use the index.rst.new that gets made here,
# but go through the motions to run through all the subsections...
indexst = 'Never used!'

# Write toctree with gallery items from gallery root folder
if len(this_toctree_items) > 0:
this_toctree = _format_toctree(this_toctree_items)

indexst += this_toctree

# list all paths to subsection index files in this array
subsection_index_files = []
subsecs = get_subsections(app.builder.srcdir,
examples_dir_abs_path, gallery_conf,
check_for_index=has_readme)
for subsection in subsecs:
src_dir = os.path.join(examples_dir_abs_path, subsection)
target_dir = os.path.join(gallery_dir_abs_path, subsection)
subsection_index_files.append(
'/'.join([
'', gallery_dir, subsection, 'index.rst'
]).replace(os.sep, '/') # fwd slashes needed in rst
)

(
subsection_index_path,
subsection_index_content,
subsection_costs,
subsection_toctree_filenames,
) = generate_dir_rst(
src_dir, target_dir, gallery_conf, seen_backrefs
)

if subsection_index_content:
# Filter out tags from subsection content
# to prevent tag duplication across the documentation
tag_regex = r"^\.\.(\s+)\_(.+)\:(\s*)$"
Expand All @@ -553,55 +583,50 @@ def generate_gallery_rst(app):
if re.match(tag_regex, line) is None
] + [''])

fhindex.write(subsection_index_content)

# Write subsection toctree in main file only if
# nested_sections is False or None, and
# toctree filenames were generated for the subsection.
if not gallery_conf["nested_sections"]:
if len(subsection_toctree_filenames) > 0:
subsection_index_toctree = """
.. toctree::
:hidden:

%s\n
""" % "\n ".join(subsection_toctree_filenames)
fhindex.write(subsection_index_toctree)
# Otherwise, a new index.rst.new file should
# have been created and it needs to be parsed
else:
_replace_md5(subsection_index_path, mode='t')

costs += subsection_costs
write_computation_times(
gallery_conf, target_dir, subsection_costs
)

# generate toctree with subsections
if gallery_conf["nested_sections"] is True:
subsections_toctree = """
.. toctree::
:hidden:
:includehidden:

%s\n
""" % "\n ".join(subsection_index_files)
# """ % "\n ".join(this_toctree_items + subsection_index_files)

# add toctree to file only if there are subsections
if len(subsection_index_files) > 0:
fhindex.write(subsections_toctree)

if gallery_conf['download_all_examples']:
download_fhindex = generate_zipfiles(
gallery_dir_abs_path, app.builder.srcdir
)
fhindex.write(download_fhindex)

if (app.config.sphinx_gallery_conf['show_signature']):
fhindex.write(SPHX_GLR_SIG)
indexst += subsection_index_content

# Write subsection toctree in main file only if
# nested_sections is False or None, and
# toctree filenames were generated for the subsection.
if not gallery_conf["nested_sections"]:
if len(subsection_toctree_filenames) > 0:
subsection_index_toctree = _format_toctree(
subsection_toctree_filenames)
indexst += subsection_index_toctree
# Otherwise, a new index.rst.new file should
# have been created and it needs to be parsed
elif has_readme:
_replace_md5(subsection_index_path, mode='t')

costs += subsection_costs
write_computation_times(
gallery_conf, target_dir, subsection_costs
)

# generate toctree with subsections
if gallery_conf["nested_sections"] is True:
subsections_toctree = _format_toctree(
subsection_index_files, includehidden=True)

# add toctree to file only if there are subsections
if len(subsection_index_files) > 0:
indexst += subsections_toctree

if gallery_conf['download_all_examples']:
download_fhindex = generate_zipfiles(
gallery_dir_abs_path, app.builder.srcdir
)
indexst += download_fhindex

if (app.config.sphinx_gallery_conf['show_signature']):
indexst += SPHX_GLR_SIG

if has_readme:
index_rst_new = os.path.join(gallery_dir_abs_path, 'index.rst.new')
with codecs.open(index_rst_new, 'w', encoding='utf-8') as fhindex:
fhindex.write(indexst)
_replace_md5(index_rst_new, mode='t')

_replace_md5(index_rst_new, mode='t')
if gallery_conf['show_api_usage'] is not False:
init_api_usage(app.builder.srcdir)
_finalize_backreferences(seen_backrefs, gallery_conf)
Expand Down
Loading