Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add option html_sourcelink_suffix #2454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 5, 2016
Merged

Add option html_sourcelink_suffix #2454

merged 8 commits into from
Jul 5, 2016

Conversation

mgeier
Copy link
Contributor

@mgeier mgeier commented Apr 16, 2016

This is a slight variation of #2399.

This changes the current behavior because it keeps the original file extension on source links and it adds .txt on top of that (except if the extension is already .txt).

I will add documentation and tests soon.

@mgeier
Copy link
Contributor Author

mgeier commented Apr 22, 2016

I've added some documentation and tests.

Right after that, however, I've seen that there might be a problem in sphinx/themes/basic/static/searchtools.js_t, where '.txt' is just appended to item[0] (which I assume to be the base name of the source).

Any ideas how to fix this?
Is the actual source file extension available in this context?
Is it DOCUMENTATION_OPTIONS.FILE_SUFFIX?

@mgeier
Copy link
Contributor Author

mgeier commented Apr 23, 2016

I tried DOCUMENTATION_OPTIONS.FILE_SUFFIX and it turns out to be html, so that's not the right suffix to use here.

This PR breaks the search preview, any suggestions for how to fix this?

@mgeier
Copy link
Contributor Author

mgeier commented May 13, 2016

Can somebody please help me with this?

@tk0miya
Copy link
Member

tk0miya commented May 16, 2016

Okay, I'll take a look (maybe this weekend).

@tk0miya
Copy link
Member

tk0miya commented May 16, 2016

In tk0miya@4e7e34b, I passed the original filename including extension into HTML search script.

Note:

  • This works fine, but I did not test
  • Always adds .txt to source filename even if html_sourcelink_txt is False

@tk0miya
Copy link
Member

tk0miya commented May 16, 2016

This PR looks good to me than #2399.
But, IMO, html_sourcelink_suffix = '.txt' is more easy to understand what the option does.

@mgeier
Copy link
Contributor Author

mgeier commented May 17, 2016

@tk0miya Thanks a lot for your help, now the search preview works!

I changed the option name in 4408baa, as you suggested.

Now there are some errors on travis-ci, but I don't know what causes them. Any ideas?

@@ -350,9 +353,10 @@ def prune(self, filenames):
for wordnames in itervalues(self._title_mapping):
wordnames.intersection_update(filenames)

def feed(self, filename, title, doctree):
def feed(self, docname, filename, title, doctree):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd read the error logs on Travis CI roughly. It seems some TypeErrors come from this new argument docname.

...
  File "/home/travis/build/sphinx-doc/sphinx/tests/test_search.py", line 42, in test_wordcollector
    ix.feed('filename', 'title', doc)
TypeError: feed() takes exactly 5 arguments (4 given)
...
  File "/home/travis/build/sphinx-doc/sphinx/sphinx/builders/websupport.py", line 79, in write_doc_serialized
    self.index_page(docname, doctree, title)
  File "/home/travis/build/sphinx-doc/sphinx/sphinx/builders/html.py", line 730, in index_page
    self.indexer.feed(pagename, filename, title, doctree)
TypeError: feed() takes exactly 4 arguments (5 given)

And it might have caused the more errors on websupport module.
Could you check this first?

@mgeier
Copy link
Contributor Author

mgeier commented Jun 7, 2016

I've added a missing argument in test_search.py and re-based the commits.
The tests pass when I run them locally (Python 2 and 3), but travis-ci still shows errors.

Strangely, two of them are

TypeError: feed() takes 4 positional arguments but 5 were given

But @tk0miya extended this to 5 arguments. Is it possible that this somehow uses the old version?

I have no clue what the other errors are about ...

Any help?

@tk0miya
Copy link
Member

tk0miya commented Jun 8, 2016

I found another implementations of indexer.feed().

$ grep -r 'def feed' sphinx
sphinx/search/__init__.py:    def feed(self, filename, title, doctree):
sphinx/websupport/search/__init__.py:    def feed(self, pagename, title, doctree):
sphinx/websupport/search/nullsearch.py:    def feed(self, pagename, title, doctree):

BTW, the files under sphinx/websupport/search are used for search-adapters for websupport.
And this is an opend interface and able to extend by 3rd parties.
http://www.sphinx-doc.org/en/stable/web/searchadapters.html#sphinx.websupport.search.BaseSearch.feed

So we have to do following actions:

  • add argument to BaseSearch.feed() and make tests passed
  • update docs for search-adapters
  • Add fallbacks to HTML builder to support old search-adapters like following:
try:
    self.indexer.feed(pagename, filename, title, doctree)
except TypeError:
    # fallback for old search-adapters
    self.indexer.feed(pagename, title, doctree)

@tk0miya
Copy link
Member

tk0miya commented Jun 8, 2016

In addition, please check your environment before testing.
Some testcases might been skipped if tests passed in your env.
It seems the failed testcases depends on sqlalchemy package.

please use test-reqs.txt to build development environment. it will install sqlalchemy too.

@mgeier
Copy link
Contributor Author

mgeier commented Jun 8, 2016

Thanks for the hint with test-reqs.txt.

I've made the suggested changes in d27386c.

I'm not really sure if everything is correct, but at least the tests are passing now.

@mgeier
Copy link
Contributor Author

mgeier commented Jun 23, 2016

@tk0miya Is it OK now?

@tk0miya
Copy link
Member

tk0miya commented Jun 23, 2016

Sorry for late response. I don't have enough time now. I'll take a look in this weekend (maybe).

@mgeier
Copy link
Contributor Author

mgeier commented Jun 23, 2016

OK, thanks, take your time, there is no hurry.
I just wasn't sure if you had seen my last changes ...

@@ -256,7 +256,7 @@ var Search = {
displayNextItem();
});
} else if (DOCUMENTATION_OPTIONS.HAS_SOURCE) {
$.ajax({url: DOCUMENTATION_OPTIONS.URL_ROOT + '_sources/' + item[0] + '.txt',
$.ajax({url: DOCUMENTATION_OPTIONS.URL_ROOT + '_sources/' + item[5] + '.txt',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line always appends .txt. This should be replaced with html_source_suffix.

In addition, this script fails searching if source_suffix is .txt. searchtools.js expects foo.txt.txt, but sphinx generates foo.txt because of source_suffix equals to html_source_suffix. I think special treatment for .txt is not needed. Certainly .txt.txt is ugly, but it is simple to know the path to the original file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a trivial matter, source_suffix = ['.rst', '.txt'] causes the filename conflict. Both foo.rst and foo.txt are copied into foo.rst.txt and get conflicted. I know this is a spiteful example.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the hints, but I'm kinda lost.
How can I test this JavaScript search stuff locally?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I always uses python -m SimpleHTTPServer 8080 command. It invokes HTTP Server locally at 8080 port. so you can access the directory through http://localhost:8080/.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

I added a commit that only adds .txt if it isn't already there: 71e1aaf.

What's left to do is to change '.txt' to the value of html_sourcelink_suffix from conf.py.
How do I get this from inside the JavaScript function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I tried something: 71dd8bf.

It seems to work, but I don't know if that's the right way to do it ...

@tk0miya tk0miya added this to the 1.5 milestone Jun 24, 2016
@mgeier mgeier changed the title Add option html_sourcelink_txt Add option html_sourcelink_suffix Jun 25, 2016
@tk0miya
Copy link
Member

tk0miya commented Jun 26, 2016

Thank you for update. I confirmed this works fine.
LGTM!!

I will merge this if no objection.

@tk0miya tk0miya merged commit 31c6beb into sphinx-doc:master Jul 5, 2016
@mgeier mgeier deleted the html-sourcelink-txt branch July 5, 2016 16:54
mitya57 added a commit to mitya57/matplotlib that referenced this pull request Feb 16, 2017
With Sphinx 1.5, this is needed by searchtools.js to display the source
snippets (see sphinx-doc/sphinx#2454).

With earlier Sphinx versions, this is a no-op because the undefined variable
will evaluate to an empty string.
vitaut pushed a commit to fmtlib/fmt that referenced this pull request Apr 15, 2017
With Sphinx 1.5, this is needed by searchtools.js to display the source
snippets (see sphinx-doc/sphinx#2454).

With earlier Sphinx versions, this is a no-op because the undefined variable
will evaluate to an empty string.
tk0miya added a commit that referenced this pull request May 2, 2017
TimKam added a commit to sphinx-contrib/sphinx-pretty-searchresults that referenced this pull request May 13, 2017
Since version 1.5.0, Sphinx is adding the source file extension to the source files it includes in the output folder.
For example, when your source file extension is `rst` (specified in the config variable `source_suffix` as `[.rst]`, your index file appears in the output's source folder as `index.rst.txt`.
If you use a Sphinx version lower than 1.5.0, it appears as `index.txt`.
Here's the link to the PR that introduces the breaking change: sphinx-doc/sphinx#2454.

The solution adjust the file name/suffix based on the Sphinx version:

* If you use Sphinx >=1.5, the file name will be filename + source_file_ending + ".txt".
* If you use Sphinx < 1.5, the file name will be filename + ".txt".

The changes also add aconfig option that allows forcing a fallback to 2).
This will help folks who use this extension for a long time and whose templates evolved accordingly.
dbkinder added a commit to dbkinder/docs-theme that referenced this pull request Aug 22, 2017
Sphinx 1.5 introduced a problem explained in
sphinx-doc/sphinx#2454
that breaks the site search box.  This patch
(together with a conf.py patch in the zephyr repo
repairs things).

Jira: INF-136

Signed-off-by: David B. Kinder <[email protected]>
nashif pushed a commit to zephyrproject-rtos/docs-theme that referenced this pull request Aug 22, 2017
Sphinx 1.5 introduced a problem explained in
sphinx-doc/sphinx#2454
that breaks the site search box.  This patch
(together with a conf.py patch in the zephyr repo
repairs things).

Jira: INF-136

Signed-off-by: David B. Kinder <[email protected]>
nashif pushed a commit to zephyrproject-rtos/zephyr that referenced this pull request Aug 22, 2017
Sphinx 1.5 introduced changes that broke the
sphinx-supplied search (as explained in
sphinx-doc/sphinx#2454)

This patch, along with a patch in the docs-theme
zephyrproject-rtos/docs-theme#10
repairs things.

Jira: INF-136

Signed-off-by: David B. Kinder <[email protected]>
mfussenegger added a commit to crate/crate-docs-theme that referenced this pull request Mar 14, 2018
Updates the bundled searchtools.js to the version shipped with sphinx
1.6.5.

The search looked for files called `name.txt`, but in sphinx 1.5 the
filename generation changed, so the files are called `name.rst.txt`
(because we use `.rst` as extension in `crate/crate`)

See aslo: sphinx-doc/sphinx#2454
mfussenegger added a commit to crate/crate-docs-theme that referenced this pull request Mar 14, 2018
Updates the bundled searchtools.js to the version shipped with sphinx
1.6.5.

The search looked for files called `name.txt`, but in sphinx 1.5 the
filename generation changed, so the files are called `name.rst.txt`
(because we use `.rst` as extension in `crate/crate`)

See aslo: sphinx-doc/sphinx#2454
mfussenegger added a commit to crate/crate-docs-theme that referenced this pull request Mar 14, 2018
Updates the bundled `searchtools.js` to the version shipped with sphinx
1.6.5.

The search looked for files called `name.txt`, but in sphinx 1.5 the
filename generation changed, so the files are called `name.rst.txt`
(because we use `.rst` as extension in `crate/crate`)

See also: sphinx-doc/sphinx#2454
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants