Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 56f1a85

Browse files
felix-hildenfelix-hilden
authored
Support default lexer and more language configurations (felix-hilden#167)
* Support default lexer, literal blocks, highlight directive and highlight_language config (felix-hilden#166) * Better docs * Recover from default parse fail and add confval
1 parent 1a6a73a commit 56f1a85

32 files changed

+118
-19
lines changed

‎docs/src/about.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Caveats
1818
-------
1919
- **Only works with HTML documentation**, disabled otherwise. If the extension
2020
is off, it silently removes directives that would produce output.
21-
- **Only processes literal blocks, not inline code**. Sphinx has great tools
21+
- **Only processes blocks, not inline code**. Sphinx has great tools
2222
for linking definitions inline, and longer code should be in a block anyway.
2323
- **Doesn't run example code**. Therefore all possible resolvable types are not
2424
found, and the runtime correctness of code cannot be validated.

‎docs/src/examples.rst

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -21,15 +21,16 @@ be analysed and linked to known reference documentation entries.
2121
2222
Different import styles are supported, along with all Python syntax.
2323
Star imports might be particularly handy in code examples.
24-
`Doctest <https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html
25-
#doctest-blocks>`_ and console blocks using :code:`.. code:: pycon` work too.
26-
Including code via :rst:dir:`literalinclude` requires using one of the following parameters:
27-
:code:`:language: python`, :code:`:language: python3`,
28-
:code:`:language: py`, :code:`:language: py3`, :code:`:language: pyi`,
29-
:code:`:language: sage`, :code:`:language: bazel`, :code:`:language: starlark`
30-
(i.e., the list of
31-
`Pygments Lexers <https://pygments.org/docs/lexers/#pygments.lexers.python.PythonLexer>`_).
3224

25+
A number of other types of blocks are also supported
26+
along with standard Sphinx configuration, like:
27+
28+
- Literal blocks, the ``.. highlight::`` directive
29+
and ``highlight_language`` configuration
30+
- `Doctest blocks <https://docutils.sourceforge.io/docs/ref/rst/restructuredtext.html#doctest-blocks>`_
31+
- Console blocks using :code:`.. code:: pycon`
32+
- Including code via :rst:dir:`literalinclude`, but requires using one of the
33+
`Python Lexers <https://pygments.org/docs/lexers/#pygments.lexers.python.PythonLexer>`_
3334

3435
.. code:: pycon
3536

‎docs/src/index.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ Caveats
7070
For a more thorough explanation, see :ref:`about`.
7171

7272
- Only works with HTML documentation
73-
- Only processes literal blocks, not inline code
73+
- Only processes blocks, not inline code
7474
- Doesn't run example code
7575
- Parsing and type hint resolving is incomplete
7676

‎docs/src/reference.rst

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -57,21 +57,27 @@ Configuration
5757

5858
.. confval:: codeautolink_warn_on_missing_inventory
5959

60-
Type: ``bool``. Issue warning when an object cannot be found in
60+
Type: ``bool``. Warn when an object cannot be found in
6161
the inventory (autodoc or intersphinx). Defaults to :code:`False`.
6262

6363
.. confval:: codeautolink_warn_on_failed_resolve
6464

65-
Type: ``bool``. Issue warning when failing to resolve the canonical location
65+
Type: ``bool``. Warn when failing to resolve the canonical location
6666
of an object that a code element references. Defaults to :code:`False`.
6767

6868
.. confval:: codeautolink_warn_on_no_backreference
6969

70-
Type: ``bool``. Issue warning when no backreference could be found
70+
Type: ``bool``. Warn when no backreference could be found
7171
from reference documentation using the :rst:dir:`autolink-examples` table.
7272
This highlights objects for which no tutorial, example or how-to exists.
7373
Defaults to :code:`False`.
7474

75+
.. confval:: codeautolink_warn_on_default_parse_fail
76+
77+
Type: ``bool``. Warn when a code block using the ``default`` lexer
78+
cannot be parsed as Python. By default these cases are ignored by the
79+
syntax highlighter, so we match the behavior. Defaults to :code:`False`.
80+
7581
Directives
7682
----------
7783
.. rst:directive:: .. autolink-examples:: object
@@ -96,7 +102,7 @@ Directives
96102
97103
.. rst:directive:: .. autolink-concat:: [mode]
98104
99-
Toggle literal block concatenation.
105+
Toggle block concatenation.
100106
Concatenated code blocks are treated as a continuous source,
101107
so that imports and statements in previous blocks affect later blocks.
102108
Concatenation is begun at the directive, not applied retroactively.

‎docs/src/release_notes.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,10 @@ sphinx-codeautolink adheres to
1515
a corresponding tutorial or how-to (:issue:`161`)
1616
- Add more Pygments lexer aliases in code blocks (:issue:`160`)
1717
- Fix undocumented class attribute leading to a crash (:issue:`165`)
18+
- Support the ``default`` lexer, literal blocks, ``.. highlight::`` directive
19+
and ``highlight_language`` configuration (:issue:`166`)
20+
- Add :confval:`codeautolink_warn_on_default_parse_fail` to warn about
21+
failing to link code blocks without a language parameter (:issue:`166`)
1822

1923
0.16.2 (2025-01-16)
2024
-------------------

‎src/sphinx_codeautolink/__init__.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,12 @@ def setup(app: Sphinx):
4949
rebuild="html",
5050
types=[bool],
5151
)
52+
app.add_config_value(
53+
"codeautolink_warn_on_default_parse_fail",
54+
default=False,
55+
rebuild="html",
56+
types=[bool],
57+
)
5258

5359
app.add_directive("autolink-concat", directive.Concat)
5460
app.add_directive("autolink-examples", directive.Examples)

‎src/sphinx_codeautolink/extension/__init__.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,8 @@ def __init__(self) -> None:
7575
self.warn_missing_inventory = None
7676
self.warn_failed_resolve = None
7777
self.warn_no_backreference = None
78+
self.warn_default_parse_fail = None
79+
self.highlight_lang = None
7880

7981
# Populated once
8082
self.outdated_docs: set[str] = set()
@@ -105,6 +107,10 @@ def build_inited(self, app) -> None:
105107
self.warn_missing_inventory = app.config.codeautolink_warn_on_missing_inventory
106108
self.warn_failed_resolve = app.config.codeautolink_warn_on_failed_resolve
107109
self.warn_no_backreference = app.config.codeautolink_warn_on_no_backreference
110+
self.warn_default_parse_fail = (
111+
app.config.codeautolink_warn_on_default_parse_fail
112+
)
113+
self.highlight_lang = app.config.highlight_language
108114

109115
# Append static resources path so references in setup() are valid
110116
app.config.html_static_path.append(
@@ -138,6 +144,8 @@ def parse_blocks(self, app, doctree) -> None:
138144
global_preface=self.global_preface,
139145
custom_blocks=self.custom_blocks,
140146
concat_default=self.concat_default,
147+
default_highlight_lang=self.highlight_lang,
148+
warn_default_parse_fail=self.warn_default_parse_fail,
141149
)
142150
doctree.walkabout(visitor)
143151
self.cache.transforms[visitor.current_document] = visitor.source_transforms

‎src/sphinx_codeautolink/extension/block.py

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919

2020
# list from https://pygments.org/docs/lexers/#pygments.lexers.python.PythonLexer
2121
BUILTIN_BLOCKS = {
22+
"default": None,
2223
"python": None,
2324
"python3": None,
2425
"py": None,
@@ -104,7 +105,7 @@ def clean_ipython(source: str) -> tuple[str, str]:
104105

105106

106107
class CodeBlockAnalyser(nodes.SparseNodeVisitor):
107-
"""Transform literal blocks of Python with links to reference documentation."""
108+
"""Transform blocks of Python with links to reference documentation."""
108109

109110
def __init__(
110111
self,
@@ -113,6 +114,8 @@ def __init__(
113114
global_preface: list[str],
114115
custom_blocks: dict[str, Callable[[str], str]],
115116
concat_default: bool,
117+
default_highlight_lang: str | None,
118+
warn_default_parse_fail: bool,
116119
**kwargs,
117120
) -> None:
118121
super().__init__(*args, **kwargs)
@@ -130,6 +133,8 @@ def __init__(
130133
self.concat_section = False
131134
self.concat_sources = []
132135
self.skip = None
136+
self.highlight_lang = default_highlight_lang
137+
self.warn_default_parse_fail = warn_default_parse_fail
133138

134139
def unknown_visit(self, node) -> None:
135140
"""Handle and delete custom directives, ignore others."""
@@ -162,6 +167,10 @@ def unknown_visit(self, node) -> None:
162167
def unknown_departure(self, node) -> None:
163168
"""Ignore unknown nodes."""
164169

170+
def visit_highlightlang(self, node) -> None:
171+
"""Set expected highlight language."""
172+
self.highlight_lang = node["lang"]
173+
165174
def visit_title(self, node) -> None:
166175
"""Track section names and break concatenation and skipping."""
167176
self.title_stack.append(node.astext())
@@ -185,7 +194,7 @@ def visit_doctest_block(self, node):
185194

186195
def visit_literal_block(self, node: nodes.literal_block):
187196
"""Visit a generic literal block."""
188-
return self.parse_source(node, node.get("language", None))
197+
return self.parse_source(node, node.get("language", self.highlight_lang))
189198

190199
def parse_source(
191200
self, node: nodes.literal_block | nodes.doctest_block, language: str | None
@@ -231,6 +240,9 @@ def parse_source(
231240
try:
232241
names = parse_names(modified_source, node)
233242
except SyntaxError as e:
243+
if language == "default" and not self.warn_default_parse_fail:
244+
return
245+
234246
show_source = self._format_source_for_error(
235247
self.global_preface, self.concat_sources, prefaces, source
236248
)

‎src/sphinx_codeautolink/extension/directive.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ def copy(self):
6262

6363

6464
class Concat(Directive):
65-
"""Toggle and cut literal block concatenation in a document."""
65+
"""Toggle and cut block concatenation in a document."""
6666

6767
has_content = False
6868
required_arguments = 0

‎tests/extension/ref/pygments_lexer_code-block_bazel.txt renamed to ‎tests/extension/ref/lexer_bazel.txt

File renamed without changes.

‎tests/extension/ref/lexer_default.txt

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
test_project
2+
test_project.bar
3+
# split
4+
# split
5+
Test project
6+
============
7+
8+
.. code::
9+
10+
import test_project
11+
12+
test_project.bar()
13+
14+
.. automodule:: test_project

‎tests/extension/ref/lexer_default_fail.txt

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# split
2+
# split
3+
Test project
4+
============
5+
6+
.. code::
7+
8+
invalid python in a default block should not crash
9+
10+
.. automodule:: test_project

‎tests/extension/ref/ref_doctest.txt renamed to ‎tests/extension/ref/lexer_doctest.txt

File renamed without changes.

‎tests/extension/ref/ref_extdoctest_doctest.txt renamed to ‎tests/extension/ref/lexer_extdoctest_doctest.txt

File renamed without changes.

‎tests/extension/ref/ref_extdoctest_testcode.txt renamed to ‎tests/extension/ref/lexer_extdoctest_testcode.txt

File renamed without changes.

‎tests/extension/ref/lexer_highlight_language.txt

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# split
2+
highlight_language = "none"
3+
# split
4+
Test project
5+
============
6+
7+
Here's an example::
8+
9+
import test_project
10+
test_project.bar()
11+
12+
.. automodule:: test_project

‎tests/extension/ref/lexer_highlight_none.txt

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# split
2+
# split
3+
Test project
4+
============
5+
6+
.. highlight:: none
7+
8+
Here's an example::
9+
10+
import test_project
11+
test_project.bar()
12+
13+
.. automodule:: test_project

‎tests/extension/ref/ref_ipython.txt renamed to ‎tests/extension/ref/lexer_ipython.txt

File renamed without changes.

‎tests/extension/ref/ref_ipython3.txt renamed to ‎tests/extension/ref/lexer_ipython3.txt

File renamed without changes.

‎tests/extension/ref/ref_ipython_directive.txt renamed to ‎tests/extension/ref/lexer_ipython_directive.txt

File renamed without changes.

‎tests/extension/ref/ref_ipython_directive_comment.txt renamed to ‎tests/extension/ref/lexer_ipython_directive_comment.txt

File renamed without changes.

‎tests/extension/ref/ref_ipython_syntax.txt renamed to ‎tests/extension/ref/lexer_ipython_syntax.txt

File renamed without changes.

‎tests/extension/ref/lexer_literal.txt

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
test_project
2+
test_project.bar
3+
# split
4+
# split
5+
Test project
6+
============
7+
8+
Here's an example::
9+
10+
import test_project
11+
test_project.bar()
12+
13+
.. automodule:: test_project

‎tests/extension/ref/non_python_block.txt renamed to ‎tests/extension/ref/lexer_non_python_block.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
Test project
44
============
55

6-
.. code::
6+
.. code:: c
77

88
import test_project
99

‎tests/extension/ref/ref_py.txt renamed to ‎tests/extension/ref/lexer_py.txt

File renamed without changes.

‎tests/extension/ref/pygments_lexer_code-block_py3.txt renamed to ‎tests/extension/ref/lexer_py3.txt

File renamed without changes.

‎tests/extension/ref/ref_pycon.txt renamed to ‎tests/extension/ref/lexer_pycon.txt

File renamed without changes.

‎tests/extension/ref/pygments_lexer_code-block_pyi.txt renamed to ‎tests/extension/ref/lexer_pyi.txt

File renamed without changes.

‎tests/extension/ref/pygments_lexer_code-block_python3.txt renamed to ‎tests/extension/ref/lexer_python3.txt

File renamed without changes.

‎tests/extension/ref/pygments_lexer_code-block_sage.txt renamed to ‎tests/extension/ref/lexer_sage.txt

File renamed without changes.

‎tests/extension/ref/pygments_lexer_code-block_starlark.txt renamed to ‎tests/extension/ref/lexer_starlark.txt

File renamed without changes.

‎tests/extension/ref/preface_consumed_non_python.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Test project
44
============
55

66
.. autolink-preface:: import test_project
7-
.. code::
7+
.. code:: c
88

99
test_project.bar()
1010

0 commit comments

Comments
 (0)