Use luatex in --luaonly mode to query kpsewhich.

anntzer · anntzer · commit 20009aec5f22 · 2021-05-22T23:59:07.000+02:00
`luatex --luaonly` runs a *lua* interpreter with relevant tex libraries
available, which avoids the overhead of repeatedly initializing
kpathsea (the old approach, very slow on macos and windows).  An
alternative approach would be to use `luatex` followed by `\directlua`
calls, but on windows it appears that one needs to use `lualatex` to get
a working interactive prompt, and just loading the latex format takes
seconds(!).

For the simple following benchmark:
```sh
python -c 'from pylab import *; mpl.use("pdf"); rcParams["text.usetex"] = True; plot(); savefig("test.pdf", backend="pdf")'
```
On a macos machine, this patch brings runtime from ~4.5s to ~2.5s.
On a windows machine, this patch brings runtime from ~6.5s to ~1.7s.

We also need to figure out how to best advertise this (do we emit a
warning suggesting to install luatex on windows and macos if luatex is
not present?).
diff --git a/doc/api/next_api_changes/deprecations/19558-AL.rst b/doc/api/next_api_changes/deprecations/19558-AL.rst
@@ -0,0 +1,3 @@
+The *format* parameter of ``dviread.find_tex_file``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+... is deprecated (with no replacement).
diff --git a/lib/matplotlib/dviread.py b/lib/matplotlib/dviread.py
@@ -25,6 +25,7 @@
 from pathlib import Path
 import re
 import struct
+import subprocess
 import sys
 import textwrap
 
@@ -970,7 +971,30 @@ def _parse_enc(path):
             "Failed to parse {} as Postscript encoding".format(path))
 
 
+class _LuatexKpsewhich:
+    @lru_cache()  # A singleton.
+    def __new__(cls):
+        self = object.__new__(cls)
+        self._proc = self._new_proc()
+        return self
+
+    def _new_proc(self):
+        return subprocess.Popen(
+            ["luatex", "--luaonly",
+             str(cbook._get_data_path("kpsewhich.lua"))],
+            stdin=subprocess.PIPE, stdout=subprocess.PIPE)
+
+    def search(self, filename):
+        if self._proc.poll() is not None:  # Dead, restart it.
+            self._proc = self._new_proc()
+        self._proc.stdin.write(os.fsencode(filename) + b"\n")
+        self._proc.stdin.flush()
+        out = self._proc.stdout.readline().rstrip()
+        return "" if out == b"nil" else os.fsdecode(out)
+
+
 @lru_cache()
+@_api.delete_parameter("3.5", "format")
 def find_tex_file(filename, format=None):
     """
     Find a file in the texmf tree.
@@ -988,6 +1012,7 @@ def find_tex_file(filename, format=None):
     format : str or bytes
         Used as the value of the ``--format`` option to :program:`kpsewhich`.
         Could be e.g. 'tfm' or 'vf' to limit the search to that type of files.
+        Deprecated.
 
     References
     ----------
@@ -1002,6 +1027,14 @@ def find_tex_file(filename, format=None):
     if isinstance(format, bytes):
         format = format.decode('utf-8', errors='replace')
 
+    if format is None:
+        try:
+            lk = _LuatexKpsewhich()
+        except FileNotFoundError:
+            pass  # Fallback to directly calling kpsewhich, as below.
+        else:
+            return lk.search(filename)
+
     if os.name == 'nt':
         # On Windows only, kpathsea can use utf-8 for cmd args and output.
         # The `command_line_encoding` environment variable is set to force it
diff --git a/lib/matplotlib/mpl-data/kpsewhich.lua b/lib/matplotlib/mpl-data/kpsewhich.lua
@@ -0,0 +1,3 @@
+-- see dviread._LuatexKpsewhich
+kpse.set_program_name("tex")
+while true do print(kpse.lookup(io.read():gsub("\r", ""))); io.flush(); end
diff --git a/lib/matplotlib/texmanager.py b/lib/matplotlib/texmanager.py
@@ -1,23 +1,23 @@
 r"""
-Support for embedded TeX expressions in Matplotlib via dvipng and dvips for the
-raster and PostScript backends.  The tex and dvipng/dvips information is cached
-in ~/.matplotlib/tex.cache for reuse between sessions.
+Support for embedded TeX expressions in Matplotlib.
 
 Requirements:
 
-* LaTeX
-* \*Agg backends: dvipng>=1.6
-* PS backend: psfrag, dvips, and Ghostscript>=9.0
-
-For raster output, you can get RGBA numpy arrays from TeX expressions
-as follows::
-
-  texmanager = TexManager()
-  s = "\TeX\ is Number $\displaystyle\sum_{n=1}^\infty\frac{-e^{i\pi}}{2^n}$!"
-  Z = texmanager.get_rgba(s, fontsize=12, dpi=80, rgb=(1, 0, 0))
+* LaTeX.
+* \*Agg backends: dvipng>=1.6.
+* PS backend: PSfrag, dvips, and Ghostscript>=9.0.
+* PDF and SVG backends: if LuaTeX is present, it will be used to speed up some
+  post-processing steps, but note that it is not used to parse the TeX string
+  itself (only LaTeX is supported).
 
 To enable TeX rendering of all text in your Matplotlib figure, set
 :rc:`text.usetex` to True.
+
+TeX and dvipng/dvips processing results are cached
+in ~/.matplotlib/tex.cache for reuse between sessions.
+
+`TexManager.get_rgba` can also be used to directly obtain raster output as RGBA
+numpy arrays.
 """
 
 import functools
@@ -274,7 +274,15 @@ def get_grey(self, tex, fontsize=None, dpi=None):
         return alpha
 
     def get_rgba(self, tex, fontsize=None, dpi=None, rgb=(0, 0, 0)):
-        """Return latex's rendering of the tex string as an rgba array."""
+        r"""
+        Return latex's rendering of the tex string as an rgba array.
+
+        Examples
+        --------
+        >>> texmanager = TexManager()
+        >>> s = r"\TeX\ is $\displaystyle\sum_n\frac{-e^{i\pi}}{2^n}$!"
+        >>> Z = texmanager.get_rgba(s, fontsize=12, dpi=80, rgb=(1, 0, 0))
+        """
         alpha = self.get_grey(tex, fontsize, dpi)
         rgba = np.empty((*alpha.shape, 4))
         rgba[..., :3] = mpl.colors.to_rgb(rgb)
diff --git a/tutorials/text/usetex.py b/tutorials/text/usetex.py
@@ -11,10 +11,13 @@
 etc.) can be used. The results can be striking, especially when you take care
 to use the same fonts in your figures as in the main document.
 
-Matplotlib's LaTeX support requires a working LaTeX_ installation.  For the
-\*Agg backends, dvipng_ is additionally required; for the PS backend, psfrag_,
-dvips_ and Ghostscript_ are additionally required.  The executables for these
-external dependencies must all be located on your :envvar:`PATH`.
+Matplotlib's LaTeX support requires a working LaTeX_ installation.  For
+the \*Agg backends, dvipng_ is additionally required; for the PS backend,
+PSfrag_, dvips_ and Ghostscript_ are additionally required.  For the PDF
+and SVG backends, if LuaTeX is present, it will be used to speed up some
+post-processing steps, but note that it is not used to parse the TeX string
+itself (only LaTeX is supported).  The executables for these external
+dependencies must all be located on your :envvar:`PATH`.
 
 There are a couple of options to mention, which can be changed using
 :doc:`rc settings </tutorials/introductory/customizing>`. Here is an example
@@ -140,6 +143,6 @@
 .. _LaTeX: http://www.tug.org
 .. _Poppler: https://poppler.freedesktop.org/
 .. _PSNFSS: http://www.ctan.org/tex-archive/macros/latex/required/psnfss/psnfss2e.pdf
-.. _psfrag: https://ctan.org/pkg/psfrag
+.. _PSfrag: https://ctan.org/pkg/psfrag
 .. _Xpdf: http://www.xpdfreader.com/
 """

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	+The format parameter of ``dviread.find_tex_file``
	`2`	`+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~`
	`3`	`+... is deprecated (with no replacement).`
Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	`+-- see dviread._LuatexKpsewhich`
	`2`	`+kpse.set_program_name("tex")`
	`3`	`+while true do print(kpse.lookup(io.read():gsub("\r", ""))); io.flush(); end`