Fix issue with tex-encoding on non-Unicode platforms #23033

oscargus · 2022-05-11T06:42:57Z

PR Summary

Closes #23019

Thanks @anntzer for the rapid fix.

PR Checklist

Tests and Styling

Has pytest style unit tests (and pytest passes).
Is Flake 8 compliant (install flake8-docstrings and run flake8 --docstring-convention=all).

Documentation

[N/A] New features are documented, with examples if plot related.
[N/A] New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
[N/A] API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).
[N/A] Documentation is sphinx and numpydoc compliant (the docs should build without error).

Actually, tests seem to fail?

oscargus · 2022-05-11T09:54:43Z

Yes, turned out that I didn't check the error message well enough. It raises a CalledProcessError independent of the underlying exception, so I do not really know if it can be tested properly that a RuntimeError (now) is raised rather than a UnicodeDecodeError (earlier).

anntzer · 2022-05-11T14:40:47Z

I'm a bit confused; it should ultimately raise a RuntimeError, not a CalledProcessError, if you do a fig.canvas.draw(), doesn't it?

oscargus · 2022-05-12T07:15:16Z

Here is the error message:

In [3]: fig, ax = plt.subplots()
   ...: ax.set_ylabel('\N{SNOWMAN}')
Out[3]: Text(0, 0.5, '☃')Traceback (most recent call last):

  File "/local/data1/matplotlib/lib/matplotlib/texmanager.py", line 254, in _run_checked_subprocess
    report = subprocess.check_output(

  File "/local/data1/miniconda3/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,

  File "/local/data1/miniconda3/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,

CalledProcessError: Command '['latex', '-interaction=nonstopmode', '--halt-on-error', '../71cab2b5aca12ed5cad4a481b3b00e42.tex']' returned non-zero exit status 1.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/local/data1/miniconda3/lib/python3.8/site-packages/IPython/core/formatters.py", line 341, in __call__
    return printer(obj)

  File "/local/data1/miniconda3/lib/python3.8/site-packages/IPython/core/pylabtools.py", line 151, in print_figure
    fig.canvas.print_figure(bytes_io, **kw)

  File "/local/data1/matplotlib/lib/matplotlib/backend_bases.py", line 2262, in print_figure
    self.figure.draw(renderer)

  File "/local/data1/matplotlib/lib/matplotlib/artist.py", line 73, in draw_wrapper
    result = draw(artist, renderer, *args, **kwargs)

  File "/local/data1/matplotlib/lib/matplotlib/artist.py", line 50, in draw_wrapper
    return draw(artist, renderer)

  File "/local/data1/matplotlib/lib/matplotlib/figure.py", line 2891, in draw
    mimage._draw_list_compositing_images(

  File "/local/data1/matplotlib/lib/matplotlib/image.py", line 131, in _draw_list_compositing_images
    a.draw(renderer)

  File "/local/data1/matplotlib/lib/matplotlib/artist.py", line 50, in draw_wrapper
    return draw(artist, renderer)

  File "/local/data1/matplotlib/lib/matplotlib/axes/_base.py", line 3078, in draw
    mimage._draw_list_compositing_images(

  File "/local/data1/matplotlib/lib/matplotlib/image.py", line 131, in _draw_list_compositing_images
    a.draw(renderer)

  File "/local/data1/matplotlib/lib/matplotlib/artist.py", line 50, in draw_wrapper
    return draw(artist, renderer)

  File "/local/data1/matplotlib/lib/matplotlib/axis.py", line 1291, in draw
    self.label.draw(renderer)

  File "/local/data1/matplotlib/lib/matplotlib/artist.py", line 50, in draw_wrapper
    return draw(artist, renderer)

  File "/local/data1/matplotlib/lib/matplotlib/text.py", line 687, in draw
    bbox, info, descent = self._get_layout(renderer)

  File "/local/data1/matplotlib/lib/matplotlib/text.py", line 321, in _get_layout
    w, h, d = _get_text_metrics_with_cache(

  File "/local/data1/matplotlib/lib/matplotlib/text.py", line 97, in _get_text_metrics_with_cache
    return _get_text_metrics_with_cache_impl(

  File "/local/data1/matplotlib/lib/matplotlib/text.py", line 105, in _get_text_metrics_with_cache_impl
    return renderer_ref().get_text_width_height_descent(text, fontprop, ismath)

  File "/local/data1/matplotlib/lib/matplotlib/backends/backend_agg.py", line 235, in get_text_width_height_descent
    w, h, d = texmanager.get_text_width_height_descent(

  File "/local/data1/matplotlib/lib/matplotlib/texmanager.py", line 361, in get_text_width_height_descent
    dvifile = cls.make_dvi(tex, fontsize)

  File "/local/data1/matplotlib/lib/matplotlib/texmanager.py", line 294, in make_dvi
    cls._run_checked_subprocess(

  File "/local/data1/matplotlib/lib/matplotlib/texmanager.py", line 262, in _run_checked_subprocess
    raise RuntimeError(

RuntimeError: latex was not able to process the following string:
b'\\u2603'

Here is the full report generated by latex:
This is pdfTeX, Version 3.1415926-2.5-1.40.14 (TeX Live 2013)
 restricted \write18 enabled.
entering extended mode
(../71cab2b5aca12ed5cad4a481b3b00e42.tex
LaTeX2e <2011/06/27>
Babel <v3.8m> and hyphenation patterns for english, dumylang, nohyphenation, lo
aded.
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2007/10/19 v1.4h Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo))
(/usr/share/texlive/texmf-dist/tex/latex/type1cm/type1cm.sty)
(/usr/share/texlive/texmf-dist/tex/latex/cm-super/type1ec.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/t1cmr.fd))
(/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/utf8.def
(/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.dfu)
(/usr/share/texlive/texmf-dist/tex/latex/base/ot1enc.dfu)
(/usr/share/texlive/texmf-dist/tex/latex/base/omsenc.dfu)))
(/usr/share/texlive/texmf-dist/tex/latex/geometry/geometry.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty)
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifpdf.sty)
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ifvtex.sty)
(/usr/share/texlive/texmf-dist/tex/generic/ifxetex/ifxetex.sty))
(/usr/share/texlive/texmf-dist/tex/latex/underscore/underscore.sty)
(/usr/share/texlive/texmf-dist/tex/latex/base/textcomp.sty
(/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.def
(/usr/share/texlive/texmf-dist/tex/latex/base/ts1enc.dfu)))
No file 71cab2b5aca12ed5cad4a481b3b00e42.aux.
(/usr/share/texlive/texmf-dist/tex/latex/base/ts1cmr.fd)
*geometry* driver: auto-detecting
*geometry* detected driver: dvips

! Package inputenc Error: Unicode char \u8:☃ not set up for use with LaTeX.

See the inputenc package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                              
                                                  
l.29 {\obeylines\sffamily ☃
                             }%
No pages of output.
Transcript written on 71cab2b5aca12ed5cad4a481b3b00e42.log.




<Figure size 432x288 with 1 Axes>

so it seems like the test sees CalledProcessError. I have not figured out a way to detect the underlying exception.

(Somehow one would really like to test that there is not a UnicodeDecodeError rather than an error is raised, in case someone has a latex-setup that actually can do a snowman... So maybe a try-except is better? Although that currently has the same problem.)

oscargus · 2022-05-12T07:21:23Z

The output above is what I get in Spyder (that automatically does drawing).

Calling fig.canvas.draw() in an ipython console gives the same error message.

anntzer · 2022-05-12T08:38:21Z

No, the actually thrown exception is the lower one? ("CalledProcessError was the direct cause of the following exception...") Try e.g. wrapping the whole thing in a try... except... to check what exception type you can catch.

oscargus · 2022-05-12T09:06:31Z

Ahh, you are right. I was confused by the test failure and thought that it raised a different exception, rather than no exception.

oscargus · 2022-05-14T10:09:49Z

Rebased. ~~Seems like the Ubuntu 3.9-failure somehow is related, but I do not understand if it is or if it is just a random failure that happened to be caused by a test that (maybe) uses the changed code.~~ Edit: restarting the test fixed it.

oscargus · 2022-05-14T12:27:03Z

Depending on the merge order of this and #23045 I will update the one that is not merged first.

anntzer · 2022-06-30T13:06:45Z

lib/matplotlib/tests/test_texmanager.py

 import matplotlib.pyplot as plt
 from matplotlib.texmanager import TexManager
 import pytest


+needs_usetex = pytest.mark.skipif(


@oscargus I think you can now import this from matplotlib.testing._markers.

Feel free to self-merge after fixing this.

oscargus added topic: text/unicode topic: text/usetex labels May 11, 2022

oscargus added this to the v3.6.0 milestone May 11, 2022

anntzer previously approved these changes May 11, 2022

View reviewed changes

oscargus force-pushed the latexerrorencoding branch from 618e86d to 3ad3dd8 Compare May 11, 2022 09:53

anntzer mentioned this pull request May 12, 2022

Suppress traceback chaining for tex subprocess failures. #23036

Merged

6 tasks

oscargus force-pushed the latexerrorencoding branch from 3ad3dd8 to dbc8b7a Compare May 12, 2022 09:05

oscargus force-pushed the latexerrorencoding branch from dbc8b7a to 8c13417 Compare May 12, 2022 10:09

QuLogic added the status: needs rebase label May 14, 2022

oscargus force-pushed the latexerrorencoding branch from 8c13417 to 23a60c0 Compare May 14, 2022 09:29

oscargus removed the status: needs rebase label May 14, 2022

oscargus mentioned this pull request May 14, 2022

MNT: Merge locally defined test marks #23045

Merged

QuLogic added the status: needs rebase label Jun 15, 2022

jklymak approved these changes Jun 30, 2022

View reviewed changes

anntzer approved these changes Jun 30, 2022

View reviewed changes

anntzer reviewed Jun 30, 2022

View reviewed changes

oscargus force-pushed the latexerrorencoding branch from 23a60c0 to 0edf0f1 Compare June 30, 2022 13:28

oscargus removed the status: needs rebase label Jun 30, 2022

Fix issue with tex-encoding on non-Unicode platforms

b81546b

oscargus force-pushed the latexerrorencoding branch from 0edf0f1 to b81546b Compare June 30, 2022 13:32

oscargus merged commit 0758a12 into matplotlib:main Jun 30, 2022

oscargus deleted the latexerrorencoding branch June 30, 2022 14:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Fix issue with tex-encoding on non-Unicode platforms #23033

Fix issue with tex-encoding on non-Unicode platforms #23033

Uh oh!

oscargus commented May 11, 2022

Uh oh!

oscargus commented May 11, 2022

Uh oh!

anntzer commented May 11, 2022

Uh oh!

oscargus commented May 12, 2022

Uh oh!

oscargus commented May 12, 2022

Uh oh!

anntzer commented May 12, 2022

Uh oh!

oscargus commented May 12, 2022

Uh oh!

oscargus commented May 14, 2022 •

edited

Loading

Uh oh!

oscargus commented May 14, 2022

Uh oh!

anntzer Jun 30, 2022

Uh oh!

Uh oh!

Uh oh!

Fix issue with tex-encoding on non-Unicode platforms #23033

Fix issue with tex-encoding on non-Unicode platforms #23033

Uh oh!

Conversation

oscargus commented May 11, 2022

PR Summary

PR Checklist

Uh oh!

oscargus commented May 11, 2022

Uh oh!

anntzer commented May 11, 2022

Uh oh!

oscargus commented May 12, 2022

Uh oh!

oscargus commented May 12, 2022

Uh oh!

anntzer commented May 12, 2022

Uh oh!

oscargus commented May 12, 2022

Uh oh!

oscargus commented May 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oscargus commented May 14, 2022

Uh oh!

anntzer Jun 30, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

oscargus commented May 14, 2022 •

edited

Loading