It's generally said not to use utf8x. Since Python strings are pretty much Unicode safe, I do wonder if maybe we should start using lualatex or xelatex instead of trying to hack around pdflatex deficiencies.

We actually use latex, not pdflatex. The reason is that we need to get dvi output, not pdf output, so that we can parse it and extract glyphs and then position them correctly for pdf and svg output. Currently agg uses dvipng to convert the dvi to png and embed the bitmap and ps uses a completely different codepath, but I would like them to also switch to the "extract-glyphs-from-dvi" approach; I guess that may be part of @aitikgupta's gsoc (mplcairo uses that approach for everything).

In fact both xetex and luatex can generate output in a dvi-like format (xdv for xetex) with extensions to support specifying any font on the filesystem, so they should be usable for that purpose as well, but that would require quite a bit of reworking of dviread to support their extensions (I have some patches lying around exploring that idea). So while I certainly support the idea of moving to xetex/luatex, it would be quite a bit of work...

One simpler possibility, though, may be to move the user preamble to the top and rewrite all our \usepackage to the same format as used for textcomp (i.e. skip the usepackage if the package has already been loaded). This way users can always use \usepackage[utf8x]{inputenc} as custom preamble if they want to live dangerously.

Looking at this again, one solution may be to just add a bunch of \DeclareUnicodeCharacter entries to our tex preamble, e.g.

rcParams["text.latex.preamble"] = r"\DeclareUnicodeCharacter{0394}{\ensuremath{\Delta}}"
figtext(.5, .5, "Δ", usetex=True)

works. In fact we already mostly have a tex<->unicode mapping in _mathtext_data (tex2uni), so we could use that to generate the preamble. The main problem I can see is that this table actually contains too many entries, some of which (e.g. \minus) only work if specific packages are loaded; I think we should split that table to into two parts, one of which correspond to all glyphs that exist by default, without loading any extra tex packages. (From some archeology, the current table seems to have been autogenerated from http://www.ams.org/STIX/bnb/stix-tbl.ascii-2005-09-24; see https://discourse.matplotlib.org/t/unicode-to-tex-symbols-type1-names-and-vice-versa/5407 and https://discourse.matplotlib.org/t/mathtext-patch/5423)