-
-
Notifications
You must be signed in to change notification settings - Fork 7.9k
[ENH]: Provide a way to avoid subcommands on import. #28488
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Related to #28485 If we do this the flag should be something like We only call the code that calls subprocesses if we do not find a cached version of the results matplotlib/lib/matplotlib/font_manager.py Lines 1567 to 1585 in f5067c1
If you ensure that cache is created (and cached) in your pipelines then you will also avoid this problem (and speed up your job startup!). For example, if you are using docker files adding |
thanks for the quick reply! agreed that in principle "always create the font cache before pipelines start" is a good discipline to have, but the tradeoff is that it means every image used anywhere in one of our pipelines needs to do have already done this: it's a build-time thing, not a runtime thing. I think there's still value in having the ability to disable this at runtime; WDYT? re: details: agreed on disabling for both OS variants at the same time, and happy to update the env var. |
On a bit more thought, I think the best path here is to add an option to ignore all fonts that are not what we bundle (which will probably make headaches for at least debian who de-bundle the fonts). Even if we drop just the sub-process we still do some directory walking and attempt to discover fonts which is requires a bunch of filesystem work and then I/O as we read the files to extract some information which can be slow (which is why there is the threading work in there) and leaving it in a state where we find some system fonts (because on some CI systems subprocesses hang(?!)) does not feel right. On the other hand "I want to have every font accessible on my system available to Matplotlib" vs "do not waste time looking for other fonts, I know I'm going to use the defaults" seems like a reasonable decision point. The first is good for interactive use (where we assume that the users home directory is presistent and fonts should "just work") and the second is good for CI / automated jobs where the home directory is not persistent. On the other hand, this does come at the cost of not (automatically) providing access to non-western fonts (e.g. we do not bundle a font that includes any CJK gylphs). |
I added a comment to a different, but related, issue here: #28485 (comment). @tacaswell What you're suggesting makes sense I think. Sounds like it would remove the need for us to pre-populate the font cache at build time. |
Problem
tl;dr: I'd like to be able to avoid
from matplotlib import pyplot as plt
invoking subprocesses at import time.More generally: we often run jobs which involve a transitive import of matplotlib; for reasons I do not fully understand, we see rare but regular flakes where the process hangs during import, specifically at this line where matplotlib first invokes
fc-list
. The code is written to be robust tofc-list
not existing, but here it's the existence check itself that's causing trouble.I'd like to be able to disable that call; in general, I suspect "this library is running subcommands during package import" may well cause these sorts of problems in other environments, so I suspect this would be helpful to other users as well.
Proposed solution
I've got a quick first pass at the type of change I'm describing at craigcitro-ant@46d64f2; if that looks acceptable, I'm more than happy to clean it up and send a PR. (The main thing that's missing is adding a mention to the docs.)
The text was updated successfully, but these errors were encountered: