-
-
Notifications
You must be signed in to change notification settings - Fork 7.9k
Improve error message for kiwisolver import error (DLL load failed) #14316
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve error message for kiwisolver import error (DLL load failed) #14316
Conversation
…or: DLL load failed'
lib/matplotlib/__init__.py
Outdated
module = importlib.import_module(modname) | ||
except ImportError as error: | ||
if sys.platform == 'win32' and 'DLL' in error.msg: | ||
msg = ('You may be missing MIcrosoft Visual C++ ' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
msg = ('You may be missing MIcrosoft Visual C++ ' | |
msg = ('You may be missing Microsoft Visual C++ ' |
The matplotlib wheels on PyPI include the Microsoft Visual C++ redistributable DLLs. To workaround this issue one could load a matplotlib extension first, e.g.: diff --git a/lib/matplotlib/__init__.py b/lib/matplotlib/__init__.py
index e0ef418f3..83345af3b 100644
--- a/lib/matplotlib/__init__.py
+++ b/lib/matplotlib/__init__.py
@@ -184,6 +184,8 @@ def compare_versions(a, b):
def _check_versions():
+ from . import ft2font
+
for modname, minver in [
("cycler", "0.10"),
("dateutil", "2.1"),
I don't understand why library versions are checked at runtime. Aren't package managers supposed to handle this based on the package metadata? |
@cgohlke when I looked into this yesterday I read somewhere (cannot remember where now...) that one could not embed the redistributable in the wheel, but it seems wrong. Do you think I should try to include it in kiwisolver wheels to avoid this issue ? If so do you have a pointer as to how to do that ? |
lib/matplotlib/__init__.py
Outdated
@@ -192,7 +192,17 @@ def _check_versions(): | |||
("numpy", "1.11"), | |||
("pyparsing", "2.0.1"), | |||
]: | |||
module = importlib.import_module(modname) | |||
try: | |||
module = importlib.import_module(modname) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is Kiwisolver the only module for which this is true? In which case, maybe check modname
to avoid confusing people who get import errors from other modules.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably not, but it is the only one I've experienced the problem with.
Also if I'm not mistaken, kiwisolver is not even used by most matplotlibusers. It plays no role in creating basic scatter plots, histograms etc., does it? Maybe one matplotlib could just issue a warning about kiwisolver not being available and load anyway. Someone suggested something like it once: #10386
So the DLL kiwisolver is missing is actually distributed with matplotlib? Then it would be nice if they could be used to prevent this issue.
Me neither, I too would assume it was pip's job to check dependencies based on metadata. Maybe it has to do with the note around line 100 of
|
Yes, but at least pip will let currently you drive an enviroment to an invalid state (for example if you install a second package which pins to an incompatible version of a dependency or even simple remove a dependency). Even if we got rid of the version checks here, we unconditionally import
Is this problem new with mpl 3.1.0? Does in depend on what version of kiwisolver is installed by pip? Do we think this is something about @jonascj 's system or that this is endemic across Windows users?
But then a major feature (constrained layout) would not work. When this dependency came in we decided to make it a hard dependency because installing it did not seem to be an undue burden and was on net less confusing than documenting and supporting an optional python dependency for functionality (rather external programs or the GUI framework wrappers to enable different kinds of output/embedding). The usage of Given @cgohlke 's comment, I think the reason we are seeing this now is #11980 which unified (and extended) the run time version checking. Previously we were getting lucky with the import order and always importing xref: #14303 |
Given my long comment above I think for 3.1.1 we should either
The "correct" fix is to fix the kiwisolver wheels, but we should do something practical side for now. |
Yes, is this a bug in the recently released Matplotlib 3.1.0, or a bug in the semi-recently released kiwisolver 1.1.0? |
I would say it is a general issue, it did a fresh install of Windows 10 64bit version 1809 and 1903 several times over on three laptops. Every time the same issue with identical version of python (3.7.3 both 32bit and 64bit, both installed per user and systemwide) and the most recent PyPi version of matplotlib. Also tried installed kiwisolver on its own, same issue. This DLL Load issue is because of kiwisolver. The mentioned version of python and matplot lib installs just fine on some other Win10 laptops I had access to, but these laptop have had tons of other software installed (Adobe this and that, Unity, Chrome, Firefox etc.) so they likely just have the Microsoft Visual C++ redist DLL's from some other piece of software. I tested the same version |
@tacaswell see my answer above. I would say it is problem for all windows users. The only reason it will seldom be seen is that it is a rare thing to do a fresh install of Windows 10 and as the first thing install python+matplotlib. Let me know if you want further testing to confirm this (and if so, what testing, other version of kiwi / matplotlib?). I am not a Windows user myself, but forced to deal with a lot of windows day-to-day. So I have plenty of test material. |
I can re-package kiwi (with the help of @matthew-brett as usual) but it seems a bit wasteful to distribute Microsoft redistributable with everything package requiring the c++ runtime (the way matplotlib seem to be doing). |
At the moment we don't have a good system in pip for installing not-Python packages, but we could look to see if we can hack round it. @zooba - are we allowed to make a pip package for the MS C++ runtime? Or do you (as Microsoft) have to do that? Have y'all already done that? |
That's the recommended way and is used by Matplotlib, Pandas, Scipy, pyzmq, numpy+mkl... Probably the easiest solution for kiwi (not generally recommended though) is to statically link the C/C++ runtime library. Set the |
By curiosity when packaging the Windows redistributable with the library, will newer redistributable installed "normally" be picked up first ? I found https://github.com/MacPython/scipy-wheels/blob/master/appveyor.yml, that include the redistributable in scipy. Should I follow that example to re-package windows wheel ? |
@tacaswell You mean try this patch and see if it mitigates the problem?
|
One way to reproduce this issue on a Windows 10 system that already has the Visual C++ Redistributable Package installed is to install Python and matplotlib in a Windows Sandbox.
|
@jonascj Yes |
@tacaswell I'll do this later today or tomorrow. @cgohlke Interesting, I will however just test it in a virtual machine (restored to a snapshot taken right after completing the Windows 10 installation). |
…mportError: DLL load failed'" This reverts commit f14ffb1.
…h otherwise fails with 'ImportError: DLL load failed'
@cgohlke solutions (importing ft2font before other imports) solves the I've reverted my previous commit (in my own feature branch) and committed this new fix/patch. Let me know if this is not the way to do it, then I'll try to do it another way. |
Squashing and force-pushing the first two commits out of existence would be marginally better but we can also squash-merge it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the minimal pragmatic solution.
So you would have preferred it if I just rewrote history by not having the original commit + revert commit? I'm not sure my git-skills allow me to do that at this point ... |
@jonascj don't worry, I can squash-merge it so everything gets squashed into one commit. |
…port error (DLL load failed)
I didn't, it was just a question to improve my future contributions. Many a place you can read that rewriting history (i.e. resetting and removing commits, only to make new ones) is bad if the history has already been published/pushed. I was in doubt whether that applied in this case. I suppose not since they were only pushed to my fork... |
In general I think pushing/pulling to a non-merged personal branch is fine, but we definitely don't squash or force push anything on to any of the |
…316-on-v3.1.x Backport PR #14316 on branch v3.1.x (Improve error message for kiwisolver import error (DLL load failed))
@dstansby is exactly right, force-pushing to your branches is ok as that is "scratch space" and we may not want to preserve every thing you did along the way into the main history. On the other hand the Thanks for your work on this @jonascj ! Hopefully we will here from you again! As always, thank you @cgohlke for being the windows wheels magician! |
Same issue on my machine (Win10 64-bit, Conda 4.7.12, Python 3.7.3.final.0) - Spent hours trying different methods found from GitHub/StackOverflow with no luck. Finally fixed the issue by downgrading to matplotlib 3.0.3 ("older stable version" as stated by the official website).
|
Which matplotlib version have you been having trouble with? |
@frank-wang-yifei please report this to anaconda (or conda-forge if you are using their packages). This is almost certainly a packaging problem, not something we can fix in Matplotilb. |
@tacaswell Reported. Thanks! |
@timhoffm If you mean my |
@tacaswell @frank-wang-yifei You can try this method: |
Thanks for the tip. At first I was concerned about mixing conda and pip packages. But next time I might give this a try. |
Installing matplotlib with pip on a fresh/clean installation of Windows might fail with an unhelpful
ImportError: DLL load failed
. One reason for this could be missing Microsoft Visual C++ Redistributable dependencies. This PR adds a check for this condition and provides a more helpful error message if the import fails.Tested on Windows 10 64bit, Python 3.7.3. The import error is caught and the helpful error message displayed. To test this I was unable to use
pip install -e .
from within the clone of the repo, because then the missing dependency is already an issue when buildingmatplotlib.ft2font
. To work around this I installed matplotlib from pypipip install matplotlib
(which does not perform the build) then replacedmy-vevn/Lib/site-packages/matplotlib/__init__.py
with my patched__init__.py
.I haven't written any pytest style tests for this, I have close to no idea how to mock the ImportError or simulate the missing dependency.
Thank you @MatthieuDartiailh for your input (nucleic/kiwi#69).