Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Bug]: matplotlib (kiwisolver) interference with other package #28991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mrrezaie opened this issue Oct 18, 2024 · 15 comments
Closed

[Bug]: matplotlib (kiwisolver) interference with other package #28991

mrrezaie opened this issue Oct 18, 2024 · 15 comments

Comments

@mrrezaie
Copy link

mrrezaie commented Oct 18, 2024

Bug summary

Hi, I use another package (opensim), which has lots of dll and exe files, along with matplotlib. It seems that matplotlib is broking something at import.

Actual outcome

If I import matplotlib first, I receive this error:

In [1]: import matplotlib
In [2]: import opensim
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-2-d100d6ac66f2> in ?()
----> 1 import opensim

F:\Python312\Lib\site-packages\opensim\__init__.py in ?()
     16 #     if (os.path.exists(dev_path)):
     17 #         os.add_dll_directory(dev_path)
     18
     19 from .simbody import *
---> 20 from .common import *
     21 from .simulation import *
     22 from .actuators import *
     23 from .analyses import *

F:\Python312\Lib\site-packages\opensim\common.py in ?()
      6
      7 from sys import version_info as _swig_python_version_info
      8 # Import the low-level C/C++ module
      9 if __package__ or "." in __name__:
---> 10     from . import _common
     11 else:
     12     import _common
     13

ImportError: DLL load failed while importing _common: A dynamic link library (DLL) initialization routine failed.

Expected outcome

If I import opensim, followed by matplotlib, everything works well:

In [1]: import opensim
In [2]: import matplotlib
In [3]: 

Operating system

Windows 11

Matplotlib Version

3.9.2

Python version

3.12.7

Installation

pip

@tacaswell
Copy link
Member

How did you install opensim? It does not look like it is on pypi and their docs reccomend using conda (https://opensimconfluence.atlassian.net/wiki/spaces/OpenSim/pages/53085346/Scripting+in+Python).

While wheels mostly work, there are some technical and inter-project coordination issues where they will fail with extension modules (see https://pypackaging-native.github.io for more details). In the short term I suggest sticking with conda or pixi (https://pixi.sh/latest/) for creating your environments and steer clear of wheels.

My suspicion is that both libraries are built against and bundle different versions of a shared dependency (which wheels require us to do), likely the c++ runtime, with the condition that the code built against the older version can run with the newer version of the library, but code built against the newer version can not run with the older. Based on the order that works, I think this means that mpl is built with the older version and opensim is built with the newer version.

I suspect that this is a relative of #28551


In the future, please always copy-paste error messages rather than screen shots. Text is much easier to deal with and while not super relevant here, if someone wants to try to reproduce

@jklymak
Copy link
Member

jklymak commented Oct 18, 2024

Have you opened an issue with Opensim? It is their library that can't load.

@mrrezaie
Copy link
Author

Thanks @tacaswell and @jklymak for your responses.

Have you opened an issue with Opensim? It is their library that can't load.

OpenSim actually works well, even if I import it after numpy, scipy, or scikit-learn. But, in this specific condition the error is raised.

In the future, please always copy-paste error messages rather than screen shots. Text is much easier to deal with and while not super relevant here, if someone wants to try to reproduce.

Sorry, I just changed the errors to text.

How did you install opensim?

I built opensim-core for win64 and py3.12 using Visual Studio 17 2022. Do you think the version of MSVC is the issue?

I investigated more and checked the imported modules using sys.modules, the kiwisolver package is actually the reason of that failure. Any idea on this?

Any help is greatly appreciated.

@jklymak
Copy link
Member

jklymak commented Oct 18, 2024

You probably need to build all the packages with c extensions the same way. Or install everything with conda/mamba if that is available.

@mrrezaie mrrezaie changed the title [Bug]: matplotlib interference with other package [Bug]: matplotlib (kiwisolver) interference with other package Oct 18, 2024
@mrrezaie
Copy link
Author

I see that many users have opened similar issue, but this is not matplotlib-specific. Is kiwisolver a required dependency? What are the limitations without kiwisolver? I could see that it has vey limited application in matplotlib, only here:

import kiwisolver as kiwi

By any chance, would it be possible to make it optional? So, no one would ever open any issue related to DLL ImportError.

@jklymak
Copy link
Member

jklymak commented Oct 19, 2024

It's used for the constrained layout manager. We don't currently have optional dependencies and I'm not sure how easy it is to make something optional. Many of our examples use constrained layout so making it optional would mean we would have to rework all of those.

But I'm not very familiar with windows packaging. Is there some reason we/kiwisolver/our users cannot fix this?

@tacaswell
Copy link
Member

tacaswell commented Oct 20, 2024

pip install --no-binary kiwisolver kiwisolver

will avoid using the kiwisolver wheels and compile it from source. I suspect that will solve the problem for @mrrezaie (not using pip/wheel and using conda/mamba/pixi instead is a more general solution).

I suspect (but am not 100% sure) that there is a technical path to making this work with wheels (it may require a one or more no-change "releases" to rebuild wheels), but issues stemming from packaging issues with one of the many distribution channels is not sufficient reason to make it an optional dependency.

@jklymak
Copy link
Member

jklymak commented Oct 20, 2024

I do wonder if we should consider vendoring kiwisolver so if there are build problems they will be for all of Matplotlib and not just in a dependency. I also am not sure how kiwisolver feels about the added maintenance burden of supporting our very broad user base; maybe @MatthieuDartiailh has an opinion?

@mrrezaie
Copy link
Author

mrrezaie commented Oct 21, 2024

Hi, thanks for your help. The pip install --no-binary command didn't help, and the same error was raised (hope I did it correctly).

I found this post (related to matplotlib and kiwisolver), and installed Microsoft Visual C++ Redistributable, it fixed the issue completely!!! Just curious, is there any integration? For example, pip install kiwisolver could detect the os and install or trigger the users to install an appropriate vc_redist afterwards. This would avoid lots of headaches :)

@QuLogic
Copy link
Member

QuLogic commented Oct 22, 2024

kiwisolver applied the same fix we did: nucleic/kiwi@7695597 but you must be using the latest version.

As this has been fixed there, I don't think we should do much else ourselves.

@QuLogic QuLogic closed this as not planned Won't fix, can't repro, duplicate, stale Oct 22, 2024
@MatthieuDartiailh
Copy link
Contributor

I am fine with doing whatever update help streamline the process as I did following the suggestion from @QuLogic.

Side note, as a user of matplotlib I very much enjoy the constraints based layout and wished it was the default so I am happy to help make kiwi integration easier.

@jklymak
Copy link
Member

jklymak commented Oct 22, 2024

Thanks @MatthieuDartiailh awesome - as long as we aren't imposing on you we are more than happy to not vendor Kiwisolver. Just wanted to make sure it wasn't an unintended obligation on your part 😉 Thanks for maintaining it, as it is a very nice abstraction of some difficult concepts.

@mrrezaie
Copy link
Author

Thanks all, please let me continue this discussion here a little bit further.

Any idea why pip install kiwisolver --no-binary kiwisolver didn't fix the issue? I did this and built opensim in GitHub Actions windows-2022 runner which has Visual Studio Enterprise 2022 by default, then downloaded the wheel files for local installation.

@MatthieuDartiailh
And, if installing the Microsoft Visual C++ Redistributable is a permanent solution for non-developers and non-conda users like me, would it be possible to guide users during installation? For example, a simple warning after kiwisolver [pip] installation that the incompatibilities between packages could be resolved by installing Microsoft Visual C++ Redistributable, followed by the link to the download page.

Thanks.

@jklymak
Copy link
Member

jklymak commented Oct 22, 2024

@mrrezaie if you are still having problems installing Kiwisolver, and you are using the latest version of kiwisolver, then perhaps the place to discuss is on the kwiwsolver issue tracker. Per the above, we think this issue was solved in the latest release.

If you are not a developer, I strongly suggest using conda/mamba for installing things that require compilation, particularly for windows, particularly if there is no wheel available for opens. Then you get the benefit of folks having already done this for you (assuming you have a supported machine).

@mrrezaie
Copy link
Author

we think this issue was solved in the latest release.

The latest version is 1.4.7 (https://pypi.org/project/kiwisolver/) since Sep 4 and at the time of opening this issue.

Anyway, thank again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants