Thanks to visit codestin.com
Credit goes to github.com

Skip to content

FileNotFoundError while import matplotlib (maybe pyplot) #9485

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ha-nso-li opened this issue Oct 20, 2017 · 17 comments · Fixed by #9504
Closed

FileNotFoundError while import matplotlib (maybe pyplot) #9485

ha-nso-li opened this issue Oct 20, 2017 · 17 comments · Fixed by #9504
Milestone

Comments

@ha-nso-li
Copy link

Bug report

Traceback

Traceback (most recent call last):
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1444, in <module>
    fontManager = json_load(_fmcache)
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 973, in json_load
    with open(filename, 'r') as fh:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\hanso\\.matplotlib\\fontList.json'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/hanso/PycharmProjects/reinforcement-learning-kr/2-cartpole/1-dqn/cartpole_dqn.py", line 3, in <module>
    import pylab
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\pylab.py", line 1, in <module>
    from matplotlib.pylab import *
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\pylab.py", line 257, in <module>
    from matplotlib import cbook, mlab, pyplot as plt
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\pyplot.py", line 29, in <module>
    import matplotlib.colorbar
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\colorbar.py", line 36, in <module>
    import matplotlib.contour as contour
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\contour.py", line 21, in <module>
    import matplotlib.font_manager as font_manager
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1454, in <module>
    _rebuild()
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1434, in _rebuild
    fontManager = FontManager()
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1060, in __init__
    self.ttffiles = findSystemFonts(paths) + findSystemFonts()
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 323, in findSystemFonts
    for f in win32InstalledFonts(fontdir):
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 240, in win32InstalledFonts
    direc = os.path.abspath(direc).lower()
  File "C:\Users\hanso\AppData\Local\Programs\Python\Python35\lib\ntpath.py", line 535, in abspath
    path = _getfullpathname(path)
ValueError: _getfullpathname: embedded null character

And I couldn't find C:\Users\hanso\.matplotlib\fontList.json

Code for reproduction

https://github.com/rlcode/reinforcement-learning-kr/blob/master/2-cartpole/1-dqn/cartpole_dqn.py

Matplotlib version

  • Operating system: Windows 10 64bit 1703
  • Matplotlib version: 2.1.0
  • Matplotlib backend : TkAgg
  • Python version: 3.5.4

I installed matplotlib via pip.

@ha-nso-li
Copy link
Author

I maybe solve this problem.

In font_manager.py, line 240:

                try:
                    key, direc, any = winreg.EnumValue( local, j)
                    if not isinstance(direc, six.string_types):
                        continue
                    if not os.path.dirname(direc):
                        direc = os.path.join(directory, direc)
                    # Comment out these 3 lines
                    # direc = os.path.abspath(direc).lower()
                    # if os.path.splitext(direc)[1][1:] in fontext:
                    #    items[direc] = 1
                    # and add following one line
                    direc = direc.split('\0', 1)[0]
                except EnvironmentError:
                    continue
                except WindowsError:
                    continue
                except MemoryError:
                    continue

Now it seems work perfectly, but I'm not sure the problem is really solved.

@anntzer
Copy link
Contributor

anntzer commented Oct 20, 2017

I am almost certain I have already seen this issue reported, but can't find it anymore...
In any case, can you print what is the value of direc just before the part you commented out?

@ha-nso-li
Copy link
Author

@anntzer

print(direc) shows C:\\WINDOWS\\Fonts\\HYHWPEQ.TTF HyhwpEQ
but the space is '\0' (null character) not whitespace.

image

@tacaswell tacaswell added this to the v2.1.1 milestone Oct 20, 2017
@tacaswell
Copy link
Member

The FileNoteFound error is expected (that is how we know if we need to rebuild the font cache). The bug is we are not paranoid enough in the handling of the paths that come back from windows.

While that patch runs, I am not sure it runs properly. The items dictionary which is being built up is finally returned as a list of font files. This effectively disables this method of finding fonts!

@dasolyn Thanks for digging into this!

@anntzer
Copy link
Contributor

anntzer commented Oct 20, 2017

  1. Can you look up what values does your windows registry report for the keys "SOFTWARE\Microsoft\Windows\CurrentVersion\Fonts" and "SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts".

  2. Does "C:\WINDOWS\Fonts\HYHWPEQ.TTF" actually exist in your filesystem?

  3. What does reg QUERY "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts at the command line give you?

The relevant docs(?) https://technet.microsoft.com/en-us/library/cc976493.aspx does not suggest anything about null bytes...

@ha-nso-li
Copy link
Author

ha-nso-li commented Oct 20, 2017

@anntzer

  1. \HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Fonts doesn't exist. \HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts has lots of string values, including HyhwpEQ.
    image

  2. Yes
    image

  3. It has 554 lines. https://pastebin.com/hSJG5g7J

@anntzer
Copy link
Contributor

anntzer commented Oct 20, 2017

I guess dropping the part about the null byte should be good enough. However, just to be sure, can you print each value of direc in the loop (as it comes out of winreg.EnumValue)? I just want to see whether there are funky things other than null bytes...

In fact this looks like a bug in cpython's winreg: REG_SZ is documented as a "null terminated string" (https://msdn.microsoft.com/en-us/library/windows/desktop/ms724884(v=vs.85).aspx and https://docs.python.org/3/library/winreg.html#winreg.REG_SZ), so the returned value should stop at the first null. Or perhaps not... maybe it means terminated by a utf-16 null, i.e. two null bytes. See https://github.com/python/cpython/blob/master/PC/winreg.c#L719

@ha-nso-li
Copy link
Author

@anntzer

            for j in range(winreg.QueryInfoKey(local)[1]):
                try:
                    key, direc, any = winreg.EnumValue( local, j)
                    if not isinstance(direc, six.string_types):
                        continue
                    if not os.path.dirname(direc):
                        direc = os.path.join(directory, direc)
                    print('j={}, key={}, direc={}, any={}, null={}'.format(j, key, direc, any, '\0' in direc))
                    if '\0' in direc:
                        direc = direc.split('\0', 1)[0]
                    direc = os.path.abspath(direc).lower()
                    if os.path.splitext(direc)[1][1:] in fontext:
                        items[direc] = 1
                except EnvironmentError:
                    continue
                except WindowsError:
                    continue
                except MemoryError:
                    continue

Result: https://pastebin.com/bFQe3SZh

Only hyhwpeq.ttf has a null character. Others are OK. (maybe?)

@Kojoley
Copy link
Member

Kojoley commented Oct 21, 2017

I still do not understand where the null char comes from.
Could you please dump and upload the registry? reg export "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Fonts" fontsdump.reg

@anntzer
Copy link
Contributor

anntzer commented Oct 21, 2017

Found it: it's a bug in CPython itself... reported by a mpl user :-) https://bugs.python.org/issue25778

@tacaswell
Copy link
Member

This is a 3.5-only bug as well.

@anntzer
Copy link
Contributor

anntzer commented Oct 21, 2017

3.6.0 is definitely affected; probably 3.4 (https://bugs.python.org/issue25778#msg257316) too.

@Cory-Kramer
Copy link

Cory-Kramer commented Sep 19, 2018

Just upgraded matplotlib from 2.2.3 to 3.0.0 and ran into this exact same issue.

Versions
Operating system: Windows 10 64bit
Matplotlib version: 3.0.0
Python version: 3.5.4

Code to reproduce
import matplotlib.pyplot as plt

This will produce a similar, but not identical stacktrace to the OP

Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import matplotlib.pyplot as plt
  File "C:\Python\Python35\lib\site-packages\matplotlib\pyplot.py", line 32, in <module>
    import matplotlib.colorbar
  File "C:\Python\Python35\lib\site-packages\matplotlib\colorbar.py", line 32, in <module>
    import matplotlib.contour as contour
  File "C:\Python\Python35\lib\site-packages\matplotlib\contour.py", line 18, in <module>
    import matplotlib.font_manager as font_manager
  File "C:\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1363, in <module>
    _rebuild()
  File "C:\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1344, in _rebuild
    fontManager = FontManager()
  File "C:\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 978, in __init__
    ttffiles = findSystemFonts(paths) + findSystemFonts()
  File "C:\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 264, in findSystemFonts
    fontfiles.update(win32InstalledFonts(fontext=fontext))
TypeError: 'NoneType' object is not iterable

We have pinned back to 2.2.3 until a fix or workaround is available.

@tacaswell
Copy link
Member

@Cory-Kramer Can you open a new issue for this? This was fixed in 2.1.1 so the breaking between 2.2.3 and 3.0 is a new 🐛 .

@DrDeviprasad
Copy link

DrDeviprasad commented Sep 22, 2018

Traceback (most recent call last):
  File "House_Prediction.py", line 9, in <module>
    import matplotlib.pyplot as plt
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\pyplot.py", line 32, in <module>
    import matplotlib.colorbar
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\colorbar.py", line 32, in <module>
    import matplotlib.contour as contour
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\contour.py", line 18, in <module>
    import matplotlib.font_manager as font_manager
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1363, in <module>
    _rebuild()
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 1344, in _rebuild
    fontManager = FontManager()
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 978, in __init__
    ttffiles = findSystemFonts(paths) + findSystemFonts()
  File "C:\Users\drdev\AppData\Local\Programs\Python\Python35\lib\site-packages\matplotlib\font_manager.py", line 264, in findSystemFonts
    fontfiles.update(win32InstalledFonts(fontext=fontext))
TypeError: 'NoneType' object is not iterable

Even after installing the latest version of matplotlib, this is not working. From the above comments it looks like an bug, doesn't seem to fixed in the code

# KNOWN ISSUES
#
#   - documentation
#   - font variant is untested
#   - font stretch is incomplete
#   - font size is incomplete
#   - default font algorithm needs improvement and testing
#   - setWeights function needs improvement
#   - 'light' is an invalid weight value, remove it.
#   - update_fonts not implemented

def findSystemFonts(fontpaths=None, fontext='ttf'):
    """
    Search for fonts in the specified font paths.  If no paths are
    given, will use a standard set of system paths, as well as the
    list of fonts tracked by fontconfig if fontconfig is installed and
    available.  A list of TrueType fonts are returned by default with
    AFM fonts as an option.
  """
    fontfiles = set()
    fontexts = get_fontext_synonyms(fontext)

    if fontpaths is None:
        if sys.platform == 'win32':
            fontpaths = [win32FontDirectory()]
            # now get all installed fonts directly...
            fontfiles.update(win32InstalledFonts(fontext=fontext))
        else:
            fontpaths = X11FontDirectories
            fontfiles.update(get_fontconfig_fonts(fontext))
            # check for OS X & load its fonts if present
            if sys.platform == 'darwin':
                fontfiles.update(OSXInstalledFonts(fontext=fontext))

    elif isinstance(fontpaths, str):
        fontpaths = [fontpaths]

    for path in fontpaths:
        fontfiles.update(map(os.path.abspath, list_fonts(path, fontexts)))

    return [fname for fname in fontfiles if os.path.exists(fname)]

@DrDeviprasad
Copy link

Please advise

@Kojoley
Copy link
Member

Kojoley commented Sep 22, 2018

@DrDeviprasad please follow to the #12173 issue

@matplotlib matplotlib deleted a comment from RDCH106 Oct 12, 2018
@matplotlib matplotlib locked as resolved and limited conversation to collaborators Oct 12, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants