-
-
Notifications
You must be signed in to change notification settings - Fork 7.9k
use class level rather than instance level caching of fontd #798
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…the too many open file handles windows bug
Ah. Good solution. |
Because this problem seems to clearly stem from too many font file openings, I am surprised that the call to FT_Done_Face in the FT2Font destructor is not enough to take care of this (in conjunction with garbage collection). This makes me wonder if we need to be triggering garbage collecting during the tests. When pyplot.close is called on a figure, it triggers a gc collect via _pylab_helpers.Gcf.destroy, and the CleanupTest does call close('all') on every test, so we should be getting a gc.collect on every test. So i don't understand why these font openings are creating too many open handles during the test. This leads me to one other possibility: we have been building and packaging freetype with the mpl windows release since before the dawn of time. If we haven't updated our version of freetype in a long time that we a re bundling, could we be seeing a bug in that version. @cgohlke , what version are you compiling mpl against, and if it's ancient could you consider upgrading? It would interesting to know if this fixes the problem even w/o this PR. Interestingly, backend_ps and backend_pdf already do the caching at the renderer class level. Unsure, why agg was doing it at the renderer level, I went to check git blame to see if maybe it had been moved in and we could get some insight into why. Unfortunately, I found a commit by me entitled "moved agg caches to instance namespace for thread safety" 175e3ec so this was done for a reason. So we probably can't just merge this as is: the question is, can we get a smarter class level font cache that implements some sort of thread safe locking? I am a thread novice, so don't feel like I am the best person to address this. @cgohlke , because there is a problem with this PR and thread safety, it doubles my interest in the question above: can we fix this by upgrading freetype? And does anyone else have any ideas why these handles are not getting freed? To rehash the above, in v1.1.x the font file handles are in the FT2Font instances, these font objects are in a Renderer instance level dict associated with the figure, the figure is getting closed by the test cleanup, this triggers gc.collect, and this should trigger the destruction of the figure which should lead to the call to the FT2Font destructor which in turn calls FT_Done_Face. In that pipeline, the file handle should be freed. So it looks like the logic is right on the mpl end. |
I am using freetype 2.4.9, the latest version available. |
…f draw, and push it back up when done
OK, scratch the second commit. I think the right approach here is to use a threading Lock right before drawing and release it right after drawing. Now we have a single class level cache that is locked during a single renderer draw. |
|
@mdboom the PR is good to test now. I had tried something more involved in the second commit, and ended up pulling it all for the simpler lock in the third commit. I had also written a long comment after the second commit which I subsequently deleted. So "scratch the second" commit is just me talking to myself and anyone reading the history to say that that line wasn't fruitful. I think what we have in there now is good. |
Thanks, replacing the rc version of backend_agg.py with the one in a00e510 resolves this issue for me on win32 and 64 and passes all the tests. |
Works for me on win-amd64-py2.7 |
This looks good to me, also. I say we merge. Still agreed? |
Yep, go ahead. Sorry I've been out of the loop recently. Just getting back from vacation and will have some time to close loose ends and finish this release |
use class level rather than instance level caching of fontd
windows users are reporting too many open file handles exceptions which appear to be triggered by too many font files being opened and parsed. I think we may be able to dramatically reduce these for normal use cases by caching the agg font dictionary at the renderer class level rather than instance level.
This needs testing and confirmation by a windows user before merging.