Thanks to visit codestin.com
Credit goes to github.com

Skip to content

FIX: move the font lock higher up the call and class tree #26302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 15, 2023

Conversation

tacaswell
Copy link
Member

PR summary

We have for a long time (~2012) had an RLock on RenderAgg that was used in FigureCanvasAgg.draw to protect the shared cache of ft2font objects (the underlying c++ is very stateful and not thread safe). This lock was also implicitly protecting the mathtext cache when using the Agg backend.

However, given the recent improvements to the layout code there are now ways to call call Figure.draw(renderer) without acquiring this lock which leads to exceptions when using mathtext and a layout manager to save independent figures on different threads.

This bug also exists for the other backendends which use both the mathtext parser and the ft2font cache in the case of rendering texts as paths (in the vector backends).

The fix is to:

  • pull the lock up to RendererBase so all renderer instances share a single lock
  • acquire the lock in Figure.draw which is always the top entry point to rendering a Figure

Closes #26289

Not sure how to test this. I guess we could add a test like:

from matplotlib.figure import Figure

import io
import threading


def test_crash2():
    for i in range(100):
        fig = Figure(tight_layout=True)
        # fig = Figure()

        ax = fig.add_subplot(1, 1, 1)

        ax.text(0, 0, "test $\pm$ 1.2")  # This crashes
        # ax.text(0, 0, 'test 1.2') # This does not

        buf = io.BytesIO()
        fig.savefig(buf, format="svg")


if True:
    threads = [threading.Thread(target=test_crash2) for _ in range(10)]
    [t.start() for t in threads]
    [t.join() for t in threads]

but it seems a bit of a dice roll if you put in enough iterations to feel safe you never lost the race but few enough it does not take too long.

If this is an acceptable change, I think we should also add a section to the docs clarifying what level of threading we think we support and what we do not (one thread per figure should work in my opinion).

PR checklist

@tacaswell tacaswell added this to the v3.8.0 milestone Jul 13, 2023
# draw the figure bounding box, perhaps none for white figure
if not self.get_visible():
return
with getattr(renderer, "lock", nullcontext()):
Copy link
Contributor

@anntzer anntzer Jul 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As argued elsewhere, I think we should either just document that third-party renderers must inherit RendererBase (and thus skip the getattr here), or at least add a check to warn that renderers with no lock attribute are deprecated. (... If I understand the point of the getattr correctly.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is the point.

I'm thinking about the best place to put that check.

Copy link
Member Author

@tacaswell tacaswell Jul 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, the other option is we stick the lock on Figure on on some module.

The other analogy I know of a global-ish lock like this is the global lock in h5py to protect calling into libhdf5: https://github.com/h5py/h5py/blob/6b5af4c6495bf865fee5f036122187c21fcb17d4/h5py/_objects.pyx#L40-L46 .

Expecting the instance to carry the lock like this is "nice" in that things seem well encapsulated, but is leaves us open to some backend opting out (or using a different lock) and bringing back a super subtle version of this bug ("this only happens when I save a mix of svg and png in a multi-threaded environment....").

I've talked my self into making this a private class level attribute on Figure.

We can make it public later if we need to.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, on the figure seems fine, too.

@tacaswell tacaswell force-pushed the fix/tightlayout_threading branch from b7ec8e9 to 2c1ecd8 Compare July 14, 2023 02:35
@tacaswell
Copy link
Member Author

tacaswell commented Jul 14, 2023

I also added a test for one of the changed lines that was not previously covered.

There are obviously other ways to get at the caches (I suspect if you ask a Text object how big it is outside of draw / draw_without_rendering you can still get your self in trouble) but that was already the case and this at least gets the "standard" paths covered.

@tacaswell
Copy link
Member Author

All of the failures are codecov uploads failing.

Comment on lines 3142 to 3143
if not self.get_visible():
return
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leave this outside the lock to save acquiring the lock and releasing right away?

tacaswell and others added 2 commits July 14, 2023 19:20
We have for a long time (~2012) had an RLock on `RenderAgg` that was used in
`FigureCanvasAgg.draw` to protect the shared cache of ft2font objects (the
underlying c++ is very stateful and not thread safe). This lock was also
implicitly protecting the mathtext cache when using the Agg backend.

However, given the recent improvements to the layout code there are now ways to
call call `Figure.draw(renderer)` without acquiring this lock which leads to
exceptions when using mathtext and a layout manager to save independent figures
on different threads.

This bug also exists for the other backendends which use both the mathtext
parser and the ft2font cache in the case of rendering texts as paths (in the
vector backends).

The fix is to:

 - pull the lock up to `Figure` so all renderer instances effectively share a
   single lock
 - acquire the lock in `Figure.draw` which is always the top entry point to
   rendering a Figure.

Closes matplotlib#26289

Co-authored-by: Greg Lucas <[email protected]>
@tacaswell tacaswell force-pushed the fix/tightlayout_threading branch from 4dbff5e to 339dcb1 Compare July 14, 2023 23:20
@ksunden ksunden merged commit 7499015 into matplotlib:main Jul 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: mathtext caching issue in multi-threaded environment with tight_layout=True
4 participants