Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Fix flaky text tests #8708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 10, 2017
Merged

Fix flaky text tests #8708

merged 4 commits into from
Jun 10, 2017

Conversation

QuLogic
Copy link
Member

@QuLogic QuLogic commented Jun 3, 2017

As noted in #7911, the mathtext tests sometimes fail if the id of the renderer is the same between tests. Since it's difficult to add the mathtext fontset to the key, I fixed/hacked it into working by adding a cache-busting "unique" value to each renderer, which is just a globally-incremented number. I thought of using a UUID or random number, but that seemed too expensive since we don't need to worry about threading locks on globals (AFAIK).

I ran the test from #7911, but using the repeat plugin to run everything 100 times over (so 50000 tests total.) So I'm not 100% certain, but I believe this fixes #7911.

Additionally, we've lately been seeing several failures of test_savefig_to_stringio on AppVeyor. This doesn't usually fail locally, but seems to be a timeout on the TeX cache. I've added some retries on these tests to hopefully reduce the problem.

PR Checklist

  • Has Pytest style unit tests
  • Code is PEP 8 compliant
  • New features are documented, with examples if plot related
  • Documentation is sphinx and numpydoc compliant
  • Added an entry to doc/users/whats_new.rst if major new feature
  • Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

QuLogic added 3 commits June 1, 2017 21:10
2.5.5 is what's in my conda env, and 2.6.1 is what's built with the
local_freetype option and is used on Travis and AppVeyor without issue.
The result of id() is only guaranteed to be unique for objects that
exist at the same time. This global integer is a quick and light way to
ensure that new renderers that match an old one don't produce the same
caching key.
The Mathtext tests should no longer be flaky based on the previous
change.

Mark some PS tests as flaky because they require a lock that sometimes
gets stuck on AppVeyor due to the multiple processes.
@QuLogic QuLogic added this to the 2.1 (next point release) milestone Jun 3, 2017
@QuLogic QuLogic requested a review from tacaswell June 3, 2017 06:25
@QuLogic
Copy link
Member Author

QuLogic commented Jun 3, 2017

Huh, just realized that @tacaswell outlined basically this procedure in #7911 but I didn't recall it; must have been stuck in my subconscious somewhere.

@tacaswell
Copy link
Member

tacaswell commented Jun 3, 2017

I don't think this is thread safe, and that is ok. I do not think we make the claim that Matplotlib is threadsafe in any way. Further most of our time is spent in python and I do not think we release the GIL in the c extensions so it is not clear to me you would get any major gains (unless you really are I/O bound).

Although the way that this will fail to be thread safe also means both objects existing at the same time in which case the id() value should be different. If the race condition is between two threads calling _uid on the same renderer at the same time then the user is trying to render the same object on more than one thread and I expect other things to break.

There are two places to get race conditions here (see https://www.youtube.com/watch?v=7SSYhuk5hmc for lots more details) If you look
x = 5


class Test:
    
    def foo():
        global x
        x += 1
        self.cache = x

and

In [91]: t = Test()

In [93]: dis.dis(t.foo)
  8           0 LOAD_GLOBAL              0 (x)
              2 LOAD_CONST               1 (1)
              4 INPLACE_ADD
              6 STORE_GLOBAL             0 (x)

  9           8 LOAD_GLOBAL              0 (x)
             10 LOAD_GLOBAL              1 (self)
             12 STORE_ATTR               2 (cache)
             14 LOAD_CONST               0 (None)
             16 RETURN_VALUE

I think you can get races conditions between all of the op-codes in line 8 which missing an increment and between lines 8 and 9 (both increment, but then both use the higher value).

@anntzer
Copy link
Contributor

anntzer commented Jun 3, 2017

Should this just go into RendererBase.__hash__? (then get_prop_tup would just stick renderer into the tuple instead of id(renderer), renderer._uid.)

@tacaswell
Copy link
Member

We probably do not want to go down the __hash__ route without implementing all of other __eq__ and friends methods.

@QuLogic
Copy link
Member Author

QuLogic commented Jun 3, 2017

For thread-safety, I was thinking more of subsequent renderers. Across the threads, the renderers would have different id anyway, so it'd be fine if they ended up with the same value.

Theoretically, thread 1 could create a couple renderers in the time it takes thread 2 to store the wrong value. Then the next time thread 1 created a new id-conflicting renderer it would have the wrong uid. But something would have to be seriously wrong with thread 2 if thread 1 was able to create multiple renderers in the time it took to load+add+store.

@anntzer
Copy link
Contributor

anntzer commented Jun 4, 2017

Actually you can just use itertools.count as a threadsafe counter (create _counter at the module level and call next(_counter) to get a uid). Yes, this is only due to a CPython implementation detail (i.e. the GIL) but it's not worse than the current PR and should be at least as fast (or better) (in any case I doubt that's the slowest part of rendering).

@QuLogic
Copy link
Member Author

QuLogic commented Jun 4, 2017

That's a good idea; even if not thread-safe on all implementation, it's at least clearer.

part of a caching key.
"""
if self._id is None:
self._id = next(_unique_renderer_id)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would just make this a regular attribute created at instantiation (and leave a comment in the ctor). I'm not sure what making this a property buys you...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant to remove this now when switching to itertools but forgot about it.

@QuLogic
Copy link
Member Author

QuLogic commented Jun 6, 2017

@tomspur This patch should help with https://bugzilla.redhat.com/show_bug.cgi?id=1401267 (though leave out the pytest parts if on 1.5.3.)

Copy link
Member

@tacaswell tacaswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved before and it is much better now.

@anntzer anntzer merged commit b07fbb8 into matplotlib:master Jun 10, 2017
@anntzer
Copy link
Contributor

anntzer commented Jun 10, 2017

thanks!

@QuLogic QuLogic deleted the flaky-tests branch June 10, 2017 04:50
@matthew-brett
Copy link
Contributor

@QuLogic - great - thanks for doing that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

mathtext/mathfont intermittent failures
4 participants