MAINT: Import time: Avoid importing python.Threading #14098
Conversation
88a59d0 to
922cee9
Compare
|
@eric-wieser do you know why azure won't run on this? I'm pretty sure CPython will work, just not PyPy. |
|
I think some heisenbug or so on windows came up a bit often on azure the last few PRs? (not quite sure though). |
|
Yeah, I think so too. |
numpy/random/bit_generator.pyx
Outdated
| from cpython.pycapsule cimport PyCapsule_New | ||
|
|
||
| # Threading isn't the cheapest import. Import from the lightweight | ||
| # _thread module, | ||
| from _thread import allocate_lock as Lock |
There was a problem hiding this comment.
It seems to me that the _thread method might be choking the pypy test? (i.e. that lock waits forever?). How OK is it really to use the _thread name?
There was a problem hiding this comment.
The python STL was documenting it.
But I guess I misunderstood what they meant by low level.
https://docs.python.org/3/library/concurrency.html
I kinda knew I was doing a bad thing, so I didn't try it first. I think I might have to rerun initial benchmarks now that some of the other PRs are going through
There was a problem hiding this comment.
Dunno, it seems that python might have started to endorse _thread specfically, and possibly pypy just has a bug.
There was a problem hiding this comment.
I have to start rerunning these benchmarks again. Some of these fixes are related to deprecation cleanups. Others like this one are more hazy. One could argue that importing from _thread is obscure and shouldn't be done even if it was supported by both PyPy.
There was a problem hiding this comment.
root@c766aae4e273:/# pypy3.6-v7.1.1-linux64/bin/pypy3
Python 3.6.1 (784b254d6699, Apr 14 2019, 10:22:42)
[PyPy 7.1.1-beta0 with GCC 6.2.0 20160901] on linux
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``snow, snow''
>>>> import _thread
>>>> from _thread import allocate_lock as Lock
>>>> Lock
<built-in function allocate_lock>
This seems to work on the latest downloaded PyPy
|
How does this behave on systems whether threading is unavailable? |
|
Oh I guess that is what dummy threading was for. Not too sure for now. PyPy has threading right? |
|
But you can compile python without threading. PyPy should fail the import if it had no threading, but it seems rather that it hangs on an infinite Lock somewhere. |
|
I am not sure about this PR. It seems to increase code fragility for a minimal decrease in import time. |
922cee9 to
37b41e7
Compare
37b41e7 to
5ab3cb3
Compare
|
I think it might have a more measurable effect once #14097 gets pulled in. I was able to get down to 70 ms in my original omnibus PR. I can bechmark after that gets approved again. |
5ab3cb3 to
974bf38
Compare
|
It might be difficult to see the results of this without getting rid of a Unfortunately, |
|
I'm not sure upstream will accept using https://github.com/python/cpython/blob/master/Lib/threading.py#L95 That said, even pypy has RLock implemented in C, so maybe :/. It might be more fruitful to fix upstream. They don't seem to have a c implementation of |
|
FYI, it seems that the above comment is under development, and might actually be fixed in python 3.8. |
See here:
#14083
Just testing on CIs for now.