You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I also looked at getting rid of the flag entirely but unfortunately the pickle format encodes dtype="O" as "O8" on 64 bit systems, coming from this line:
Unfortunately we didn't change that to writing just "O" instead of "O4" or "O8" when we deprecated this and added the global flag back in 2012, so we need to keep the evil global flag. I have no idea if changing how object dtypes get written to pickle files would break other things or subtly break compatibility across numpy versions.
However, we can at least make the evil global flag thread safe by marking it as a thread-local variable, just like the promotion state.
If we ever wanted to support subinterpreters, we'd need to move this state into the module state and it would no longer need to be thread-local.
You could test this by having one thread load a pickle file and another thread create e.g. an "O4" dtype simultaneously. In practice it's tricky to actually write such a test because pytest isn't thread-safe, so I can't use e.g. pytest.warns inside a thread.
ngoldbaum
changed the title
MNT: mark evil_global_disable_warn_O4O8_flag as thread-local
MAINT: mark evil_global_disable_warn_O4O8_flag as thread-local
Jun 12, 2024
If we ever wanted to support subinterpreters, we'd need to move this state into the module state and it would no longer need to be thread-local.
Subinterpreters can be free-threaded, no? So it would have to be thread-local there also, I think?
OTOH, I suspect that thread-local is sufficient (enough?) there, since I am not sure it is possible for someone to pause one subinterpreter and then run a different one in the same thread while we are inside the unpickling (C-API) function.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I also looked at getting rid of the flag entirely but unfortunately the pickle format encodes dtype="O" as "O8" on 64 bit systems, coming from this line:
numpy/numpy/_core/src/multiarray/descriptor.c
Line 2744 in 1854f26
Unfortunately we didn't change that to writing just "O" instead of "O4" or "O8" when we deprecated this and added the global flag back in 2012, so we need to keep the evil global flag. I have no idea if changing how object dtypes get written to pickle files would break other things or subtly break compatibility across numpy versions.
However, we can at least make the evil global flag thread safe by marking it as a thread-local variable, just like the promotion state.
If we ever wanted to support subinterpreters, we'd need to move this state into the module state and it would no longer need to be thread-local.
You could test this by having one thread load a pickle file and another thread create e.g. an "O4" dtype simultaneously. In practice it's tricky to actually write such a test because pytest isn't thread-safe, so I can't use e.g.
pytest.warnsinside a thread.