BUG: have _array_from_buffer_3118 correctly handle errors
#10882
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5ca9322 to
8c479ee
Compare
eric-wieser
commented
Apr 11, 2018
numpy/core/tests/test_multiarray.py
Outdated
| # construct a memoryview with 33 dimensions | ||
| ct = ctypes.c_uint8 | ||
| for i in range(33): | ||
| ct = 1 * ct |
There was a problem hiding this comment.
Tests fail due to a reference cycle involving ct, which is somehow detected in a completely different test. Printing gc.garbage in the failing test gives:
# ndim = 18
A = numpy.core.tests.test_multiarray.c_ubyte_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1
B = numpy.core.tests.test_multiarray.c_ubyte_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1_Array_1
[<class 'A'>,
(<class 'A'>,
<class '_ctypes.Array'>,
<class '_ctypes._CData'>,
<class 'object'>),
{'__dict__': <attribute '__dict__' of 'A' objects>,
'__doc__': None,
'__module__': 'numpy.core.tests.test_multiarray',
'__weakref__': <attribute '__weakref__' of 'A' objects>,
'_length_': 1,
'_type_': <class 'B'>},
<attribute '__weakref__' of 'A' objects>,
<attribute '__dict__' of 'A' objects>]
This happens even if we remove the np.array call below, so the problem must be in either CPython, or parallelization in pytest
Fixes numpygh-6741 If an object claims to support the PEP3118 buffer protocol, then any mistakes in the implementation should be raised as errors, rather than falling back on other approaches. This has some behavior changes: * np.array(mem_view_33d) now gives a more useful error message than "memoryview: unsupported format <B" * np.array(ctypes_pointer) now errors rather than creating an object array containing a single pointer Also adds support for 32-dimensional buffers, where previously the max was 31-dimensional.
99bf2ca to
e6ee8d3
Compare
e6ee8d3 to
63e9cfb
Compare
|
Thanks Eric. |
|
I'm not 100% confident that the behavior this patch fixes wasn't something that we rely upon. In particular, perhaps |
|
Yeah, I figure we will find out, and if not, it probably doesn't matter. But if you come upon some info that suggests it does matter, please update. These changes in behavior are tricky, but sometimes worth exploring. |
|
Yep, this broke |
eric-wieser
added a commit
to eric-wieser/numpy
that referenced
this pull request
May 26, 2018
Everything behaves a lot better if we let the array constructor handle it, which will use the ctypes PEP3118 support. Bugs this fixes: * Stale state being attached to pointer objects (fixes numpygh-2671, closes numpygh-6214) * Weird failure modes on structured arrays (fixes-10978) * A regression in numpygh-10882 (fixes numpygh-10968)
_array_from_buffer_3118 correctly handle errors
uds5501
pushed a commit
to uds5501/numpy
that referenced
this pull request
Jun 28, 2018
Everything behaves a lot better if we let the array constructor handle it, which will use the ctypes PEP3118 support. Bugs this fixes: * Stale state being attached to pointer objects (fixes numpygh-2671, closes numpygh-6214) * Weird failure modes on structured arrays (fixes-10978) * A regression in numpygh-10882 (fixes numpygh-10968)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes gh-6741, closes gh-9348
If an object claims to support the PEP3118 buffer protocol, then any mistakes in the implementation should be raised as errors, rather than falling back on other approaches.
This has some behavior changes:
Also adds support for 32-dimensional buffers, where previously the max was 31-dimensional.