Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BUG: Stack overflow on double inheritance from numpy.flexible and numpy.ma.core.MaskedArray #23737

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
JelleZijlstra opened this issue May 8, 2023 · 4 comments · Fixed by #24358
Open

BUG: Stack overflow on double inheritance from numpy.flexible and numpy.ma.core.MaskedArray #23737

JelleZijlstra opened this issue May 8, 2023 · 4 comments · Fixed by #24358
Labels
00 - Bug

Comments

@JelleZijlstra
Copy link

Describe the issue:

>>> import numpy
>>> numpy.__version__
'1.24.3'
>>> class X(numpy.flexible, numpy.ma.core.MaskedArray): pass
... 
>>> with memoryview(X()): pass
... 
zsh: segmentation fault  python

Reproduce the code example:

import numpy
class X(numpy.flexible, numpy.ma.core.MaskedArray): pass
with memoryview(X()): pass

Error message:

(lldb) target create "/Users/jelle/py/venvs/py311/bin/python"
Current executable set to '/Users/jelle/py/venvs/py311/bin/python' (arm64).
(lldb) r 
Process 62767 launched: '/Users/jelle/py/venvs/py311/bin/python' (arm64)
Python 3.11.1 (main, Dec 21 2022, 16:19:04) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.__version__
'1.24.3'
>>> class X(numpy.flexible, numpy.ma.core.MaskedArray):
...     pass
... 
>>> with memoryview(X()): pass
... 
Process 62767 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x16edfffc0)
    frame #0: 0x00000001a4226994 libsystem_malloc.dylib`nanov2_allocate_from_block + 8
libsystem_malloc.dylib`nanov2_allocate_from_block:
->  0x1a4226994 <+8>:  stp    x28, x27, [sp, #0x20]
    0x1a4226998 <+12>: stp    x26, x25, [sp, #0x30]
    0x1a422699c <+16>: stp    x24, x23, [sp, #0x40]
    0x1a42269a0 <+20>: stp    x22, x21, [sp, #0x50]
Target 0: (python) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x16edfffc0)
  * frame #0: 0x00000001a4226994 libsystem_malloc.dylib`nanov2_allocate_from_block + 8
    frame #1: 0x00000001a42261e0 libsystem_malloc.dylib`nanov2_allocate + 128
    frame #2: 0x00000001a42260fc libsystem_malloc.dylib`nanov2_malloc + 64
    frame #3: 0x00000001a4243748 libsystem_malloc.dylib`_malloc_zone_malloc + 156
    frame #4: 0x00000001000a0998 python`_PyObject_Malloc + 64
    frame #5: 0x00000001001eedcc python`state_init + 164
    frame #6: 0x00000001001ecf1c python`_sre_SRE_Pattern_match + 224
    frame #7: 0x0000000100058a2c python`method_vectorcall_FASTCALL_KEYWORDS_METHOD + 144
    frame #8: 0x000000010004e02c python`PyObject_VectorcallMethod + 144
    frame #9: 0x0000000100104b30 python`check_matched + 108
    frame #10: 0x00000001001035ac python`warn_explicit + 1284
    frame #11: 0x000000010010487c python`do_warn + 828
    frame #12: 0x0000000100102eac python`PyErr_WarnEx + 72
    frame #13: 0x00000001036ada78 _multiarray_umath.cpython-311-darwin.so`PyArray_DescrFromTypeObject + 496
    frame #14: 0x00000001036acbb8 _multiarray_umath.cpython-311-darwin.so`PyArray_DescrFromScalar + 212
    frame #15: 0x000000010009ef28 python`_PyObject_GenericGetAttrWithDict + 200
    frame #16: 0x000000010009e7dc python`PyObject_GetAttr + 100
    frame #17: 0x000000010009e720 python`PyObject_GetAttrString + 72
    frame #18: 0x00000001036acccc _multiarray_umath.cpython-311-darwin.so`PyArray_DescrFromScalar + 488
    frame #19: 0x000000010009ef28 python`_PyObject_GenericGetAttrWithDict + 200
    frame #20: 0x000000010009e7dc python`PyObject_GetAttr + 100
    frame #21: 0x000000010009e720 python`PyObject_GetAttrString + 72
    frame #22: 0x00000001036acccc _multiarray_umath.cpython-311-darwin.so`PyArray_DescrFromScalar + 488
    frame #23: 0x000000010009ef28 python`_PyObject_GenericGetAttrWithDict + 200
    frame #24: 0x000000010009e7dc python`PyObject_GetAttr + 100
    frame #25: 0x000000010009e720 python`PyObject_GetAttrString + 72
(lots more of this, snipped)
    frame #279490: 0x00000001036acccc _multiarray_umath.cpython-311-darwin.so`PyArray_DescrFromScalar + 488
    frame #279491: 0x000000010009ef28 python`_PyObject_GenericGetAttrWithDict + 200
    frame #279492: 0x000000010009e7dc python`PyObject_GetAttr + 100
    frame #279493: 0x0000000100138d50 python`_PyEval_EvalFrameDefault + 17896
    frame #279494: 0x00000001001346c8 python`_PyEval_Vector + 200
    frame #279495: 0x000000010004e300 python`object_vacall + 256
    frame #279496: 0x000000010004e564 python`PyObject_CallFunctionObjArgs + 48
    frame #279497: 0x0000000103603278 _multiarray_umath.cpython-311-darwin.so`PyArray_NewFromDescr_int + 1268
    frame #279498: 0x00000001035fa958 _multiarray_umath.cpython-311-darwin.so`PyArray_View + 84
    frame #279499: 0x0000000103688110 _multiarray_umath.cpython-311-darwin.so`array_view + 324
    frame #279500: 0x000000010005867c python`method_vectorcall_FASTCALL_KEYWORDS + 136
    frame #279501: 0x000000010004ccd4 python`PyObject_Vectorcall + 80
    frame #279502: 0x000000010013cf1c python`_PyEval_EvalFrameDefault + 34740
    frame #279503: 0x00000001001346c8 python`_PyEval_Vector + 200
    frame #279504: 0x000000010004c64c python`_PyObject_FastCallDictTstate + 272
    frame #279505: 0x000000010004d394 python`_PyObject_Call_Prepend + 148
    frame #279506: 0x00000001000b6b68 python`slot_tp_new + 88
    frame #279507: 0x00000001000b3550 python`type_call + 84
    frame #279508: 0x000000010004c834 python`_PyObject_MakeTpCall + 344
    frame #279509: 0x000000010013cf1c python`_PyEval_EvalFrameDefault + 34740
    frame #279510: 0x0000000100134598 python`PyEval_EvalCode + 272
    frame #279511: 0x000000010018c5a8 python`PyRun_InteractiveOneObjectEx + 744
    frame #279512: 0x000000010018bad0 python`_PyRun_InteractiveLoopObject + 152
    frame #279513: 0x000000010018b97c python`_PyRun_AnyFileObject + 96
    frame #279514: 0x000000010018c234 python`PyRun_AnyFileExFlags + 68
    frame #279515: 0x00000001001ac1dc python`Py_RunMain + 2196
    frame #279516: 0x00000001001ac424 python`pymain_main + 324
    frame #279517: 0x00000001001ac4c4 python`Py_BytesMain + 40
    frame #279518: 0x000000010057508c dyld`start + 520

Runtime information:

>>> import sys, numpy; print(numpy.__version__); print(sys.version)
1.24.3
3.11.1 (main, Dec 21 2022, 16:19:04) [Clang 14.0.0 (clang-1400.0.29.202)]
>>> print(numpy.show_runtime())
[{'simd_extensions': {'baseline': ['NEON', 'NEON_FP16', 'NEON_VFPV4', 'ASIMD'],
                      'found': ['ASIMDHP', 'ASIMDDP'],
                      'not_found': ['ASIMDFHM']}},
 {'architecture': 'armv8',
  'filepath': '/Users/jelle/py/venvs/py311/lib/python3.11/site-packages/numpy/.dylibs/libopenblas64_.0.dylib',
  'internal_api': 'openblas',
  'num_threads': 8,
  'prefix': 'libopenblas',
  'threading_layer': 'pthreads',
  'user_api': 'blas',
  'version': '0.3.21'}]
None

Context for the issue:

No concrete use case; I found this while trying to instantiate all pairs of buffer classes to look for real-world impact of python/cpython#104297.
@seberg
Copy link
Member

seberg commented May 8, 2023

Thanks! numpy.flexible is an abstract base class really (or maybe should be one). It should not be valid to subclass it in Python at all.
(I am a bit surprised nothing prevents this, since masked array inherits from ndarray and ndarray and flexible have incompatible layouts.)

@JelleZijlstra
Copy link
Author

The memoryview() part in my original report was unnecessary; just instantiating the class is enough:

>>> class X(numpy.flexible, numpy.ma.core.MaskedArray): pass
... 
>>> X()
zsh: segmentation fault  python

tylerjereddy added a commit to tylerjereddy/numpy that referenced this issue Aug 7, 2023
@tylerjereddy
BUG: flexible inheritance segfault
4570f1f 
* Fixes numpy#23737

* based on the discussion in the issue above, this
completely blocks subclassing `np.flexible` at the Python
level and adds a test to ensure the segfault codepath
is no longer accessible
@tylerjereddy tylerjereddy mentioned this issue Aug 7, 2023
Merged
tylerjereddy added a commit to tylerjereddy/numpy that referenced this issue Aug 7, 2023
@tylerjereddy
BUG: flexible inheritance segfault
aa9b23d 
* Fixes numpy#23737

* based on the discussion in the issue above, this
completely blocks subclassing `np.flexible` at the Python
level and adds a test to ensure the segfault codepath
is no longer accessible
@mattip
Copy link
Member

mattip commented Aug 13, 2023

@seberg could you expand on "numpy.flexible is an abstract base class" and "It should not be valid to subclass it in Python"? I think abstract base classes are designed to be subclasses, no? Or do you mean numpy.flexible is an internal NumPy class, and should not be exposed to users. In any case, removing the ability to create a subclass of numpy.flexible should break this line, and allowing it seems like a bug in CPython.

numpy/numpy/core/src/multiarray/multiarraymodule.c

Line 464 in 250e147

Reopening for discussion.

@mattip mattip reopened this Aug 13, 2023
@seberg
Copy link
Member

seberg commented Aug 13, 2023

I think CPython is fine with allowing it in C presuming you know what you do, but not sure it is intentional or not.

Yes, I suppose you are right. Subclassing it makes in theory sense (honestly, I don't think it does in practice, you cannot have a flexible user dtype in the old machinery anyway).

So yes, it makes a bit more sense to allow it, but move the actual buffer stuff to the concrete subclasses, becasuse there is no reason to have anything related in the superclass. I am not even sure if it is used there, so maybe it can just be deleted from there.

FFY00 pushed a commit to FFY00/numpy that referenced this issue Aug 14, 2023
@tylerjereddy @FFY00
BUG: flexible inheritance segfault
e00c8b0 
* Fixes numpy#23737

* based on the discussion in the issue above, this
completely blocks subclassing `np.flexible` at the Python
level and adds a test to ensure the segfault codepath
is no longer accessible

Signed-off-by: Filipe Laíns <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

BUG: flexible inheritance segfault tylerjereddy/numpy
3 participants
@seberg @mattip @JelleZijlstra