Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BUG: PyObject_GetBuffer on a numpy array does not match the Python spec #28272

Open
@jabraham17

Description

@jabraham17

Describe the issue:

I am finding that numpy's implementation of the bf_getbuffer function does not match the Python docs. The docs specify that certain fields are request-independent and should always be filled in, regardless of flags (see https://docs.python.org/3/c-api/buffer.html#request-independent-fields). These are obj, buf, len, itemsize, and ndim. I interpret this to mean with flags=PyBUF_SIMPLE that ndim should be set to a value. However this is not the case (as the code example shows). Reading the implementation of array_getbuffer in numpy/_core/src/multiarray/buffer.c shows that only when PyBUF_ND is apart of the flags is ndim filled in, which I believe is incorrect.

Note that the following patch is sufficient to fix the issue, if others agree I am happy to open a PR with it

diff --git a/numpy/_core/src/multiarray/buffer.c b/numpy/_core/src/multiarray/buffer.c
index fcff3ad6ca..a45399cfab 100644
--- a/numpy/_core/src/multiarray/buffer.c
+++ b/numpy/_core/src/multiarray/buffer.c
@@ -820,12 +820,11 @@ array_getbuffer(PyObject *obj, Py_buffer *view, int flags)
     } else {
         view->format = NULL;
     }
+    view->ndim = info->ndim;
     if ((flags & PyBUF_ND) == PyBUF_ND) {
-        view->ndim = info->ndim;
         view->shape = info->shape;
     }
     else {
-        view->ndim = 0;
         view->shape = NULL;
     }
     if ((flags & PyBUF_STRIDES) == PyBUF_STRIDES) {

Reproduce the code example:

#define PY_SSIZE_T_CLEAN
#include <Python.h>

// clang bug.c `python3-config --embed --cflags` `python3-config --embed --ldflags`

PyObject* newList() {
  PyObject* lst = PyList_New(5);
  if (!lst) {
    return NULL;
  }
  PyList_SetItem(lst, 0, PyLong_FromLong(1));
  PyList_SetItem(lst, 1, PyLong_FromLong(2));
  PyList_SetItem(lst, 2, PyLong_FromLong(3));
  PyList_SetItem(lst, 3, PyLong_FromLong(4));
  PyList_SetItem(lst, 4, PyLong_FromLong(5));
  return lst;
}

int main() {
  Py_Initialize();


  PyObject* numpy = PyImport_ImportModule("numpy");
  if (!numpy) {
    PyErr_Print();
    return 1;
  }

  PyObject* numpy_array = PyObject_GetAttrString(numpy, "array");
  if (!numpy_array) {
    PyErr_Print();
    return 1;
  }

  PyObject* lst = newList();
  if (!lst) {
    PyErr_Print();
    return 1;
  }
  PyObject* array = PyObject_CallOneArg(numpy_array, lst);
  if (!array) {
    PyErr_Print();
    return 1;
  }

  PyObject* ndim = PyObject_GetAttrString(array, "ndim");
  if (!ndim) {
    PyErr_Print();
    return 1;
  }
  printf("ndim from numpy: %ld\n", PyLong_AsLong(ndim));

  if (PyObject_CheckBuffer(array)) {
    Py_buffer view;
    PyObject_GetBuffer(array, &view, PyBUF_SIMPLE);
    printf("ndim from buffer: %d\n", view.ndim);
    PyBuffer_Release(&view);
  } else {
    return 1;
  }

  Py_Finalize();

  return 0;
}

Error message:

ndim from numpy: 1
ndim from buffer: 0

Python and NumPy Versions:

numpy 2.2.1
pythom 3.12.7

Runtime Environment:

[{'numpy_version': '2.2.1',
'python': '3.12.7 (main, Oct 1 2024, 02:05:46) [Clang 15.0.0 '
'(clang-1500.1.0.2.5)]',
'uname': uname_result(system='Darwin', node='<>', release='22.6.0', version='Darwin Kernel Version 22.6.0: Wed Jul 5 22:22:05 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T6000', machine='arm64')},
{'simd_extensions': {'baseline': ['NEON', 'NEON_FP16', 'NEON_VFPV4', 'ASIMD'],
'found': ['ASIMDHP'],
'not_found': ['ASIMDFHM']}},
{'architecture': 'neoversen1',
'filepath': '.venv/lib/python3.12/site-packages/numpy/.dylibs/libscipy_openblas64_.dylib',
'internal_api': 'openblas',
'num_threads': 10,
'prefix': 'libscipy_openblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.28'}]

Context for the issue:

The Chapel language team is working on Python interop support, especially being able to interop closely with numpy arrays. I don't think this issue blocks that work in anyway (passing PyBUF_ND is an suitable workaround), but it would be nice to fix this and match the Python docs

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions