Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 9a2d99e

Browse files
author
Stefan Krah
committed
- Issue #10181: New memoryview implementation fixes multiple ownership
and lifetime issues of dynamically allocated Py_buffer members (#9990) as well as crashes (#8305, #7433). Many new features have been added (See whatsnew/3.3), and the documentation has been updated extensively. The ndarray test object from _testbuffer.c implements all aspects of PEP-3118, so further development towards the complete implementation of the PEP can proceed in a test-driven manner. Thanks to Nick Coghlan, Antoine Pitrou and Pauli Virtanen for review and many ideas. - Issue #12834: Fix incorrect results of memoryview.tobytes() for non-contiguous arrays. - Issue #5231: Introduce memoryview.cast() method that allows changing format and shape without making a copy of the underlying memory.
1 parent 5a3d046 commit 9a2d99e

24 files changed

Lines changed: 9896 additions & 1056 deletions

Doc/c-api/buffer.rst

Lines changed: 321 additions & 164 deletions
Large diffs are not rendered by default.

Doc/c-api/memoryview.rst

Lines changed: 19 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -17,16 +17,19 @@ any other object.
1717
1818
Create a memoryview object from an object that provides the buffer interface.
1919
If *obj* supports writable buffer exports, the memoryview object will be
20-
readable and writable, otherwise it will be read-only.
20+
read/write, otherwise it may be either read-only or read/write at the
21+
discretion of the exporter.
2122
23+
.. c:function:: PyObject *PyMemoryView_FromMemory(char *mem, Py_ssize_t size, int flags)
24+
25+
Create a memoryview object using *mem* as the underlying buffer.
26+
*flags* can be one of :c:macro:`PyBUF_READ` or :c:macro:`PyBUF_WRITE`.
2227
2328
.. c:function:: PyObject *PyMemoryView_FromBuffer(Py_buffer *view)
2429
2530
Create a memoryview object wrapping the given buffer structure *view*.
26-
The memoryview object then owns the buffer represented by *view*, which
27-
means you shouldn't try to call :c:func:`PyBuffer_Release` yourself: it
28-
will be done on deallocation of the memoryview object.
29-
31+
For simple byte buffers, :c:func:`PyMemoryView_FromMemory` is the preferred
32+
function.
3033
3134
.. c:function:: PyObject *PyMemoryView_GetContiguous(PyObject *obj, int buffertype, char order)
3235
@@ -43,10 +46,16 @@ any other object.
4346
currently allowed to create subclasses of :class:`memoryview`.
4447
4548
46-
.. c:function:: Py_buffer *PyMemoryView_GET_BUFFER(PyObject *obj)
49+
.. c:function:: Py_buffer *PyMemoryView_GET_BUFFER(PyObject *mview)
50+
51+
Return a pointer to the memoryview's private copy of the exporter's buffer.
52+
*mview* **must** be a memoryview instance; this macro doesn't check its type,
53+
you must do it yourself or you will risk crashes.
54+
55+
.. c:function:: Py_buffer *PyMemoryView_GET_BASE(PyObject *mview)
4756
48-
Return a pointer to the buffer structure wrapped by the given
49-
memoryview object. The object **must** be a memoryview instance;
50-
this macro doesn't check its type, you must do it yourself or you
51-
will risk crashes.
57+
Return either a pointer to the exporting object that the memoryview is based
58+
on or *NULL* if the memoryview has been created by one of the functions
59+
:c:func:`PyMemoryView_FromMemory` or :c:func:`PyMemoryView_FromBuffer`.
60+
*mview* **must** be a memoryview instance.
5261

Doc/c-api/typeobj.rst

Lines changed: 58 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1198,46 +1198,74 @@ Buffer Object Structures
11981198

11991199
.. sectionauthor:: Greg J. Stein <[email protected]>
12001200
.. sectionauthor:: Benjamin Peterson
1201+
.. sectionauthor:: Stefan Krah
12011202

1203+
.. c:type:: PyBufferProcs
12021204
1203-
The :ref:`buffer interface <bufferobjects>` exports a model where an object can expose its internal
1204-
data.
1205+
This structure holds pointers to the functions required by the
1206+
:ref:`Buffer protocol <bufferobjects>`. The protocol defines how
1207+
an exporter object can expose its internal data to consumer objects.
12051208

1206-
If an object does not export the buffer interface, then its :attr:`tp_as_buffer`
1207-
member in the :c:type:`PyTypeObject` structure should be *NULL*. Otherwise, the
1208-
:attr:`tp_as_buffer` will point to a :c:type:`PyBufferProcs` structure.
1209+
.. c:member:: getbufferproc PyBufferProcs.bf_getbuffer
12091210
1211+
The signature of this function is::
12101212

1211-
.. c:type:: PyBufferProcs
1213+
int (PyObject *exporter, Py_buffer *view, int flags);
1214+
1215+
Handle a request to *exporter* to fill in *view* as specified by *flags*.
1216+
A standard implementation of this function will take these steps:
1217+
1218+
- Check if the request can be met. If not, raise :c:data:`PyExc_BufferError`,
1219+
set :c:data:`view->obj` to *NULL* and return -1.
1220+
1221+
- Fill in the requested fields.
1222+
1223+
- Increment an internal counter for the number of exports.
1224+
1225+
- Set :c:data:`view->obj` to *exporter* and increment :c:data:`view->obj`.
1226+
1227+
- Return 0.
1228+
1229+
The individual fields of *view* are described in section
1230+
:ref:`Buffer structure <buffer-structure>`, the rules how an exporter
1231+
must react to specific requests are in section
1232+
:ref:`Buffer request types <buffer-request-types>`.
1233+
1234+
All memory pointed to in the :c:type:`Py_buffer` structure belongs to
1235+
the exporter and must remain valid until there are no consumers left.
1236+
:c:member:`~Py_buffer.shape`, :c:member:`~Py_buffer.strides`,
1237+
:c:member:`~Py_buffer.suboffsets` and :c:member:`~Py_buffer.internal`
1238+
are read-only for the consumer.
1239+
1240+
:c:func:`PyBuffer_FillInfo` provides an easy way of exposing a simple
1241+
bytes buffer while dealing correctly with all request types.
1242+
1243+
:c:func:`PyObject_GetBuffer` is the interface for the consumer that
1244+
wraps this function.
1245+
1246+
.. c:member:: releasebufferproc PyBufferProcs.bf_releasebuffer
1247+
1248+
The signature of this function is::
1249+
1250+
void (PyObject *exporter, Py_buffer *view);
12121251

1213-
Structure used to hold the function pointers which define an implementation of
1214-
the buffer protocol.
1252+
Handle a request to release the resources of the buffer. If no resources
1253+
need to be released, this field may be *NULL*. A standard implementation
1254+
of this function will take these steps:
12151255

1216-
.. c:member:: getbufferproc bf_getbuffer
1256+
- Decrement an internal counter for the number of exports.
12171257

1218-
This should fill a :c:type:`Py_buffer` with the necessary data for
1219-
exporting the type. The signature of :data:`getbufferproc` is ``int
1220-
(PyObject *obj, Py_buffer *view, int flags)``. *obj* is the object to
1221-
export, *view* is the :c:type:`Py_buffer` struct to fill, and *flags* gives
1222-
the conditions the caller wants the memory under. (See
1223-
:c:func:`PyObject_GetBuffer` for all flags.) :c:member:`bf_getbuffer` is
1224-
responsible for filling *view* with the appropriate information.
1225-
(:c:func:`PyBuffer_FillView` can be used in simple cases.) See
1226-
:c:type:`Py_buffer`\s docs for what needs to be filled in.
1258+
- If the counter is 0, free all memory associated with *view*.
12271259

1260+
The exporter MUST use the :c:member:`~Py_buffer.internal` field to keep
1261+
track of buffer-specific resources (if present). This field is guaranteed
1262+
to remain constant, while a consumer MAY pass a copy of the original buffer
1263+
as the *view* argument.
12281264

1229-
.. c:member:: releasebufferproc bf_releasebuffer
12301265

1231-
This should release the resources of the buffer. The signature of
1232-
:c:data:`releasebufferproc` is ``void (PyObject *obj, Py_buffer *view)``.
1233-
If the :c:data:`bf_releasebuffer` function is not provided (i.e. it is
1234-
*NULL*), then it does not ever need to be called.
1266+
This function MUST NOT decrement :c:data:`view->obj`, since that is
1267+
done automatically in :c:func:`PyBuffer_Release`.
12351268

1236-
The exporter of the buffer interface must make sure that any memory
1237-
pointed to in the :c:type:`Py_buffer` structure remains valid until
1238-
releasebuffer is called. Exporters will need to define a
1239-
:c:data:`bf_releasebuffer` function if they can re-allocate their memory,
1240-
strides, shape, suboffsets, or format variables which they might share
1241-
through the struct bufferinfo.
12421269

1243-
See :c:func:`PyBuffer_Release`.
1270+
:c:func:`PyBuffer_Release` is the interface for the consumer that
1271+
wraps this function.

0 commit comments

Comments
 (0)