Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[3.12] gh-106939: document ShareableList nul-strip quirk. (GH-107266) #107269

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 25, 2023
Merged

[3.12] gh-106939: document ShareableList nul-strip quirk. (GH-107266) #107269

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 37 additions & 7 deletions Doc/library/multiprocessing.shared_memory.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -255,16 +255,17 @@ shared memory blocks created using that manager are all released when the
:keyword:`with` statement's code block finishes execution.


.. class:: ShareableList(sequence=None, *, name=None)
.. class:: ShareableList(sequence=None, \*, name=None)

Provides a mutable list-like object where all values stored within are
stored in a shared memory block. This constrains storable values to
only the ``int``, ``float``, ``bool``, ``str`` (less than 10M bytes each),
``bytes`` (less than 10M bytes each), and ``None`` built-in data types.
It also notably differs from the built-in ``list`` type in that these
lists can not change their overall length (i.e. no append, insert, etc.)
and do not support the dynamic creation of new :class:`ShareableList`
instances via slicing.
only the ``int`` (signed 64-bit), ``float``, ``bool``, ``str`` (less
than 10M bytes each when encoded as utf-8), ``bytes`` (less than 10M
bytes each), and ``None`` built-in data types. It also notably
differs from the built-in ``list`` type in that these lists can not
change their overall length (i.e. no append, insert, etc.) and do not
support the dynamic creation of new :class:`ShareableList` instances
via slicing.

*sequence* is used in populating a new ``ShareableList`` full of values.
Set to ``None`` to instead attach to an already existing
Expand All @@ -275,6 +276,35 @@ shared memory blocks created using that manager are all released when the
existing ``ShareableList``, specify its shared memory block's unique
name while leaving ``sequence`` set to ``None``.

.. note::

A known issue exists for :class:`bytes` and :class:`str` values.
If they end with ``\x00`` nul bytes or characters, those may be
*silently stripped* when fetching them by index from the
:class:`ShareableList`. This ``.rstrip(b'\x00')`` behavior is
considered a bug and may go away in the future. See :gh:`106939`.

For applications where rstripping of trailing nulls is a problem,
work around it by always unconditionally appending an extra non-0
byte to the end of such values when storing and unconditionally
removing it when fetching:

.. doctest::

>>> from multiprocessing import shared_memory
>>> nul_bug_demo = shared_memory.ShareableList(['?\x00', b'\x03\x02\x01\x00\x00\x00'])
>>> nul_bug_demo[0]
'?'
>>> nul_bug_demo[1]
b'\x03\x02\x01'
>>> nul_bug_demo.shm.unlink()
>>> padded = shared_memory.ShareableList(['?\x00\x07', b'\x03\x02\x01\x00\x00\x00\x07'])
>>> padded[0][:-1]
'?\x00'
>>> padded[1][:-1]
b'\x03\x02\x01\x00\x00\x00'
>>> padded.shm.unlink()

.. method:: count(value)

Returns the number of occurrences of ``value``.
Expand Down