Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit c9dc4a2

Browse files
committed
Issue #17810: Implement PEP 3154, pickle protocol 4.
Most of the work is by Alexandre.
1 parent 95401c5 commit c9dc4a2

12 files changed

Lines changed: 3123 additions & 997 deletions

File tree

Doc/library/pickle.rst

Lines changed: 26 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -459,12 +459,29 @@ implementation of this behaviour::
459459
Classes can alter the default behaviour by providing one or several special
460460
methods:
461461

462+
.. method:: object.__getnewargs_ex__()
463+
464+
In protocols 4 and newer, classes that implements the
465+
:meth:`__getnewargs_ex__` method can dictate the values passed to the
466+
:meth:`__new__` method upon unpickling. The method must return a pair
467+
``(args, kwargs)`` where *args* is a tuple of positional arguments
468+
and *kwargs* a dictionary of named arguments for constructing the
469+
object. Those will be passed to the :meth:`__new__` method upon
470+
unpickling.
471+
472+
You should implement this method if the :meth:`__new__` method of your
473+
class requires keyword-only arguments. Otherwise, it is recommended for
474+
compatibility to implement :meth:`__getnewargs__`.
475+
476+
462477
.. method:: object.__getnewargs__()
463478

464-
In protocol 2 and newer, classes that implements the :meth:`__getnewargs__`
465-
method can dictate the values passed to the :meth:`__new__` method upon
466-
unpickling. This is often needed for classes whose :meth:`__new__` method
467-
requires arguments.
479+
This method serve a similar purpose as :meth:`__getnewargs_ex__` but
480+
for protocols 2 and newer. It must return a tuple of arguments `args`
481+
which will be passed to the :meth:`__new__` method upon unpickling.
482+
483+
In protocols 4 and newer, :meth:`__getnewargs__` will not be called if
484+
:meth:`__getnewargs_ex__` is defined.
468485

469486

470487
.. method:: object.__getstate__()
@@ -496,10 +513,10 @@ the methods :meth:`__getstate__` and :meth:`__setstate__`.
496513

497514
At unpickling time, some methods like :meth:`__getattr__`,
498515
:meth:`__getattribute__`, or :meth:`__setattr__` may be called upon the
499-
instance. In case those methods rely on some internal invariant being true,
500-
the type should implement :meth:`__getnewargs__` to establish such an
501-
invariant; otherwise, neither :meth:`__new__` nor :meth:`__init__` will be
502-
called.
516+
instance. In case those methods rely on some internal invariant being
517+
true, the type should implement :meth:`__getnewargs__` or
518+
:meth:`__getnewargs_ex__` to establish such an invariant; otherwise,
519+
neither :meth:`__new__` nor :meth:`__init__` will be called.
503520

504521
.. index:: pair: copy; protocol
505522

@@ -511,7 +528,7 @@ objects. [#]_
511528

512529
Although powerful, implementing :meth:`__reduce__` directly in your classes is
513530
error prone. For this reason, class designers should use the high-level
514-
interface (i.e., :meth:`__getnewargs__`, :meth:`__getstate__` and
531+
interface (i.e., :meth:`__getnewargs_ex__`, :meth:`__getstate__` and
515532
:meth:`__setstate__`) whenever possible. We will show, however, cases where
516533
using :meth:`__reduce__` is the only option or leads to more efficient pickling
517534
or both.

Doc/whatsnew/3.4.rst

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,7 @@ New expected features for Python implementations:
109109
Significantly Improved Library Modules:
110110

111111
* Single-dispatch generic functions in :mod:`functoools` (:pep:`443`)
112+
* New :mod:`pickle` protocol 4 (:pep:`3154`)
112113
* SHA-3 (Keccak) support for :mod:`hashlib`.
113114
* TLSv1.1 and TLSv1.2 support for :mod:`ssl`.
114115
* :mod:`multiprocessing` now has option to avoid using :func:`os.fork`
@@ -285,6 +286,20 @@ described in the PEP. Existing importers should be updated to implement
285286
the new methods.
286287

287288

289+
Pickle protocol 4
290+
=================
291+
292+
The new :mod:`pickle` protocol addresses a number of issues that were present
293+
in previous protocols, such as the serialization of nested classes, very
294+
large strings and containers, or classes whose :meth:`__new__` method takes
295+
keyword-only arguments. It also brings a couple efficiency improvements.
296+
297+
.. seealso::
298+
299+
:pep:`3154` - Pickle protocol 4
300+
PEP written by Antoine Pitrou and implemented by Alexandre Vassalotti.
301+
302+
288303
Other Language Changes
289304
======================
290305

Lib/copyreg.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,12 @@ def _reduce_ex(self, proto):
8787
def __newobj__(cls, *args):
8888
return cls.__new__(cls, *args)
8989

90+
def __newobj_ex__(cls, args, kwargs):
91+
"""Used by pickle protocol 4, instead of __newobj__ to allow classes with
92+
keyword-only arguments to be pickled correctly.
93+
"""
94+
return cls.__new__(cls, *args, **kwargs)
95+
9096
def _slotnames(cls):
9197
"""Return a list of slot names for a given class.
9298

0 commit comments

Comments
 (0)