-
Notifications
You must be signed in to change notification settings - Fork 32
Open
Labels
Description
Currently when pickling an IMP C++ object we serialize into a C++ std::string and then pass that back to Python as a bytes object, which Python can then pickle. This creates several copies of the underlying data. Consider adding support for pickle protocol version 5 if available (Python >= 3.8, or the backported pickle5 module in Python >= 3.5) as per PEP 574 to store the data in a Python buffer. This should in principle use less memory and be faster, particularly for large objects such as IMP::em::DensityMap or IMP::Model. For example, see the work done for NumPy arrays at numpy/numpy#12011.