Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Bug]: Making an RGB image from pickled data throws error #28448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
scottshambaugh opened this issue Jun 24, 2024 · 4 comments · Fixed by #28458
Closed

[Bug]: Making an RGB image from pickled data throws error #28448

scottshambaugh opened this issue Jun 24, 2024 · 4 comments · Fixed by #28458

Comments

@scottshambaugh
Copy link
Contributor

scottshambaugh commented Jun 24, 2024

Bug summary

Getting an error when saving an animated RGB image that was loaded from a pickled figure. I've isolated the error to matplotlib 3.9.0, with this code working in 3.8.3, which makes me think that this is to do with the pybind11 upgrade in #26275?

Things I've tried:

  • Grayscale images (eg data = np.random.rand(100, 100)) work.
  • Numpy v1.26.4 and v2.0.0 show no difference in behavior
  • This shows up at least on WSL and Ubuntu
  • In the debugger, both data.dtype and out.dtype are showing 'float64' prior to the _image.resample call.
    • However, if I re-cast the arrays with data = data.astype('float64'), out = ..., then the _image.resample call no longer fails!
    • If I re-cast only one, then out.dtype == data.dtype returns True, but on the function call I get the error ValueError: Input and output arrays have mismatched types
    • ... so something is up with the types, and the C++ code is bombing. But python is saying things line up.

See these parts of the source:

out = np.zeros(out_shape + data.shape[2:], data.dtype) # 2D->2D, 3D->3D.
if resample is None:
resample = image_obj.get_resample()
_image.resample(data, out, transform,
_interpd_[interpolation],
resample,
alpha,
image_obj.get_filternorm(),
image_obj.get_filterrad())

if (auto resampler =
(ndim == 2) ? (
(dtype.is(py::dtype::of<std::uint8_t>())) ? resample<agg::gray8> :
(dtype.is(py::dtype::of<std::int8_t>())) ? resample<agg::gray8> :
(dtype.is(py::dtype::of<std::uint16_t>())) ? resample<agg::gray16> :
(dtype.is(py::dtype::of<std::int16_t>())) ? resample<agg::gray16> :
(dtype.is(py::dtype::of<float>())) ? resample<agg::gray32> :
(dtype.is(py::dtype::of<double>())) ? resample<agg::gray64> :
nullptr) : (
// ndim == 3
(dtype.is(py::dtype::of<std::uint8_t>())) ? resample<agg::rgba8> :
(dtype.is(py::dtype::of<std::int8_t>())) ? resample<agg::rgba8> :
(dtype.is(py::dtype::of<std::uint16_t>())) ? resample<agg::rgba16> :
(dtype.is(py::dtype::of<std::int16_t>())) ? resample<agg::rgba16> :
(dtype.is(py::dtype::of<float>())) ? resample<agg::rgba32> :
(dtype.is(py::dtype::of<double>())) ? resample<agg::rgba64> :
nullptr)) {
Py_BEGIN_ALLOW_THREADS
resampler(
input_array.data(), input_array.shape(1), input_array.shape(0),
output_array.mutable_data(), output_array.shape(1), output_array.shape(0),
params);
Py_END_ALLOW_THREADS
} else {
throw std::invalid_argument("arrays must be of dtype byte, short, float32 or float64");
}

Code for reproduction

import io
import pickle
import numpy as np
import matplotlib.pyplot as plt
from pathlib import Path
from matplotlib.animation import FuncAnimation

dir = Path(__file__).parent.resolve()

# generate random rgb data
fig, ax = plt.subplots()
np.random.seed(0)
data = np.random.rand(100, 100, 3)
ax.imshow(data)

# pick the figure and reload
buf = io.BytesIO()
pickle.dump(fig, buf)
buf.seek(0)
fig_pickled = pickle.load(buf)

# Animate
def update(frame):
    return ax,

ani = FuncAnimation(fig_pickled, update, frames=2)

# Save the animation
filepath = dir / 'test.gif' 
ani.save(filepath)

Actual outcome

Exception has occurred: ValueError
arrays must be of dtype byte, short, float32 or float64
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/image.py", line 208, in _resample
    _image.resample(data, out, transform,
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/image.py", line 567, in _make_image
    output = _resample(  # resample rgb channels
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/image.py", line 952, in make_image
    return self._make_image(self._A, bbox, transformed_bbox, clip,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/image.py", line 653, in draw
    im, l, b, trans = self.make_image(
                      ^^^^^^^^^^^^^^^^
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/artist.py", line 72, in draw_wrapper
    return draw(artist, renderer)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/image.py", line 132, in _draw_list_compositing_images
    a.draw(renderer)
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/axes/_base.py", line 3110, in draw
    mimage._draw_list_compositing_images(
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/artist.py", line 72, in draw_wrapper
    return draw(artist, renderer)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/image.py", line 132, in _draw_list_compositing_images
    a.draw(renderer)
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/figure.py", line 3157, in draw
    mimage._draw_list_compositing_images(
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/artist.py", line 72, in draw_wrapper
    return draw(artist, renderer)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/artist.py", line 95, in draw_wrapper
    result = draw(artist, renderer, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/backends/backend_agg.py", line 387, in draw
    self.figure.draw(self.renderer)
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/backends/backend_agg.py", line 432, in print_raw
    FigureCanvasAgg.draw(self)
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/backend_bases.py", line 2054, in <lambda>
    print_method = functools.wraps(meth)(lambda *args, **kwargs: meth(
                                                                 ^^^^^
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/backend_bases.py", line 2204, in print_figure
    result = print_method(
             ^^^^^^^^^^^^^
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/backends/backend_qtagg.py", line 75, in print_figure
    super().print_figure(*args, **kwargs)
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/figure.py", line 3390, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/animation.py", line 371, in grab_frame
    self.fig.savefig(self._proc.stdin, format=self.frame_format,
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/lib/matplotlib/animation.py", line 1109, in save
    writer.grab_frame(**savefig_kwargs)
  File "/mnt/c/Users/Scott/Documents/Documents/Coding/matplotlib/_test_pybind11_error.py", line 35, in <module>
    ani.save(filepath)
ValueError: arrays must be of dtype byte, short, float32 or float64

Matplotlib Version

3.9.0

@scottshambaugh
Copy link
Contributor Author

scottshambaugh commented Jun 24, 2024

I am able to work around this issue by manually re-casting the image data prior to the call, so my hunch is that this is an error to do with the pickling:

Updated example with workaround:

import io
import pickle
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
from pathlib import Path
from matplotlib.animation import FuncAnimation

dir = Path(__file__).parent.resolve()

# generate random rgb data
fig, ax = plt.subplots()
np.random.seed(0)
data = np.random.rand(100, 100, 3)
ax.imshow(data)

# pick the figure and reload
buf = io.BytesIO()
pickle.dump(fig, buf)
buf.seek(0)
fig_pickled = pickle.load(buf)

# Workaround
ax = fig_pickled.get_axes()[0]
artists = ax.get_children()
for artist in artists:
    if isinstance(artist, mpl.image.AxesImage):
        array = artist.get_array()
        artist.set_array(array.data.astype('float64'))

# Animate
def update(frame):
    return ax,

ani = FuncAnimation(fig_pickled, update, frames=2)

# Save the animation
filepath = dir / 'test.gif' 
ani.save(filepath)

@scottshambaugh scottshambaugh added the topic: pickle Issues to do with pickling label Jun 24, 2024
@ianthomas23
Copy link
Member

I can reproduce this on macOS without animation using:

import io
import numpy as np
import matplotlib.pyplot as plt
import pickle

fig, ax = plt.subplots()

rng = np.random.default_rng(4181)
data = rng.uniform(size=(2, 2, 3))
axes_image = ax.imshow(data)
print(axes_image._A.shape, axes_image._A.dtype)
im = axes_image.make_image(None)[0]

buf = io.BytesIO()
pickle.dump(axes_image, buf)
buf.seek(0)
axes_image2 = pickle.load(buf)
print(axes_image2._A.shape, axes_image2._A.dtype)

#axes_image2._A = axes_image2._A.astype("float64")
print("Same dtype?", axes_image._A.dtype == axes_image2._A.dtype)

im = axes_image2.make_image(None)[0]

Using this you get a ValueError: arrays must be of dtype byte, short, float32 or float64. If you remove the # to force a dtype change it works fine.

The problem occurs on this line

(dtype.is(py::dtype::of<double>())) ? resample<agg::rgba64> :

After pickling and unpickling the numpy array dtype is fine from a Python point of view, but from a C++ pybind11 point of view the dtype has all the right properties but its PyObject has a different address so we conclude that it is not really a double (i.e. np.float64) dtype. I haven't got any further than this yet, but If my analysis is correct it should be possible to write a reproducer that doesn't use Matplotlib at all.

@scottshambaugh scottshambaugh changed the title [Bug]: Saving an animated pickled RGB images throws error [Bug]: Making an RGB image from pickled data throws error Jun 24, 2024
@tacaswell
Copy link
Member

Can we fallback to eq in the c++ code instead of is ? A version of this is reproducible without pickle:

import matplotlib.pyplot as plt
import numpy as np

fig, ax = plt.subplots()

rng = np.random.default_rng(4181)
data = rng.uniform(size=(2, 2, 3)).astype(np.dtype('float64', copy=True))
axes_image = ax.imshow(data)
print(axes_image._A.shape, axes_image._A.dtype)
im = axes_image.make_image(None)[0]

@ianthomas23
Copy link
Member

Can we fallback to eq in the c++ code instead of is ?

It looks like dtype1.equal(dtype2) is good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants