[Bug]: Ragged Poly3DCollection projects uninitialized padding, causing nondeterministic overflow #32272

Bug summary

Matplotlib 3.11.0 can project uninitialized padding when Poly3DCollection receives polygons with different numbers of vertices. This occurs with Axes3D.plot_surface() when its default downsampling produces non-uniform surface patches.

Code for reproduction

import io

import matplotlib.pyplot as plt
import numpy as np
from mpl_toolkits.mplot3d.art3d import Poly3DCollection

# Different vertex counts make Poly3DCollection create padded storage.
square = np.zeros((4, 3))
triangle = np.zeros((3, 3))
collection = Poly3DCollection([square, triangle])

# Simulate a large value left in the uninitialized padding. Matplotlib
# already marks this entry as invalid, so it should never be projected.
collection._faces[collection._invalid_vertices] = np.finfo(float).max

figure = plt.figure()
axis = figure.add_subplot(projection="3d")
axis.add_collection(collection, autolim=False)
axis.set(xlim=(0, 1), ylim=(0, 1), zlim=(0, 1))

figure.savefig(io.BytesIO(), format="jpg")

Actual outcome

mpl_toolkits/mplot3d/proj3d.py:184: RuntimeWarning:
overflow encountered in dot
  product = np.dot(axes.M, vec)

Expected outcome

Poly3DCollection marks padding for shorter polygons as invalid, but projects the complete backing array before applying that mask. The MRE places a fixed large value only in the entries Matplotlib has marked invalid. Those values should have no effect, but they are passed to the projection matrix and cause an overflow. This deterministically reproduces the projection problem without depending on arbitrary heap contents.

Additional information

Matplotlib 3.11.0 can project uninitialized padding when Poly3DCollection receives polygons with different numbers of vertices.

This occurs with Axes3D.plot_surface() when its default downsampling produces non-uniform surface patches. Poly3DCollection._get_vector() converts the ragged polygons into a rectangular array:

segments = np.empty((num_faces, max_verts, 3))
for i, face in enumerate(segments3d):
    segments[i, :len(face)] = face

Unused entries are marked in _invalid_vertices, but do_3d_projection() projects the complete array before applying that mask:

pfaces = proj3d._scale_proj_transform_vectors(self._faces, self.axes)

The uninitialized entries can contain values near the float64 maximum, resulting in:

RuntimeWarning: overflow encountered in dot

The behavior is nondeterministic because it depends on heap contents.

This appears related to the vectorized ragged Poly3DCollection handling introduced by #29397, implementing the performance work from #16659.

Workaround

Force plot_surface() to use its uniform full-resolution path:

ax.plot_surface(x, y, z, rcount=z.shape[0], ccount=z.shape[1])

Equivalent:

ax.plot_surface(x, y, z, rstride=1, cstride=1)

This produces equally sized quadrilateral faces and avoids the ragged padded representation.

Suggested Fix

Invalid padding should not participate in projection. Possible approaches include:

• Initialize padding with a safe value rather than np.empty().
• Replace or exclude invalid padding before projection.
• Flatten and project only valid vertices, then reconstruct the polygon boundaries.

Padding with NaN was already discussed in #16659, although the current implementation uses uninitialized storage.

Operating system

Windows 11, 64-bit

Matplotlib Version

3.11.0

Matplotlib Backend

TkAgg / Agg during JPEG output

Python version

3.14.6

Jupyter version

No response

Installation

uv