Fix crash reading variable-length region reference data - #2966

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.52%. Comparing base (fbf1164) to head (c0f32b8).
⚠️ Report is 12 commits behind head on master.

@@            Coverage Diff             @@
##           master    #2966      +/-   ##
==========================================
+ Coverage   90.47%   90.52%   +0.04%     
==========================================
  Files          16       16              
  Lines        2521     2532      +11     
==========================================
+ Hits         2281     2292      +11     
  Misses        240      240              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Forgot to mention, I used memset() because there is no currently ecalloc() in h5py. Willing to add ecalloc() to replace the emalloc()/memset() combo.

I plan to merge this on September 7, unless changes are requested before then.

It's September 8, so feel free to merge now if you want but just as a note, I'm finally going back to my backlog and I could review this after lunch (a couple hours from now) if that's still on the table.

so it decrefs whatever pointer happened to be on the heap.

Let me get this straight: back_buf is only ever included in bkg_obj if it's initialized first ? If so, where is that behavior defined ? Otherwise, clearly I must be misunderstanding something.

It's September 8

I forgot yesterday was a holiday in the US. 😁

Let me get this straight: back_buf is only ever included in bkg_obj if it's initialized first ? If so, where is that behavior defined ? Otherwise, clearly I must be misunderstanding something.

You are not misunderstanding it, there is no such condition. back_buf is passed through to bkg_obj unconditionally whenever a background buffer is requested at all, initialized or not. Below is the chain of calls I followed for this bug, all in _conv.templ.pyx on this branch:

1. conv_vlen2ndarray allocates it but does not zero the buffer at line 724, back_buf = emalloc(H5Tget_size(outtype.id)*size).
2. The buffer goes straight into H5Tconvert(intype.id, outtype.id, size, data, back_buf, H5P_DEFAULT) at line 728, as the background argument.
3. HDF5 hands that pointer to the registered converter as bkg_i. For this conversion that is conv_regref2pyref, registered at line 474. Note the H5T_BKG_YES there.
4. generic_converter (line 58) does char* bkg = <char*>bkg_i and then, per element, op(buf + i*src_size, buf + i*dst_size, bkg + i*bkg_stride, priv) around lines 93-97.
5. conv_regref2pyref (line 386) casts that straight back: bkg_obj = <PyObject**>bkg, reads bkg_obj0 = bkg_obj[0], and calls Py_XDECREF(bkg_obj0) at line 400.

So bkg_obj[0] is literally the first eight bytes of the emalloc'd block, and those bytes get treated as a PyObject* and decref'd. Py_XDECREF only prevents NULL, which is why zeroing is a sufficient fix.

As for where the behavior is defined: the H5T_BKG_YES at line 474 is the h5py side, and the HDF5 side for the background argument of H5Tconvert is documented as holding the existing destination value. HDF5 neither allocates nor initializes it. Supplying meaningful contents is the caller's job. For a conversion whose destination is H5PY_OBJ, "existing destination value" means a PyObject* so the buffer has to hold either real object pointers or NULL. emalloc gives neither.

This is also why object references never tripped on this bug: conv_objref2pyref is registered with H5T_BKG_NO at line 462. Region references are the only reference converter registered with H5T_BKG_YES.