Yeah, I explicitly mentioned this in the 1.13 and 1.14 release notes. Actually, since we have not provided a pack_fields method it is difficult for downstream users to update their code to get the old behavior: I originally thought the user could just replace arr[fields'] by arr[fields].copy(), but that does not remove padding bytes either.

Should slicing a dtype shrink the itemsize?

ie, if a dtype has fields a, b, c, d, and you slice with ['b', 'd'], should the resulting view dtype still span the entire original, or just the fields selected? Doing the latter would solve the problem for contiguous views within dtypes (which are probably most common)

A consequence of this would be that the resulting array would be non-"contiguous"

You mean like arr[['b', 'c']].view('i8') if all the fields are originally i4. True, that could potentially be allowed.

On the other hand sometimes it's better to fail more rather than less often. If people get errors about viewing multifield slices more easily, maybe that's better than hiding it in special cases. Then people are forced to write code that is safe in all situations, for example using repack_fields.

Even the less objectionable case where the dtypes match

dt = np.dtype([
    ('a', np.int16),
    ('b', np.int16),
    ('c', np.int16),
])
arr = np.empty(10, dt)

# both of these no longer work, but could have well-defined behavior
arr[['a']].view(np.int16)
arr[['a', 'b']].view((np.int16, 2))

Essentially, I'm proposing a trim_fields behavior, where one offset is always zero, and the itemsize is never beyond the element occupying the last byte

Right, but it still bothers me that arr[['a', 'c']].view((np.int16, 2)) would still fail, which would be unexpected so someone who found that [['a', 'b']] worked. It might be better to force the user to write repack_fields(arr[['a', 'c']]).view((np.int16, 2)) in all 3 cases.

Also, I'm pretty sure that even with this trim_fields behavior the view often isn't allowed because the resulting array is not C-contiguous. Compare to:

>>> a = np.ones((10,3), dtype='i4')
>>> a[:,0:2].view('i8')
ValueError: To change to a dtype of a different size, the array must be C-contiguous
>>> a[:,0:2].view(('i4', 2))
ValueError: Changing the dtype to a subarray type is only supported if the total itemsize is unchanged

@ahaldane Is this now fixed for 1.15?

Yes, I just checked master. Master now gives the 1.13 behavior, just with an updated futurewarning.

I'm not sure we should close, since this issue will pop up again in 1.16.

Good, I'll just update the milestone :).

I ran across code in mapping.c that is marked for removal in 1.16, but could not find a pull request that is waiting for a merge. It seems there is work to be done here?

Yes, that's on my TODO list. See discussion in #10411, essentially we need to fix #8100 first.

I should have some time to work on it starting in two weeks, but feel free to take a shot at it before. There is already some work on it in #10931 but as I comment there a more extensive fix is needed.

We have returned to the 1.13 behavior now. PR #10411 was merged, issue #8100 is closed. Can we close this or do we want to explore returning views again?

Hmm in my opinion #8100 was mistakenly closed, since I accidentally used the word "fix" in a sentence in #10411's description.

I am also not sure we've made a decision about structured views. As far as I am aware (I was out for a bit so may have missed it) I think we are still considering turning it into a view. #10411 just pushed off the decision/implementation.