BUG: numpy.ma.min fails for uint dtypes #27580

jbkoch7 · 2024-10-16T19:44:31Z

Describe the issue:

The numpy.ma.min operation fails for cases where the input array has a dtype = uint and the resultant array and mask has ndim > 0. There could be other operations where this same case fails, e.g., numpy.ma.max.

Reproduce the code example:

import numpy as np
import numpy.ma as ma

_dtype = np.uint8
fill_value = _dtype( 0 )
a = np.array( [ [ 1, 127, 2 ],
                [ 2, 127, 1 ] ], dtype = _dtype )
a = ma.masked_array( a, a == 127 )
a.fill_value = fill_value
amin = a.min( axis = 0 )

Error message:

╭───────────────────────── Traceback (most recent call last) ──────────────────────────╮
│ test.py:10 in <module>                                                               │
│                                                                                      │
│    7 │   │   │   │   [ 2, 127, 1 ] ], dtype = _dtype )                               │
│    8 a = ma.masked_array( a, a == 127 )                                              │
│    9 a.fill_value = fill_value                                                       │
│ ❱ 10 amin = a.min( axis = 0 )                                                        │
│   11                                                                                 │
│                                                                                      │
│ .venv/lib/python3.12/site-packages/numpy/ma/core.py:5985                             │
│ in min                                                                               │
│                                                                                      │
│   5982 │   │   │   │   result.__setmask__(newmask)                                   │
│   5983 │   │   │   │   # Get rid of Infs                                             │
│   5984 │   │   │   │   if newmask.ndim:                                              │
│ ❱ 5985 │   │   │   │   │   np.copyto(result, result.fill_value, where=newmask)       │
│   5986 │   │   │   elif newmask:                                                     │
│   5987 │   │   │   │   result = masked                                               │
│   5988 │   │   │   return result                                                     │
╰──────────────────────────────────────────────────────────────────────────────────────╯
TypeError: Cannot cast scalar from dtype('int64') to dtype('uint8') according to the
rule 'same_kind'

Python and NumPy Versions:

2.1.2
3.12.5 (main, Oct 14 2024, 12:30:20) [GCC 13.2.0]

Runtime Environment:

No response

Context for the issue:

No response

The text was updated successfully, but these errors were encountered:

eendebakpt · 2024-10-16T21:28:17Z

Here is another example:

import numpy as np
import numpy.ma as ma
print(np.__version__)

a = np.array( [ [ 1, 127, 2 ],
                [ 2, 127, 1 ] ], dtype =  np.uint8 )

mask = (a == 127)
b = ma.masked_array( a, mask )
print(f'{b=}')

m = b.min() # works
print('## {m}')
b.min(axis=0) # fails

The output of print(f'{b=}') is

b=masked_array(data=[1, --, 2],
             mask=[False,  True, False],
       fill_value=np.int64(999999),
            dtype=uint8)

The masked array b as type uint8, but the fill value is an uint64. This creates issues in a.min. It is a bit strange that b.min() works, but b.min(axis=0) fails.

The value np.int64(999999) is retrieved from numpy.ma.core.default_fill_value, which has docstring

def default_fill_value(obj):
    """
    Return the default fill value for the argument object.

    The default filling value depends on the datatype of the input
    array or the type of the input scalar:

       ========  ========
       datatype  default
       ========  ========
       bool      True
       int       999999
       float     1.e20
       complex   1.e20+0j
       object    '?'
       string    'N/A'
       ========  ========

Maybe np.ma.core.default_fill_value(np.dtype(np.uint8)) should return 254, but I am not familiar enough with masked arrays to make that call.

fengluoqiuwu · 2024-10-17T05:50:09Z

There.

np.copyto(result, result.fill_value, where=newmask)

In document we can find that fill_value has int64 for integer type, but in the default casting mode , int64 could not be cast to uint8, that is why dtype int8 can run but uint8 failed. Maybe we should add uint64(999999) in default fill_value for usigned integer type or add type check and transform before copyto.

fengluoqiuwu · 2024-10-17T08:28:25Z

It is a bit strange that b.min() works, but b.min(axis=0) fails.

In b.min(), result.ndim equals to 0, and it skip copyto.

result = self.filled(fill_value).min(
                axis=axis, out=out, **kwargs).view(type(self))
            if result.ndim:
                # Set the mask
                result.__setmask__(newmask)
                # Get rid of Infs
                if newmask.ndim:
                    test = result.fill_value
                    np.copyto(result, result.fill_value, where=newmask)
            elif newmask:
                result = masked
            return result

fengluoqiuwu · 2024-10-17T11:29:28Z

I delete the infinity checking module.

This is the most minimal fix I could think off... If we have uints still use the out-of-bounds value, but make it uint. This is because the code uses copyto with the default same-kind casting in places... This doesn't remove all the other quirks, and surely, it is a behavior change in some awkward situations (since you got a uint, someone might mix the newly uint value with an integer and get float64 results, etc.) But... it is by far the most minimal fix I could think of after a longer time. A more thorough fix may be to just always store the exact dtype, but it would propagate differently e.g. when casting. A start for this is currently in numpygh-27596. Closes numpygh-27269, numpygh-27580

jbkoch7 added the 00 - Bug label Oct 16, 2024

tylerjereddy added the component: numpy.ma masked arrays label Oct 16, 2024

fengluoqiuwu linked a pull request Oct 17, 2024 that will close this issue

BUG: numpy.ma.min/max fails for uint and float16 dtypes #27584

Open

This was referenced Oct 18, 2024

BUG: filled MaskedArray becomes object dtype if given a fill_value of None #27269

Closed

DISCUSS: About issue of masked array #27588

Open

seberg mentioned this issue Nov 20, 2024

BUG: Ensure that same-kind casting works for uints (mostly) #27802

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: numpy.ma.min fails for uint dtypes #27580

BUG: numpy.ma.min fails for uint dtypes #27580

jbkoch7 commented Oct 16, 2024 •

edited

Loading

eendebakpt commented Oct 16, 2024 •

edited

Loading

fengluoqiuwu commented Oct 17, 2024

fengluoqiuwu commented Oct 17, 2024

fengluoqiuwu commented Oct 17, 2024 •

edited

Loading

BUG: numpy.ma.min fails for uint dtypes #27580

BUG: numpy.ma.min fails for uint dtypes #27580

Comments

jbkoch7 commented Oct 16, 2024 • edited Loading

Describe the issue:

Reproduce the code example:

Error message:

Python and NumPy Versions:

Runtime Environment:

Context for the issue:

eendebakpt commented Oct 16, 2024 • edited Loading

fengluoqiuwu commented Oct 17, 2024

fengluoqiuwu commented Oct 17, 2024

fengluoqiuwu commented Oct 17, 2024 • edited Loading

jbkoch7 commented Oct 16, 2024 •

edited

Loading

eendebakpt commented Oct 16, 2024 •

edited

Loading

fengluoqiuwu commented Oct 17, 2024 •

edited

Loading