BUG: Silencing errors of filled_value's casting in array_finalize in MaskedArray #27596
Conversation
|
@fengluoqiuwu two things here:
|
The
I will add it later. |
|
Thanks! Looking a bit closer, I suspect this is right and we could probably just do it. We don't know that the dtype matches, and we have to just fall back to the default when the "inherited" fill-value fails (this is part of The reason I ask about where it comes from is that it may better to handle this in In this case, I actually like if we could audit/clean up things a bit more. That probably means pushing this logic into I think in most places probably we should fall back to (It is unfortunate, that this was always weird, but the integer casting is now floating up these issues...) |
I think using the default value is better. In the comment for |
This is the most minimal fix I could think off... If we have uints still use the out-of-bounds value, but make it uint. This is because the code uses copyto with the default same-kind casting in places... This doesn't remove all the other quirks, and surely, it is a behavior change in some awkward situations (since you got a uint, someone might mix the newly uint value with an integer and get float64 results, etc.) But... it is by far the most minimal fix I could think of after a longer time. A more thorough fix may be to just always store the exact dtype, but it would propagate differently e.g. when casting. A start for this is currently in numpygh-27596. Closes numpygh-27269, numpygh-27580
This is the most minimal fix I could think off... If we have uints still use the out-of-bounds value, but make it uint. This is because the code uses copyto with the default same-kind casting in places... This doesn't remove all the other quirks, and surely, it is a behavior change in some awkward situations (since you got a uint, someone might mix the newly uint value with an integer and get float64 results, etc.) But... it is by far the most minimal fix I could think of after a longer time. A more thorough fix may be to just always store the exact dtype, but it would propagate differently e.g. when casting. A start for this is currently in numpygh-27596. Closes numpygh-27269, numpygh-27580
This is the most minimal fix I could think off... If we have uints still use the out-of-bounds value, but make it uint. This is because the code uses copyto with the default same-kind casting in places... This doesn't remove all the other quirks, and surely, it is a behavior change in some awkward situations (since you got a uint, someone might mix the newly uint value with an integer and get float64 results, etc.) But... it is by far the most minimal fix I could think of after a longer time. A more thorough fix may be to just always store the exact dtype, but it would propagate differently e.g. when casting. A start for this is currently in numpygh-27596. Closes numpygh-27269, numpygh-27580
Silencing errors related to the casting of
fill_valuein__array_finalize__ofMaskedArray, and using a default value if the originalfill_valuecannot be cast even with the method_check_fill_value.While this approach may introduce some level of risk, it is reasonable to assume that if users set the
fill_valuethemselves, they are aware that it may lead to undefined behavior with specific type casting. It is unfortunate that the casting ofmasked_arraytypes is hindered simply because we assignedfill_valuewith the default value indirectly. This not only causes issues with methods that usedtype=directly, but also impedes any calls toufuncthat return a dtype different from the original dtype (since calls to theufuncwon't be applied to the storedfill_value), exacerbating the situation.I believe this is the most effective way to address these issues, rather than designing a new ufunc for
masked_arrayor marking whether the value was set by the user or not.fixes #27165