Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BUG: Error when using nanmax() and nanmin() on an object array with an axis specified. #9008

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
WarrenWeckesser opened this issue Apr 27, 2017 · 12 comments

Comments

@WarrenWeckesser
Copy link
Member

This was the underlying cause of a problem reported on stackoverflow: http://stackoverflow.com/questions/43659827/numpy-error-when-specifying-axis-in-nanmax-while-nansum-works-an-the-same-case

The error is raised when applying nanmax() or nanmin() to an array with object data type and specifying an axis.

Here's the example from my answer:

In [2]: import numpy as np

In [3]: np.__version__
Out[3]: '1.13.0.dev0+bca7922'

In [4]: a = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=object)

In [5]: np.nanmax(a, axis=0)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-5-a020f98a2536> in <module>()
----> 1 np.nanmax(a, axis=0)

/Users/warren/miniconda3numpy/lib/python3.5/site-packages/numpy-1.13.0.dev0+bca7922-py3.5-macosx-10.6-x86_64.egg/numpy/lib/nanfunctions.py in nanmax(a, axis, out, keepdims)
    343         # Fast, but not safe for subclasses of ndarray
    344         res = np.fmax.reduce(a, axis=axis, out=out, **kwargs)
--> 345         if np.isnan(res).any():
    346             warnings.warn("All-NaN slice encountered", RuntimeWarning, stacklevel=2)
    347     else:

TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

There is no error when the axis is not given:

In [6]: np.nanmax(a)
Out[6]: 4.0

nansum() handles the axis without an error:

In [7]: np.nansum(a, axis=0)
Out[7]: array([4.0, 6.0], dtype=object)

There are several other issues involving the nan-functions and object arrays, but I couldn't tell if this issue is a duplicate. Sorry for the noise if it is.

@Satyadev592
Copy link

Satyadev592 commented Apr 27, 2017

http://stackoverflow.com/questions/43659827/numpy-error-when-specifying-axis-in-nanmax-while-nansum-works-an-the-same-case/43660147#43660147

There is an answer with three downvotes that explains why this is not a bug. Happy realisation :)

Nansum and Nanmax are not coded in the same fashion , a detailed explanation is available on the post. It's not an axis issue to begin with...

@eric-wieser
Copy link
Member

eric-wieser commented Apr 27, 2017

So, ideally this breaks down into:

As a quick workaround, we should just not allow object arrays to take that special case branch

@Satyadev592
Copy link

Satyadev592 commented Apr 27, 2017

Yes, explicity type cast to float.
Nanmax does in the beginning of it's source code , the following :
res = np.fmax.reduce(a, axis=axis, out=out, **kwargs)
if np.isnan(res).any():

The np.isnan() fails on object type columns.

@eric-wieser
Copy link
Member

eric-wieser commented Apr 27, 2017

@Satyadev592: Not the right fix. np.fmax.reduce is broken on object columns too.

This is essentially a different manifestation of the problem at #8974, and the quick fix to numpy would be as described there. This is absolutely a bug in numpy.

@eric-wieser
Copy link
Member

eric-wieser commented Apr 27, 2017

@Satyadev592:

Yes, fmax is broken on object columns as well. But the assumption that axis is the reason for this is wrong. It's purely the data type of the column. The code works with row wise axis just because it coerces the data type to float (purely because of the dataset here)

The code example at the top of this page directly contradicts that claim, does it not?

The problem manifests itself only when all of the following are true: axis is specified, dtype is object, and input is 2d or higher

@Satyadev592
Copy link

Satyadev592 commented Apr 27, 2017

Take a numpy array with only string values and try the nanmax on either axis. It still pops an error. The error this time is something to do with the fmax, dig a bit deeper , it's got something to do with numpy not having good support for the object datatype. If this is a bug , then a bug for each numpy function should be created for lack of support for object datatypes. This particular problem makes you think axis is a reason while it has nothing to do with the problem.

Also shifting an explicit casting to float inside source code is probably not a good idea. Numpy does not seem to be designed for object types [Well established i think]

Also , the documentation for np.nanmax does not seem to indicate support for object data type (general norm from whatever i have gathered about numpy) :

@eric-wieser
Copy link
Member

Take a numpy array with only string values

Strings are an entirely different beast, and have little to do with the object dtype. The problem there is that flexible types (np.bytes_, np.void, np.unicode_) are not supported by the ufunc system, due to having variable sizes. object arrays are perfectly supported by ufuncs - it just so happens that we don't have an implementation of isnan for them.

If this is a bug , then a bug for each numpy function should be created for lack of support for object datatypes.

Absolutely. If you can find numpy functions that fail with object arrays, where it would make sense for them not to, then either:

  • Their documentation should be updated to warn about this
  • They should be fixed so that they do.

@ahaldane
Copy link
Member

This issue would be fixed by #6320 (I just tested the example).

[1]: a = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=object)
[2]: np.nanmax(a, axis=0)
array([3.0, 4.0], dtype=object)
[3]: x = np.array([1, np.nan])
[4]: np.fmin.reduce(x)
1.0
[5]: np.fmin.reduce(x.astype(object))
1.0

eric-wieser added a commit to eric-wieser/numpy that referenced this issue Apr 29, 2017
mherkazandjian pushed a commit to mherkazandjian/numpy that referenced this issue May 30, 2017
@mattip
Copy link
Member

mattip commented Mar 17, 2020

Closing, all the examples work correctly (including the last one np.fmin.reduce(x.astype(object)) which returns nan). Please reopen if I misunderstood.

@mattip mattip closed this as completed Mar 17, 2020
@eric-wieser
Copy link
Member

The bug here is that the case you mention should not return nan

@eric-wieser
Copy link
Member

However that's covered in gh-8975, so fine to close this

@eric-wieser
Copy link
Member

Closed by gh-9013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants