[MRG] Raise exception on providing complex data to estimators #9551

pravarmahajan · 2017-08-14T18:20:25Z

Reference Issue

What does this implement/fix? Explain your changes.

An exception should be raised on giving complex data as input to the estimators. The changed code
does the same, via the function check_array. A new test function has been added to test_validation.py
corresponding to the code changes.

Any other comments?

- Fixes rendering of docstring examples - Instead of importing cross_val_score in example, cross_validate is imported

…into complex_data merging changes from the master branch

…into complex_data Merging with upstream

amueller · 2017-08-14T18:44:13Z

sklearn/utils/validation.py

    if not hasattr(dtype_orig, 'kind'):
        # not a data type (e.g. a column named dtype in a pandas DataFrame)
        dtype_orig = None

+    if dtype_orig is not None and dtype_orig.kind == "c":
+        raise ValueError("Complex data is not supported\n{}\n".format(array))
+    elif isinstance(array, list) or isinstance(array, tuple):


I'm not sure this is the best way. We might also not catch everything with it (how about dataframes that contain complex data?). I'm not sure this is easy to catch later if dtype is not None, though. With your addition we're converting into an array twice: here and in 411. It would be great if we could use the conversion at 411, but as I said, it's not entirely clear to me how to do that.

amueller · 2017-08-14T19:02:32Z

I just realized that we are not doing anything with complex data, as we pass anything that's not object through as numeric.
I would have expected we convert it to float64, which would give a

ComplexWarning: Casting complex values to real discards the imaginary part
warning.

Looking into the numeric part, things don't look great overall:

check_array([['a', 'b']], dtype="numeric")
passes the check?!

I guess that's in accordance with the documentation, though.
I think my preferred solution would be to only catch complex data when dtype is not None, and catch the ComplexWarning in the cast and raise an error.

If dtype was numeric, we should check after the array creation, as we do for "O" data in 425.

Can you please also add a test to check_estimators.py?

pravarmahajan · 2017-08-14T19:16:38Z

Thank you @amueller . Let me make changes as you have suggested.

pravarmahajan · 2017-08-15T21:05:48Z

@amueller
I am working on the changes as you have suggested, a couple of questions as I was trying to understand the code:
(1) In case the passed argument dtype is a list or a tuple, we are either setting dtype to None or taking the first element of the list. Why do we even allow dtype to be list when we are discarding everything other than the first element?
(2) In line 425-426, array is casted as type np.float64 in case dtype of the array is 'O'. I believe this case is already taken care of by lines 382-387 and then 411. (all line numbers are according to the pull request I created above). Therefore, is line 425 needed?

pravarmahajan · 2017-08-15T21:35:46Z

Can you please also add a test to check_estimators.py?

Did you mean test_estimator_checks.py in sklearn/utils/tests?

…into complex_data

jnothman · 2017-08-15T23:04:50Z

We have common tests for all estimators, implemented in check_estimators.py. That those tests work correctly is tested in test_estimator_checks.py

…

On 16 August 2017 at 07:35, pravarmahajan ***@***.***> wrote: Can you please also add a test to check_estimators.py? Did you mean test_estimator_checks.py in sklearn/utils/tests? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#9551 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEz6_3kdz0XqyPjQE8mgFTSj7OZDcTHks5sYg81gaJpZM4O2t8W> .

jnothman · 2017-08-15T23:06:56Z

Why do we even allow dtype to be list when we are discarding everything

other than the first element? You missed the part where we *accepted* the data as is if its dtype is in that list. Otherwise we *try* to convert the data with the first dtype.

pravarmahajan · 2017-08-16T16:44:03Z

@jnothman
Got it, thank you!

amueller · 2017-08-16T19:27:31Z

Answering 2):
Lines 382-387 only work if an array was passed. If a list was passed, we can't know what the dtype is until after we converted to an array.

…into complex_data

pravarmahajan · 2017-08-17T18:51:19Z

Thanks @amueller
I have made changes as you suggested and added a test to estimator_checks.py

amueller · 2017-08-17T20:25:59Z

sklearn/utils/tests/test_validation.py

@@ -433,6 +433,44 @@ def test_check_array_min_samples_and_features_messages():
    assert_array_equal(y, y_checked)


+def test_check_array_complex_data_error():
+    X = np.array([[1 + 2j, 3 + 4j, 5 + 7j], [2 + 3j, 4 + 5j, 6 + 7j]])


maybe instead of repeating the content every time create a list of lists in the beginning and then create everything else from that, so it's more obvious what is tested? Don't have a strong opinion though.

I had thought of that, but it becomes very messy for some of the cases. For example converting list of list to tuple of tuple, or to tuple of np-arrays. Maybe I can add one line comment for what case is being tested.

amueller · 2017-08-17T20:28:24Z

sklearn/utils/validation.py

+                warnings.simplefilter('error', ComplexWarning)
+                array = np.array(array, dtype=dtype, order=order, copy=copy)
+            except ComplexWarning:
+                raise ValueError("Complex data not supported\n"


I see that this branch is covered, but I don't see where. We're never passing dtype right?

Never passing dtype to check_array? Even if we don't pass it to check_array, the value of dtype changes within this function. For example lines 381-386. The ComplexWarning comes only if we are setting dtype to some one of the real types. However, if dtype is "None", no conversion takes place and no warning is produced, so we need to check that case again in subsequentl lines

One example where dtype is explicitly passed to the function is that of svm decision function

amueller · 2017-08-17T20:29:24Z

LGTM, I think unfortunately this is the easiest we can get away with. I'm a bit confused about when we hit the ComplexWarning branch, though.

…into complex_data

ogrisel · 2017-09-01T09:16:56Z

sklearn/utils/validation.py

+                array = np.array(array, dtype=dtype, order=order, copy=copy)
+            except ComplexWarning:
+                raise ValueError("Complex data not supported\n"
+                                 "{}\n".format(array))


Unfortunately, at this point the array has been converted to float or int and the error message will be confusing. We need to keep a reference to the original input with complex values to report it in this error message (and del original_array otherwise to let the gabage collector free the memory as soon as possible otherwise).

Or maybe:

with warnings.catch_warnings(): try: warnings.simplefilter('error', ComplexWarning) new_array = np.array(array, dtype=dtype, order=order, copy=copy) except ComplexWarning: raise ValueError("Complex data not supported\n" "{}\n".format(array)) array = new_array del new_array

Hum sorry my previous comments are wrong. Your code is fine.

ogrisel

+1 once my comment is addressed

…-learn#9551) * Modifies model_selection.cross_validate docstring (scikit-learn#9534) - Fixes rendering of docstring examples - Instead of importing cross_val_score in example, cross_validate is imported * raise error on complex data input to estimators * Raise exception on providing complex data to estimators * adding checks to check_estimator for complex data * removing some unnecessary parts * autopep8 changes * removing ipdb, restoring some autopep8 fixes * removing ipdb, restoring some autopep8 fixes * adding documentation for complex data handling * adding one line explanation for each test case

lesteve · 2018-02-23T14:51:35Z

Stupid question, why did we not use array.dtype.kind == 'c' to detect complex arrays? This seems simpler than catching a warning ...

pravarmahajan · 2018-02-25T04:41:39Z

@lesteve
Are you talking about this part?

with warnings.catch_warnings():
            try:
                warnings.simplefilter('error', ComplexWarning)
                new_array = np.array(array, dtype=dtype, order=order, copy=copy)
            except ComplexWarning:
                raise ValueError("Complex data not supported\n"
                                 "{}\n".format(array))
            array = new_array
            del new_array

The array here is not necessarily an ndarray object, it could be a list of list too. In the latter case array.dtype.kind==c won't work. So it's easier to try convert the array to ndarray, and check for any complex warnings raised and then throw that warning as exception.

lesteve · 2018-02-26T14:04:23Z

OK fair enough ... I feel like this could be simplified but it will require a bit more thoughts.

jrbourbeau and others added 4 commits August 13, 2017 15:26

Modifies model_selection.cross_validate docstring (scikit-learn#9534)

9a9cbfe

- Fixes rendering of docstring examples - Instead of importing cross_val_score in example, cross_validate is imported

Merge branch 'master' of https://github.com/scikit-learn/scikit-learn …

709437e

…into complex_data merging changes from the master branch

Merge branch 'master' of https://github.com/scikit-learn/scikit-learn …

09c5732

…into complex_data Merging with upstream

raise error on complex data input to estimators

8d55a60

amueller reviewed Aug 14, 2017

View reviewed changes

pravarmahajan added 2 commits August 15, 2017 19:00

Merge branch 'master' of https://github.com/scikit-learn/scikit-learn …

63ef59f

…into complex_data

Raise exception on providing complex data to estimators

cd863c8

pravarmahajan closed this Aug 16, 2017

pravarmahajan reopened this Aug 16, 2017

pravarmahajan added 8 commits August 16, 2017 18:28

Merge branch 'master' of https://github.com/scikit-learn/scikit-learn …

939d781

…into complex_data

adding checks to check_estimator for complex data

b84e1ee

removing some unnecessary parts

7d977f2

autopep8 changes

e4b437f

Merge branch 'master' of https://github.com/scikit-learn/scikit-learn …

1a3abb6

…into complex_data

removing ipdb, restoring some autopep8 fixes

b26e99d

removing ipdb, restoring some autopep8 fixes

694b3b3

adding documentation for complex data handling

843230f

amueller reviewed Aug 17, 2017

View reviewed changes

Merge branch 'master' of https://github.com/scikit-learn/scikit-learn …

c065851

…into complex_data

adding one line explanation for each test case

51d7f47

ogrisel reviewed Sep 1, 2017

View reviewed changes

ogrisel approved these changes Sep 1, 2017

View reviewed changes

ogrisel merged commit ecc96be into scikit-learn:master Sep 1, 2017

Uh oh!

[MRG] Raise exception on providing complex data to estimators #9551

[MRG] Raise exception on providing complex data to estimators #9551

Uh oh!

Conversation

pravarmahajan commented Aug 14, 2017

Reference Issue

What does this implement/fix? Explain your changes.

Any other comments?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amueller commented Aug 14, 2017

Uh oh!

pravarmahajan commented Aug 14, 2017

Uh oh!

pravarmahajan commented Aug 15, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pravarmahajan commented Aug 15, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jnothman commented Aug 15, 2017 via email

Uh oh!

jnothman commented Aug 15, 2017 via email

Uh oh!

pravarmahajan commented Aug 16, 2017

Uh oh!

amueller commented Aug 16, 2017

Uh oh!

pravarmahajan commented Aug 17, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pravarmahajan Aug 17, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amueller commented Aug 17, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ogrisel left a comment

Choose a reason for hiding this comment

Uh oh!

lesteve commented Feb 23, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pravarmahajan commented Feb 25, 2018

Uh oh!

lesteve commented Feb 26, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

pravarmahajan commented Aug 15, 2017 •

edited

Loading

pravarmahajan commented Aug 15, 2017 •

edited

Loading

pravarmahajan Aug 17, 2017 •

edited

Loading

lesteve commented Feb 23, 2018 •

edited

Loading

lesteve commented Feb 26, 2018 •

edited

Loading