Thanks to visit codestin.com
Credit goes to github.com

Skip to content

MNT use raise from in validation.py #17545

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 4, 2020

Conversation

cool-RR
Copy link
Contributor

@cool-RR cool-RR commented Jun 9, 2020

I recently went over Matplotlib, Pandas and NumPy, fixing a small mistake in the way that Python 3's exception chaining is used. If you're interested, I can do it here too. I've done it on just one file right now.

The mistake is this: In some parts of the code, an exception is being caught and replaced with a more user-friendly error. In these cases the syntax raise new_error from old_error needs to be used.

Python 3's exception chaining means it shows not only the traceback of the current exception, but that of the original exception (and possibly more.) This is regardless of raise from. The usage of raise from tells Python to put a more accurate message between the tracebacks. Instead of this:

During handling of the above exception, another exception occurred:

You'll get this:

The above exception was the direct cause of the following exception:

The first is inaccurate, because it signifies a bug in the exception-handling code itself, which is a separate situation than wrapping an exception.

Let me know what you think!

@cool-RR cool-RR marked this pull request as ready for review June 9, 2020 14:25
@glemaitre
Copy link
Member

This is most certainly valuable. We should check where the pattern can be found in our codebase. I assume that we are using the from ... since a while but we should have historical code that does not follow it.

@jnothman I always relied on you on these raising issues. Do you have any opinion? Maybe @thomasjpfan or @NicolasHug?

@NicolasHug
Copy link
Member

I've been using raise from e but inconsistently (I think). Happy to have them

@cool-RR
Copy link
Contributor Author

cool-RR commented Jun 15, 2020

Awesome. After this PR is merged, I'll create more PR(s) with the other cases. Here's a list of possible cases. Note that it may contain false positives.

benchmarks/bench_tsne_mnist.py: 110
benchmarks/bench_tsne_mnist.py: 135
doc/conftest.py: 50
doc/conftest.py: 57
doc/conftest.py: 64
doc/tutorial/machine_learning_map/parse_path.py: 58
doc/tutorial/machine_learning_map/pyparsing.py: 1358
doc/tutorial/machine_learning_map/pyparsing.py: 1376
doc/tutorial/machine_learning_map/pyparsing.py: 1418
doc/tutorial/machine_learning_map/pyparsing.py: 1627
doc/tutorial/machine_learning_map/pyparsing.py: 1700
doc/tutorial/machine_learning_map/pyparsing.py: 1743
doc/tutorial/machine_learning_map/pyparsing.py: 1770
doc/tutorial/machine_learning_map/pyparsing.py: 2164
doc/tutorial/machine_learning_map/pyparsing.py: 3386
doc/tutorial/machine_learning_map/pyparsing.py: 3388
doc/tutorial/machine_learning_map/pyparsing.py: 5584
doc/tutorial/machine_learning_map/pyparsing.py: 5606
sklearn/_build_utils/__init__.py: 35
sklearn/cluster/_agglomerative.py: 434
sklearn/cluster/_bicluster.py: 470
sklearn/compose/_column_transformer.py: 470
sklearn/compose/_column_transformer.py: 633
sklearn/covariance/_graph_lasso.py: 271
sklearn/datasets/_openml.py: 166
sklearn/datasets/_samples_generator.py: 653
sklearn/datasets/_samples_generator.py: 718
sklearn/datasets/_samples_generator.py: 849
sklearn/datasets/_samples_generator.py: 852
sklearn/datasets/tests/test_lfw.py: 76
sklearn/ensemble/_gb.py: 451
sklearn/ensemble/_gb.py: 1211
sklearn/ensemble/_gb.py: 1262
sklearn/ensemble/_weight_boosting.py: 259
sklearn/externals/_arff.py: 251
sklearn/externals/_arff.py: 288
sklearn/externals/_arff.py: 289
sklearn/externals/_arff.py: 435
sklearn/externals/_arff.py: 483
sklearn/externals/_arff.py: 542
sklearn/externals/_arff.py: 546
sklearn/externals/_arff.py: 602
sklearn/externals/_arff.py: 606
sklearn/externals/_arff.py: 758
sklearn/externals/_arff.py: 897
sklearn/externals/_lobpcg.py: 388
sklearn/externals/_lobpcg.py: 614
sklearn/impute/_base.py: 259
sklearn/inspection/_partial_dependence.py: 168
sklearn/inspection/_plot/partial_dependence.py: 273
sklearn/inspection/_plot/partial_dependence.py: 284
sklearn/linear_model/_stochastic_gradient.py: 167
sklearn/linear_model/_stochastic_gradient.py: 173
sklearn/linear_model/_stochastic_gradient.py: 181
sklearn/manifold/_locally_linear.py: 167
sklearn/manifold/_spectral_embedding.py: 219
sklearn/metrics/_classification.py: 2359
sklearn/metrics/_scorer.py: 312
sklearn/metrics/_scorer.py: 350
sklearn/mixture/_gaussian_mixture.py: 319
sklearn/mixture/_gaussian_mixture.py: 328
sklearn/model_selection/_validation.py: 554
sklearn/preprocessing/_encoders.py: 345
sklearn/svm/_base.py: 515
sklearn/utils/__init__.py: 98
sklearn/utils/__init__.py: 240
sklearn/utils/__init__.py: 269
sklearn/utils/__init__.py: 378
sklearn/utils/_encode.py: 41
sklearn/utils/_encode.py: 86
sklearn/utils/_testing.py: 357
sklearn/utils/_testing.py: 741
sklearn/utils/_testing.py: 746
sklearn/utils/estimator_checks.py: 157
sklearn/utils/estimator_checks.py: 742
sklearn/utils/estimator_checks.py: 775
sklearn/utils/estimator_checks.py: 779
sklearn/utils/estimator_checks.py: 1160
sklearn/utils/estimator_checks.py: 1194
sklearn/utils/estimator_checks.py: 1478
sklearn/utils/estimator_checks.py: 1482
sklearn/utils/estimator_checks.py: 1496
sklearn/utils/estimator_checks.py: 1511
sklearn/utils/estimator_checks.py: 1776
sklearn/utils/estimator_checks.py: 1782
sklearn/utils/estimator_checks.py: 1788
sklearn/utils/estimator_checks.py: 2564
sklearn/utils/tests/test_validation.py: 806
sklearn/utils/validation.py: 207
sklearn/utils/validation.py: 602

@thomasjpfan
Copy link
Member

I would prefer "raise from". I think we haven't used them everywhere because it is a Python 3 feature (and we use to support Python 2).

@cool-RR
Copy link
Contributor Author

cool-RR commented Jun 21, 2020

@thomasjpfan Can we move this forward?

except TypeError:
raise TypeError(message)
except TypeError as type_error:
raise TypeError(message) from type_error
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Surely this should just be bare raise?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we still want the message to be passed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm with @NicolasHug here, the message is valuable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jnothman Nudge

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the argument for keeping the bare raise here, is that the additional message may case too much noise?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have the error message in master, I see no reason to remove it now.

I'll merge the PR, @jnothman please feel free to raise any concern if any

Copy link
Member

@thomasjpfan thomasjpfan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am happy with this change.

except TypeError:
raise TypeError(message)
except TypeError as type_error:
raise TypeError(message) from type_error
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the argument for keeping the bare raise here, is that the additional message may case too much noise?

@NicolasHug NicolasHug changed the title Fix exception causes in validation.py MNT use raise from in validation.py Jul 4, 2020
@NicolasHug NicolasHug merged commit 52cd110 into scikit-learn:master Jul 4, 2020
@NicolasHug
Copy link
Member

Thanks @cool-RR

@jnothman
Copy link
Member

jnothman commented Jul 5, 2020 via email

glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Jul 17, 2020
jayzed82 pushed a commit to jayzed82/scikit-learn that referenced this pull request Oct 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants