Fix FEATURE_THRESHOLD initialization in trees #32259
Conversation
eb15f6b to
b3efc12
Compare
b3efc12 to
76e630e
Compare
76e630e to
2a3f7ec
Compare
| # Mitigate precision differences between 32 bit and 64 bit | ||
| FEATURE_THRESHOLD = 1e-7 |
There was a problem hiding this comment.
I've been working on sklearn/tree/* quite a lot lately, but this comment has remained a mystery to me. It seems you rely on this behavior, so maybe you can detail a bit more what's the purpose of "mitigating precision differences between 32 bit and 64 bit"?
(100% optional though)
There was a problem hiding this comment.
Thanks @cakedev0 , I am also unsure of the purpose of this threshold. Actually, the test that failed on our side was based on randomly generated fake data. I don't believe we have features with such low min/max values. So, I think we also don't rely on this behavior.
sklearn/tree/tests/test_tree.py
Outdated
|
|
||
| def test_almost_constant_feature(): | ||
| random_state = check_random_state(0) | ||
| X = random_state.rand(10, 20) |
There was a problem hiding this comment.
| X = random_state.rand(10, 20) | |
| X = random_state.rand(10, 2) |
I think you just need 2 features for this test to work. It would make it clearer IMO.
There was a problem hiding this comment.
Makes sense, let me push a commit for this . Also I will add an assertion that the other feature has an importance higher than 0.
|
Side note on force pushing: I like doing it as well but it seems to mess with links from notifications. Which means people get a notification, click on the link in it and then end up "in the middle of nowhere". So we recommend that people don't force push. The PR gets merged via squashing, so an "ugly" history doesn't matter so much. |
@betatim understood. Sorry for the noise! Will keep in mind for future contributions. |
Reference Issues/PRs
What does this implement/fix? Explain your changes.
I noticed one of our tests failing after upgrading from 1.5 to 1.6 and above. I traced the issue to the tree implementation change in #29458. The initialization of
cdefconstant cannot be made in the pxd file. This resulted inFEATURE_THRESHOLDto be initialized to0.0instead of1e-7. This PR fixes that by moving the initialization to thepyxfile.Any other comments?
It's my first time contributing to scikit-learn, so please let me know if anything is missing.