FEA Trees: Add support for missing values with criterion="absolute_error" by greatly simplifying the logic #32119
+269
−455
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Note: at this day, tests passes on my laptop and most CI unit tests pipelines are successful. But some are failing, I managed to reproduce one of the failing pipelines locally using a Docker image. I still need to find the bug though. |
|
Tests pass! 🎊 Well, I learned the difference between |
|
Let's keep this in draft mode until we merge #32100. Ping here for a review after the initial PRs are sorted out |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR refactors how missing values are handled in trees by:
CriterionsubclassesThis greatly simplifies the logic and unlocks for free the support of missing values for MAE trees.
Reference Issues/PRs
This PR accidentally fixes #32178
Otherwise, I looked but I didn't find any issue requesting this feature. I think it's because MAE trees are just too slow in sklearn for now so it's not much used... People wanting to use the MAE will just search for other options in sklearn or other libs.
What does this implement/fix? Explain your changes.
Currently, a part of the missing values support is done by each subclass of
Criterion. I believe it's not a great design because:Criterionis "X-blind", it's not aware of X values. It just looks atyandsample_weightsin the order defined by the sorted indices (sample_indices). It never looks at X values. But somehow, by making it handle missing values, it does have some dealing with X values. Why not just use the ordering ofsample_indicesto take into account missing values? Like what we do for any other value (even inf/-inf for instance).So, to the question "Why not just use the ordering of
sample_indicesto take into account missing values? " my answer is :"yes, let's just do that". The result is removing 200 lines from_criterion.pxywhile not increasing the complexity of the splitter and the partitionner (actually, it also simplifies a bit the splitter).Any other comments?
I think it might unlock making the support for missing values + monotonic constraints easy, but I haven't look into it yet.
It might also simplify a bit the support for missing values + sparse, but this is still not easy to do.