Thanks to visit codestin.com
Credit goes to github.com

Skip to content

FIX Raise error for categorical features with criterion='absolute_error' - #34594

Merged
lesteve merged 10 commits into
scikit-learn:mainfrom
cakedev0:fix/cat_split_not_best_for_ae
Jul 30, 2026
Merged

FIX Raise error for categorical features with criterion='absolute_error'#34594
lesteve merged 10 commits into
scikit-learn:mainfrom
cakedev0:fix/cat_split_not_best_for_ae

Conversation

@cakedev0

@cakedev0 cakedev0 commented Jul 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Reference Issues/PRs

Fixes #34578

What does this implement/fix? Explain your changes.

The categorical split-finding algorithm is not valid for criterion="absolute_error": it can produce splits that aren't actually optimal for MAE, which is why test_split_impurity was flaky on CI for the dense-categorical/DecisionTreeRegressor/absolute_error combination on certain global_random_seed values (see #34578).

This implements option 1 from #34578 (comment): raise a clear ValueError for this case.

AI usage

100%, then reviewed

Any other comments?

Once this is merged, I think it would be good to open the debate around allowing near-optimal splits in trees:

  • From first quick experiments, current mean-based categories ordering already achieves near optimal splits for AE (within 1% of the optimal AE). You can of course design adversarial cases where it's much worse, but not sure that really happens in real data.
  • With @adam2392 we started discussing the idea of sampling-based search for large number of categories (and multi-class/multi-output too I guess?), for the best splitter.
  • Over the past year, I solve a good number of bugs that were leading to sub-optimal splits. Those bug didn't prevent RF to be an overall good
  • If we want to be strict about "exact best split", then we need to remove/fix monotonic constraints support (fix is hard), see BUG: Decision trees: tree.tree_.impurity reports incorrect impurity for monotonic trees #34078 (comment)

cc @lorentzenchr

cakedev0 and others added 2 commits July 29, 2026 11:43
@cakedev0 @claude
FIX Raise error for categorical features with criterion='absolute_error'
40024be 
The categorical split-finding algorithm is not valid for the absolute
error criterion, which made test_split_impurity flaky on CI (scikit-learngh-34578).
Rather than silently return a possibly suboptimal split, raise a clear
error, matching the other categorical-features incompatibility checks.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
@cakedev0 @claude
Rename changelog fragment to match PR number
22e55a7 
Co-Authored-By: Claude Sonnet 5 <[email protected]>
adam2392
adam2392 approved these changes Jul 29, 2026
View reviewed changes

@adam2392 adam2392 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

StefanieSenger
StefanieSenger reviewed Jul 29, 2026
View reviewed changes

@StefanieSenger StefanieSenger left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks almost good to me. Thank you @cakedev0!

I have nit comment regarding documentation.
Another very tiny nit maybe: The error message reads a bit redundant.

Comment thread sklearn/tree/tests/test_split.py Outdated
Comment thread sklearn/tree/tests/test_tree.py Outdated
Comment thread sklearn/tree/_classes.py
@lorentzenchr

lorentzenchr commented Jul 29, 2026

Copy link
Copy Markdown
Member

This PR also lgtm. One question: Do we need the new changelog entry? I would prefer to add that bit of information to the entry of #33354.

@StefanieSenger

StefanieSenger commented Jul 29, 2026

Copy link
Copy Markdown
Member

Do we need the new changelog entry? I would prefer to add that bit of information to the entry of #33354.

This is a good point. 33354 has not been released yet, so an extra changelog entry for this fix would only be confusing.

@cakedev0 @StefanieSenger
Clearer skip msg
e6ef1b3 
Co-authored-by: Stefanie Senger <[email protected]>
@github-actions github-actions Bot added the CI:Linter failure The linter CI is failing on this PR label Jul 29, 2026
@github-actions github-actions Bot removed the CI:Linter failure The linter CI is failing on this PR label Jul 29, 2026
StefanieSenger
StefanieSenger approved these changes Jul 29, 2026
View reviewed changes

@StefanieSenger StefanieSenger left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now looks good to me. Thanks again. :)

@cakedev0

cakedev0 commented Jul 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Thanks for the quick reviews!

It's good to merge on my side

@StefanieSenger
StefanieSenger enabled auto-merge (squash) July 29, 2026 15:00
auto-merge was automatically disabled July 29, 2026 19:20

Head branch was pushed to by a user without write access

@lesteve
lesteve merged commit 63f2abf into scikit-learn:main Jul 30, 2026
38 checks passed
@lesteve lesteve mentioned this pull request Jul 31, 2026
Closed
prady0t pushed a commit to prady0t/scikit-learn that referenced this pull request Sep 2, 2026
@cakedev0 @StefanieSenger @prady0t
FIX Raise error for categorical features with criterion='absolute_err…
2f3fbb3 
…or' (scikit-learn#34594)

Co-authored-by: Stefanie Senger <[email protected]>
@jeremiedbb jeremiedbb mentioned this pull request Sep 8, 2026
Merged
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@adam2392 adam2392 adam2392 approved these changes
@StefanieSenger StefanieSenger StefanieSenger approved these changes

Assignees

No one assigned

Labels

module:tree No Changelog Needed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

test_split_impurity or the code it tests seem to be unstable

5 participants

@cakedev0 @lorentzenchr @StefanieSenger @adam2392 @lesteve