FIX HGBT exclude zero-weight samples in tree grower - #27
Draft
ogrisel wants to merge 16 commits into
Draft
Conversation
ogrisel
force-pushed
the
cursor/hgb-zero-weight-fix-b6c7
branch
from
September 4, 2026 11:13
d636a81 to
b6073d1
Compare
cursor Bot
pushed a commit
that referenced
this pull request
Sep 4, 2026
Towncrier requires <PR_NUMBER>.<type>.rst; this branch's PR is #27. Co-authored-by: Olivier Grisel <[email protected]>
ogrisel
force-pushed
the
cursor/hgb-zero-weight-fix-b6c7
branch
from
September 4, 2026 16:12
c9c64ab to
6c7b102
Compare
Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Olivier Grisel <[email protected]>
Keep a tiny absolute gain floor on the existing per-bin comparison and restrict float32-aware tie-breaking to the O(n_features) feature loop. Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Antoine Baker <[email protected]>
Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Christian Lorentzen <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reference Issues/PRs
Addresses the zero-weight / deep-tree failure discussed on scikit-learn#34838 and scikit-learn#34745.
What does this implement/fix?
Root cause: HGBT split constraints use unweighted histogram counts (
hist.count). Zero-weight rows still entered the grower partition, so they inflatedmin_samples_leaf/ split-stopping even though their gradients and hessians were already 0. Withmin_samples_leaf=1, deeper trees could keep splitting nodes that would already be leaves if weight-0 rows were absent.Fix: Pass
sample_weightintoTreeGrower/Splitterand initializepartitionwith positively weighted sample indices only (same approach asDecisionTree*). Zero-weight samples never enter a leaf, so they cannot form a zero-weight leaf and no longer affect unweighted count constraints.Removed the previous
fit-time pre-filter / integer-weight expansion helper.Verification:
test_sample_weight_zero_equivalence_deep_treesand grower unit test for partition exclusionmin_samples_leaf=1, max_depth=1(deeper trees can still diverge for positive integer weights due to float32weight * gradvs summing unweighted gradients)Still limited:
min_samples_leaf > 1with non-unit positive weights can still diverge from sample repetition until leaf-size constraints use weight sums.AI usage disclosure
This pull request includes code written with the assistance of AI.
The code has not yet been reviewed by a human.