Thanks to visit codestin.com
Credit goes to github.com

Skip to content

FIX HGBT exclude zero-weight samples in tree grower - #27

Draft
ogrisel wants to merge 16 commits into
mainfrom
cursor/hgb-zero-weight-fix-b6c7
Draft

FIX HGBT exclude zero-weight samples in tree grower#27
ogrisel wants to merge 16 commits into
mainfrom
cursor/hgb-zero-weight-fix-b6c7

Conversation

@ogrisel

@ogrisel ogrisel commented Sep 4, 2026

Copy link
Copy Markdown
Owner

Reference Issues/PRs

Addresses the zero-weight / deep-tree failure discussed on scikit-learn#34838 and scikit-learn#34745.

What does this implement/fix?

Root cause: HGBT split constraints use unweighted histogram counts (hist.count). Zero-weight rows still entered the grower partition, so they inflated min_samples_leaf / split-stopping even though their gradients and hessians were already 0. With min_samples_leaf=1, deeper trees could keep splitting nodes that would already be leaves if weight-0 rows were absent.

Fix: Pass sample_weight into TreeGrower / Splitter and initialize partition with positively weighted sample indices only (same approach as DecisionTree*). Zero-weight samples never enter a leaf, so they cannot form a zero-weight leaf and no longer affect unweighted count constraints.

Removed the previous fit-time pre-filter / integer-weight expansion helper.

Verification:

  • test_sample_weight_zero_equivalence_deep_trees and grower unit test for partition exclusion
  • Dense common check enabled with min_samples_leaf=1, max_depth=1 (deeper trees can still diverge for positive integer weights due to float32 weight * grad vs summing unweighted gradients)

Still limited: min_samples_leaf > 1 with non-unit positive weights can still diverge from sample repetition until leaf-size constraints use weight sums.

AI usage disclosure

This pull request includes code written with the assistance of AI.
The code has not yet been reviewed by a human.

Open in Web Open in Cursor 

@ogrisel
ogrisel force-pushed the cursor/hgb-zero-weight-fix-b6c7 branch from d636a81 to b6073d1 Compare September 4, 2026 11:13
@cursor cursor Bot changed the title FIX HGBT sample_weight zero-weight / integer equivalence FIX HGBT exclude zero-weight samples in tree grower Sep 4, 2026
cursor Bot pushed a commit that referenced this pull request Sep 4, 2026
Towncrier requires <PR_NUMBER>.<type>.rst; this branch's PR is #27.

Co-authored-by: Olivier Grisel <[email protected]>
@ogrisel
ogrisel force-pushed the cursor/hgb-zero-weight-fix-b6c7 branch from c9c64ab to 6c7b102 Compare September 4, 2026 16:12
cursoragent and others added 8 commits September 7, 2026 08:54
Keep a tiny absolute gain floor on the existing per-bin comparison and
restrict float32-aware tie-breaking to the O(n_features) feature loop.

Co-authored-by: Olivier Grisel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants