DOC Improve description of l2_regularization for hgbt models #28652
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ArturoAmorQ
commented
Mar 18, 2024
doc/modules/ensemble.rst
Outdated
| of lenght :math:`T` containing the leaf weights. | ||
|
|
||
| Notice that :math:`\gamma` penalizes the number of leaves (which makes it a | ||
| smooth version of `max_leaf_nodes` and is not implemented in scikit-learn), |
There was a problem hiding this comment.
See
for the claim "not implemented in scikit-learn"
added 2 commits
March 18, 2024 15:20
…nto hgbt_regularization
ogrisel
reviewed
Mar 19, 2024
Co-authored-by: Olivier Grisel <[email protected]>
doc/modules/ensemble.rst
Outdated
| `n_classes >= 3`, it uses the multi-class log loss function, with multinomial deviance | ||
| and categorical cross-entropy as alternative names. The appropriate loss version is | ||
| selected based on :term:`y` passed to :term:`fit`. | ||
| Available losses for regression are 'squared_error', 'absolute_error', which is |
There was a problem hiding this comment.
For another PR: quantile is missing as loss.
Co-authored-by: Christian Lorentzen <[email protected]>
…nto hgbt_regularization
|
Otherwise I think all comments have been addressed. |
lorentzenchr
approved these changes
Apr 5, 2024
doc/modules/ensemble.rst
Outdated
Comment on lines
148
to
149
| The leaf values :math:`w_k` are then a continuous value corresponding to the | ||
| loss function to use in the boosting process. Those values contribute to the |
There was a problem hiding this comment.
I do not understand this sentence.
There was a problem hiding this comment.
@ArturoAmorQ Could you fix, rephrase or remove this sentence? Then I can merge.
There was a problem hiding this comment.
I agree the wording here was somewhat vague. I re-wrote the sentence in 20254ab, hoping it does represent an improvement over the previous wording.
Co-authored-by: Christian Lorentzen <[email protected]>
lorentzenchr
reviewed
Apr 8, 2024
doc/modules/ensemble.rst
Outdated
|
|
||
| The leaf values :math:`w_k` are then a continuous value corresponding to the | ||
| loss function to use in the boosting process. Those values contribute to the | ||
| model's prediction for a given input that ends up the corresponding leaf. The final |
There was a problem hiding this comment.
Suggested change
| model's prediction for a given input that ends up the corresponding leaf. The final | |
| model's prediction for a given input that ends up in the corresponding leaf. The final |
added 2 commits
April 8, 2024 10:27
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reference Issues/PRs
What does this implement/fix? Explain your changes.
The current docstring description of the the hyperparameter
l2_regularizationin histogram gradient boosting models is not very descriptive:Visiting the user guide is not helpful either. This PR aims to solve the issue.
Any other comments?
By writing this PR I came out with the doubt: if
l2_regularizationdoes penalize the magnitude of the individual tree predictions, wouldn't that just mean the convergence takes more iterations?