Halving and sample size sensitive hyperparams #31586
cpita-mutt
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi all, I've been testing HalvingGridSearchCV for a large grid of hyperparams, but always getting really bad winners compared to an exhaustive search. Tracking down the problem I realized what is obvious in hindsight: regularization controls (ridge) where set too high because of the small size of initial rounds samples, so the best candidates were rejected at the beginning. Now, it's not that I can set
min_resources
to a larger value without throwing the baby with the bathwater. Even if the estimator provided some sort of size-normalized regularization level, it would be heuristic at best. So is there any sensible strategy to cope with this? If not, it may be a good idea to add a big disclaimer in the user guide / reference. Thanks!Beta Was this translation helpful? Give feedback.
All reactions