Proposal

Make the options to choose a loss or a split criterion consistent/unified over all estimators, classifiers as well as regressors.

Current State of Affairs

The API for setting a loss function or a split criterion is scattered. Here is an incomplete list as of v0.23.0:

Classifiers

• sklearn.ensemble.GradientBoostingClassifier
  • loss : {‘deviance’, ‘exponential’}, default=’deviance’
  • criterion : {‘friedman_mse’, ‘mse’, ‘mae’}, default=’friedman_mse’
• sklearn.ensemble.HistGradientBoostingClassifier
  • loss : {‘auto’, ‘binary_crossentropy’, ‘categorical_crossentropy’}, optional (default=’auto’)
• sklearn.linear_model.SGDClassifier
  • loss : {‘hinge’, ‘log’, ‘modified_huber’, ‘squared_hinge’, ‘perceptron’, ‘squared_loss’, ‘huber’, ‘epsilon_insensitive’, ‘squared_epsilon_insensitive’.}, default=’hinge’
• sklearn.svm.LinearSVC
  • loss : {‘hinge’, ‘squared_hinge’}, default=’squared_hinge’
• sklearn.tree.DecisionTreeClassifier
  • criterion : {“gini”, “entropy”}, default=”gini”

Regressors

• sklearn.ensemble.GradientBoostingRegressor

  • loss : {‘ls’, ‘lad’, ‘huber’, ‘quantile’}, default=’ls’
  • criterion : {‘friedman_mse’, ‘mse’, ‘mae’}, default=’friedman_mse’

• sklearn.ensemble.HistGradientBoostingRegressor

  • loss : {‘least_squares’, ‘least_absolute_deviation’, ‘poisson’}, optional (default=’least_squares’)

• sklearn.linear_model.SGDRegressor

  • loss : {‘squared_loss’, ‘huber’, ‘epsilon_insensitive’, ‘squared_epsilon_insensitive’}, default=’squared_loss’

• sklearn.tree.DecisionTreeRegressor

  • criterion : {“mse”, “friedman_mse”, “mae”}, default=”mse”

• sklearn.svm.LinearSVR

  • loss : {‘epsilon_insensitive’, ‘squared_epsilon_insensitive’}, default=’epsilon_insensitive’

Additional context

Supersedes #3481. Furthermore, #15123 is a bit related, though that one is about private functionality while this one is about public API.

Update

• "squared_error" in [MRG] ENH Consistent loss name for squared error #19310
• "absolute_error in ENH Consistent loss name for absolute error #19733
• Decide name for log loss
  • DEP deviance in favor of log_loss for GradientBoostingClassifier #23036
  • DEP auto, binary_crossentropy, categorical_crossentropy in HGBT #23040
  • DEP loss "log" in favor of "log loss" in SGDClassifier #23046
  • ENH add criterion log_loss as alternative to entropy in trees and forests #23047