DEP parameter penalty in LogisticRegression and LogisticRegressionCV #32659

lorentzenchr · 2025-11-06T06:06:22Z

Reference Issues/PRs

Partially solves #28711.

This is the the no-brainer of #32042 (comment).

What does this implement/fix? Explain your changes.

This PR

for class LogisticRegression
deprecates the parameters penalty
changes the default of l1_ratio from None to 0
l1_ratio=None is deprecated and forbidden as of 1.10
for class LogisticRegressionCV
deprecates the parameters penalty
changes the default of l1_ratios from None to "warn" (=deprecation of None)
default of l1_ratios will change to (0,) in 1.10
l1_ratios=None is deprecated and forbidden as of 1.10

Any other comments?

github-actions · 2025-11-06T06:07:21Z

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: 689dbee. Link to the linter CI: here}

ogrisel

Thanks for getting this is in. I just have a concern with the warning strategy for l1_ratios in LogisticRegressionCV (see below).

sklearn/linear_model/_logistic.py

lorentzenchr · 2025-11-10T06:20:05Z

I just have a concern with the warning strategy for l1_ratios in LogisticRegressionCV

I am rather indifferent to the possibilities, just say which one:
Change default of l1_ratios from None to (0,)

with proper warning first, i.e. None -> "warn" now (1.8) and "warn" -> (0,) in (1.10) because it changes the shapes of attribues l1_ratios_, coefs_paths_ and scores_
without warning, so just None -> (0,) in (1.8) (like l1_ratio for LogisticRegression
This is now implemented in eab936a.

In any case, l1_ratio(s) = None should be forbidden as of (1.10).

…lready in 1.8, remove 1.10

doc/whats_new/upcoming_changes/sklearn.linear_model/32659.api.rst

sklearn/linear_model/_logistic.py

sklearn/feature_selection/_from_model.py

glemaitre

LGTM. Only 3 tiny changes.

adrinjalali · 2025-11-18T12:06:08Z

A bunch of conflicts now here @lorentzenchr

lorentzenchr · 2025-11-18T19:22:12Z

A bunch of conflicts now here @lorentzenchr

Let's first merge #32073 and then resolve merge conflicts here. In case I don't respond in time for your release schedule, please take over.

lorentzenchr · 2025-11-21T10:27:42Z

@glemaitre @ogrisel It would be nice to get actual review approvals.

This reverts commit 527de21.

lorentzenchr · 2025-11-25T14:06:40Z

@lesteve Thanks for the finish.

lesteve

I pushed a few tweaks and I am posting this comment before I ran out of steam.

Once thing I noticed, sure it's a bit edge casy, it's not super clear in this PR who wins between penalty and l1 (I think it's penalty even if it is deprecated which is kind the same behaviour in 1.7 but there is not something about this in the output)

In [1]: from sklearn.datasets import make_classification
   ...: from sklearn.linear_model import LogisticRegression
   ...: 
   ...: X, y = make_classification()
   ...: lr = LogisticRegression(penalty='l1', l1_ratio=0, solver='saga')
   ...: lr.fit(X, y)
   ...: 
/home/lesteve/dev/scikit-learn/sklearn/linear_model/_logistic.py:1132: FutureWarning: 'penalty' was deprecated in version 1.8 and will be removed in 1.10. To avoid this warning, leave 'penalty' set to its default value and use 'l1_ratio' or 'C' instead. Use l1_ratio=0 instead of penalty='l2', l1_ratio=1 instead of penalty='l1', and C=np.inf instead of penalty=None.
  warnings.warn(

On scikit-learn 1.7 there is a warning that l1_ratio gets ignored ...


In [1]: from sklearn.datasets import make_classification
   ...: from sklearn.linear_model import LogisticRegression
   ...: 
   ...: X, y = make_classification()
   ...: lr = LogisticRegression(penalty='l1', l1_ratio=0, solver='saga')
   ...: lr.fit(X, y)
   ...: 
/home/lesteve/micromamba/lib/python3.12/site-packages/sklearn/linear_model/_logistic.py:1221: UserWarning: l1_ratio parameter is only used when penalty is 'elasticnet'. Got (penalty=l1)
  warnings.warn(
/home/lesteve/micromamba/lib/python3.12/site-packages/sklearn/linear_model/_sag.py:348: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge
  warnings.warn(
Out[1]: LogisticRegression(l1_ratio=0, penalty='l1', solver='saga')

lesteve · 2025-11-25T13:14:55Z

sklearn/linear_model/tests/test_logistic.py

    # Check that an informative error message is raised when penalty="elasticnet"
    # but l1_ratio is not specified.
-    model = LR(penalty="elasticnet", solver="saga")
+    model = LR(penalty="elasticnet", **{arg: None}, solver="saga")


This is a small change of behaviour but I guess fine since it is a bit more permissive than before and this is a bit of a edge case ...

With this PR, LogisticRegression(penalty='elasticnet').fit(X, y) is fine (equivalent to penalty='l2'), because l1_ratio=0. by default. Before it was raising an error because l1_ratio=None.

Similar thing for LogisticRegressionCV

With this PR, LogisticRegression(penalty='elasticnet').fit(X, y) is fine (equivalent to penalty='l2'), because l1_ratio=0. by default. Before it was raising an error because l1_ratio=None.

I think it's acceptable as well. It will be there only until 1.10.

lesteve · 2025-11-25T15:30:51Z

sklearn/linear_model/_logistic.py

        with an L1 penalty, see:
        :ref:`sphx_glr_auto_examples_linear_model_plot_logistic_path.py`.

+    l1_ratio : float, default=0.0


@lorentzenchr just curious, can you explain the reordering, is it to be in the same order as the parameters of the constructor?

This makes the PR harder to review and I feel this is not really necessary but whatever at this point, I will just push on and try to ignore the pain 😅 ...

The parameters for the penalty should be at the start. If penalty is gone, it is C and l1_ratio. Similiar to class ElasticNet.

lesteve · 2025-11-25T15:33:13Z

sklearn/linear_model/_logistic.py

    l1_ratios_ : ndarray of shape (n_l1_ratios)
-        Array of l1_ratios used for cross-validation. If no l1_ratio is used
+        Array of l1_ratios used for cross-validation. If l1_ratios=None is used
        (i.e. penalty is not 'elasticnet'), this is set to ``[None]``


I am not sure "penalty is not 'elasticnet'" is fully accurate here, but I don't have a good suggestion

This will have to change in 1.10 and in the mean time is correct so I wouldn't bother too much

jeremiedbb

A few small comments

sklearn/svm/_bounds.py

sklearn/linear_model/_logistic.py

jeremiedbb · 2025-11-25T15:58:32Z

sklearn/linear_model/_logistic.py

+        if penalty != "elasticnet" and (
+            self.l1_ratio is not None and 0 < self.l1_ratio < 1
+        ):
            warnings.warn(
                "l1_ratio parameter is only used when penalty is "
                "'elasticnet'. Got "
-                "(penalty={})".format(self.penalty)
+                "(penalty={})".format(penalty)


I would add an error for the 2 special non compatible cases penalty=l1 + l1_ratio=0 and penalty=l2 + l1_ratio=1

@lesteve for the case you mention in #32659 (review), I think we should detect it and raise an error here

@lesteve for the case you mention in #32659 (review), I think we should detect it and raise an error here

To be sure we are talking about the same thing you would like the following to raise a snippet in 1.8 even if it's only a UserWarning in 1.7 ¹?

from sklearn.datasets import make_classification from sklearn.linear_model import LogisticRegression X, y = make_classification() lr = LogisticRegression(penalty='l1', l1_ratio=0, solver='saga') lr.fit(X, y)

Footnotes

the warning is saying that l1_ratio gets ignored: UserWarning: l1_ratio parameter is only used when penalty is 'elasticnet'. Got (penalty=l1) ↩

Yeah actually an error is not good because it would break backward compat. I added a warning instead in 49ed659

sklearn/linear_model/_logistic.py

jeremiedbb

LGTM

lesteve · 2025-11-26T04:01:35Z

OK let's merge this one so we can start the rc release process, thanks @lorentzenchr and everyone else for the reviews!

DEP penalty in LogisticRegression

78e2155

github-actions bot added the module:linear_model label Nov 6, 2025

lorentzenchr added this to the 1.8 milestone Nov 6, 2025

lorentzenchr mentioned this pull request Nov 6, 2025

DEP deprecate LogisticRegression parameters penalty and C, introduce new regularization parameter alpha #32042

Open

lorentzenchr added 6 commits November 6, 2025 19:50

DOC add whatsnew entry

3986510

DOC/FIX develop.rst

37eeb63

TST take care of warnings in tests

69b11b5

CLN adapt deprecation of penalty throught codebase

1cc08fb

Merge branch 'main' into dep_logistic_penalty

b0b243a

CLN docstring

f85d664

ogrisel reviewed Nov 7, 2025

View reviewed changes

sklearn/linear_model/_logistic.py Show resolved Hide resolved

sklearn/linear_model/_logistic.py Outdated Show resolved Hide resolved

sklearn/linear_model/_logistic.py Show resolved Hide resolved

lorentzenchr added 2 commits November 10, 2025 07:59

MNT add deprecation warning for l1_ratio=None and review comments

dbbdb07

DEP change default of l1_ratios from "warn" to None, deprecate None a…

eab936a

…lready in 1.8, remove 1.10

glemaitre self-requested a review November 14, 2025 13:12

glemaitre reviewed Nov 14, 2025

View reviewed changes

doc/whats_new/upcoming_changes/sklearn.linear_model/32659.api.rst Outdated Show resolved Hide resolved

glemaitre reviewed Nov 14, 2025

View reviewed changes

sklearn/linear_model/_logistic.py Outdated Show resolved Hide resolved

glemaitre reviewed Nov 14, 2025

View reviewed changes

sklearn/feature_selection/_from_model.py Show resolved Hide resolved

glemaitre reviewed Nov 14, 2025

View reviewed changes

lorentzenchr added 2 commits November 17, 2025 12:07

DOC docstring of l1_ratio and table of solver compatibility

c745f77

CLN review comments

b3b50fa

lesteve force-pushed the dep_logistic_penalty branch from b19fa52 to b3b50fa Compare November 21, 2025 01:43

Merge branch 'main' into dep_logistic_penalty

f0f181a

lorentzenchr mentioned this pull request Nov 21, 2025

FIX LogisticRegressionCV with CV split with missing class labels #32747

Merged

lorentzenchr added 2 commits November 21, 2025 09:14

TST fix warnings in tests

06f6d18

Merge branch 'main' into dep_logistic_penalty

2fe94f6

lesteve added 4 commits November 25, 2025 12:20

[azure parallel] yet another changelog tweak

8fcb264

Revert "This does not seem necessary"

5991522

This reverts commit 527de21.

[azure parallel] tweak

7a71a56

[azure parallel] keep tweaking

370823f

lesteve added 5 commits November 25, 2025 15:26

tweak

7e28999

tweaks in test

f079c5d

tweak warning wording

36533a8

Tweak wording of l1_ratios deprecation + default change

32c26e2

[azure parallel] trigger CI

ffc7c97

lesteve reviewed Nov 25, 2025

View reviewed changes

[azure parallel] tweak changelog again

0ed5277

jeremiedbb reviewed Nov 25, 2025

View reviewed changes

jeremiedbb added 2 commits November 25, 2025 17:23

Update sklearn/svm/_bounds.py

aea4cef

Update sklearn/linear_model/_logistic.py

3646004

github-actions bot added the CI:Linter failure The linter CI is failing on this PR label Nov 25, 2025

jeremiedbb reviewed Nov 25, 2025

View reviewed changes

sklearn/linear_model/_logistic.py Outdated Show resolved Hide resolved

jeremiedbb added 2 commits November 25, 2025 17:30

Apply suggestion from @jeremiedbb

297b1cf

lint

bdec717

github-actions bot removed the CI:Linter failure The linter CI is failing on this PR label Nov 25, 2025

warn for inconsistent penalty and l1_ratio values

49ed659

github-actions bot added the CI:Linter failure The linter CI is failing on this PR label Nov 25, 2025

lint

983d6a5

github-actions bot removed the CI:Linter failure The linter CI is failing on this PR label Nov 25, 2025

ignore other warning in test

689dbee

jeremiedbb approved these changes Nov 25, 2025

View reviewed changes

lesteve merged commit e932688 into scikit-learn:main Nov 26, 2025
36 checks passed

lorentzenchr deleted the dep_logistic_penalty branch November 28, 2025 12:54

lucyleeow mentioned this pull request Dec 23, 2025

LogisticRegression: Conflicting warnings when following deprecation advice for penalty=None (sklearn 1.8.0) #32927

Closed

Uh oh!

DEP parameter penalty in LogisticRegression and LogisticRegressionCV #32659

DEP parameter penalty in LogisticRegression and LogisticRegressionCV #32659

Uh oh!

Conversation

lorentzenchr commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Any other comments?

Uh oh!

github-actions bot commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✔️ Linting Passed

Uh oh!

ogrisel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lorentzenchr commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

glemaitre left a comment

Choose a reason for hiding this comment

Uh oh!

adrinjalali commented Nov 18, 2025

Uh oh!

lorentzenchr commented Nov 18, 2025

Uh oh!

lorentzenchr commented Nov 21, 2025

Uh oh!

lorentzenchr commented Nov 25, 2025

Uh oh!

lesteve left a comment

Choose a reason for hiding this comment

Uh oh!

lesteve Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

jeremiedbb Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

lesteve Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lorentzenchr Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

lesteve Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeremiedbb Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

jeremiedbb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jeremiedbb Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

jeremiedbb Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

lesteve Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Footnotes

Uh oh!

jeremiedbb Nov 25, 2025

Choose a reason for hiding this comment

lorentzenchr commented Nov 6, 2025 •

edited

Loading

github-actions bot commented Nov 6, 2025 •

edited

Loading

lorentzenchr commented Nov 10, 2025 •

edited

Loading

lesteve Nov 25, 2025 •

edited

Loading

lesteve Nov 25, 2025 •

edited

Loading

lesteve Nov 25, 2025 •

edited

Loading