Codestin Search App

s3alfisc · 2025-08-24T10:42:37Z

Reference Issues/PRs

See this issue. #31708

What does this implement/fix? Explain your changes.

Implements a fast Frisch-Newton Interior Point Solver for Vanilla Quantile Regression without penalization.

…main

github-actions · 2025-08-24T10:43:33Z

❌ Linting issues

This PR is introducing linting issues. Here's a summary of the issues. Note that you can avoid having linting issues by enabling pre-commit hooks. Instructions to enable them can be found here.

You can see the details of the linting issues under the lint job here

`ruff check`

ruff detected issues. Please run ruff check --fix --output-format=full locally, fix the remaining issues, and push the changes. Here you can see the detected issues. Note that the installed ruff version is ruff=0.11.7.

Details


sklearn/linear_model/_quantile.py:262:89: E501 Line too long (91 > 88)
    |
260 |         else:
261 |             if self.solver in ["highs", "highs-ds", "highs-ipm"]:
262 |                 # Note that highs methods always use a sparse CSC memory layout internally,
    |                                                                                         ^^^ E501
263 |                 # even for optimization problems parametrized using dense numpy arrays.
264 |                 # Therefore, we work with CSC matrices as early as possible to limit
    |

sklearn/linear_model/_quantile.py:375:21: RUF013 PEP 484 prohibits implicit `Optional`
    |
373 |     q: float,
374 |     backoff: float = 0.9995,
375 |     solver_options: dict = None,
    |                     ^^^^ RUF013
376 | ):
    |
    = help: Convert to `T | None`

Found 2 errors.
No fixes available (1 hidden fix can be enabled with the `--unsafe-fixes` option).

`ruff format`

ruff detected issues. Please run ruff format locally and push the changes. Here you can see the detected issues. Note that the installed ruff version is ruff=0.11.7.

Details


--- sklearn/linear_model/_quantile.py
+++ sklearn/linear_model/_quantile.py
@@ -348,7 +348,6 @@
 
 
 def _cold_start(A: np.ndarray, c: np.ndarray, q: float):
-
     "Set starting values for the Frisch-Newton algorithm."
     n = A.shape[1]
     x = np.full(n, 1.0 - q)
@@ -374,7 +373,6 @@
     backoff: float = 0.9995,
     solver_options: dict = None,
 ):
-
     """
     Implements the Frisch-Newton Interior Point Solver for Quantile Regression
     following Koenker and Ng (2005, https://link.springer.com/article/10.1007/s10255-005-0231-1).

1 file would be reformatted, 925 files already formatted

`mypy`

mypy detected issues. Please fix them locally and push the changes. Here you can see the detected issues. Note that the installed mypy version is mypy=1.15.0.

Details


sklearn/linear_model/_quantile.py:375: error: Incompatible default for argument "solver_options" (default has type "None", argument has type "dict[Any, Any]")  [assignment]
sklearn/linear_model/_quantile.py:375: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
sklearn/linear_model/_quantile.py:375: note: Use https://github.com/hauntsaninja/no_implicit_optional to automatically upgrade your codebase
Found 1 error in 1 file (checked 565 source files)

_{Generated for commit: 9a5238d. Link to the linter CI: here}

…main

ogrisel

Here is a shallow first pass of review. Please also update the existing tests to include the new solver (or extend them to deal with the fact that it only accept alpha=0).

Please also add a changelog entry as explained here:

https://github.com/scikit-learn/scikit-learn/blob/main/doc/whats_new/upcoming_changes/README.md

BTW, do you think this solver could be generalized to handle non-zero l2 or (l1 + l2) regularization?

ogrisel · 2025-08-25T16:02:26Z

@@ -46,7 +46,11 @@ class QuantileRegressor(LinearModel, RegressorMixin, BaseEstimator):
    solver : {'highs-ds', 'highs-ipm', 'highs', 'interior-point', \
            'revised simplex'}, default='highs'


You need to also add the new method to the list of possible values above.

ogrisel · 2025-08-25T16:03:00Z

            )

+        if self.solver == "frisch-newton" and alpha != 0:
+            raise ValueError("Frisch-Newton solver does not support alpha != 0.")


Please recall the actual value of alpha in the error message.

ogrisel · 2025-08-25T16:06:35Z

+    if not success:
+        warnings.warn(
+            f"The 'frisch-newton' solver did not converge after {max_iter} iterations."
+        )


This warning should be an instance of sklearn.exceptions.ConvergenceWarning.

ogrisel · 2025-08-25T16:10:51Z

+    """
+    if solver_options is None:
+        solver_options = {}
+    tol = solver_options.get("tol", 1e-7)


We might want to instead tol as an estimator parameter directly, as is customary for many scikit-learn estimators. We might also want to make this tolerance criterion independent of data scale if this is not already the case. Since the convergence is related to a duality gap, we could have a look at how it's done in ElasticNet and co.

Co-authored-by: Olivier Grisel <[email protected]>

s3alfisc · 2025-08-25T19:43:12Z

Thanks for taking a first look at this @ogrisel, I'll work on this within the next days!

lorentzenchr · 2025-08-26T07:26:59Z

+
+    """
+    Implements the Frisch-Newton Interior Point Solver for Quantile Regression
+    following Koenker and Ng (2005, https://link.springer.com/article/10.1007/s10255-005-0231-1).


As the Paper is behind a paywall, it would be nice to give a short derivation, like in

scikit-learn/sklearn/linear_model/_quantile.py

Lines 200 to 216 in 969df01

# After rescaling alpha, the minimization problem is

# min sum(pinball loss) + alpha * L1

# Use linear programming formulation of quantile regression

# min_x c x

# A_eq x = b_eq

# 0 <= x

# x = (s0, s, t0, t, u, v) = slack variables >= 0

# intercept = s0 - t0

# coef = s - t

# c = (0, alpha * 1_p, 0, alpha * 1_p, quantile * 1_n, (1-quantile) * 1_n)

# residual = y - X@coef - intercept = u - v

# A_eq = (1_n, X, -1_n, -X, diag(1_n), -diag(1_n))

# b_eq = y

# p = n_features

# n = n_samples

# 1_n = vector of length n with entries equal one

# see https://stats.stackexchange.com/questions/384909/

lorentzenchr · 2025-08-26T08:15:11Z

+
+            params, _it = frisch_newton_solver(
+                A=X.T,
+                b=(1 - self.quantile) * X.T @ np.ones(n_indices),


Suggested change

b=(1 - self.quantile) * X.T @ np.ones(n_indices),

b=(1 - self.quantile) * X.sum(axis=0),

lorentzenchr · 2025-08-26T08:16:45Z

-                A_eq = sparse.hstack([X, -X, eye, -eye], format="csc")
-        else:
-            eye = np.eye(n_indices)
+        if self.solver == "frisch-newton":


Suggested change

if self.solver == "frisch-newton":

if self.solver == "frisch-newton":

Could you add frisch-newton after the linprog? This way the comment around line 202/210 makes more sense and the FN-solver gets closer to its implementation.

lorentzenchr · 2025-08-26T08:17:52Z

+
+    "Set starting values for the Frisch-Newton algorithm."


Suggested change

"Set starting values for the Frisch-Newton algorithm."

"""Set starting values for the Frisch-Newton algorithm."""

lorentzenchr · 2025-08-26T08:19:01Z

+    return min(_bound(x, dx, backoff), _bound(s, ds, backoff))
+
+
+def _cold_start(A: np.ndarray, c: np.ndarray, q: float):


Suggested change

def _cold_start(A: np.ndarray, c: np.ndarray, q: float):

def _cold_start(A, c, q):

I guess it's fine without type annotations. A can be sparse, so ndarray is not the correct type.

lorentzenchr · 2025-08-26T08:20:34Z

+
+
+def frisch_newton_solver(
+    A: np.ndarray,


Suggested change

A: np.ndarray,

A: np.ndarray,

Again annotations, at least A can be sparse.

lorentzenchr · 2025-08-26T08:21:16Z

+
+    mu_curr = _duality_gap(x=x, z=z, s=s, w=w)
+
+    for _it in range(max_iter):


Suggested change

for _it in range(max_iter):

for it in range(max_iter):

or just i.

github-actions · 2026-09-08T13:58:39Z

⏰ This pull request might be automatically closed in two weeks from now.

Thank you for your contribution to scikit-learn and for the effort you have put into this PR. This pull request does not yet meet the quality and clarity needed for an effective review. Project maintainers have limited time for code reviews, and our goal is to prioritize well-prepared contributions to keep scikit-learn maintainable.

To increase the chance of a productive review, please refer to: How do I improve my issue or pull request? As the author, you are responsible for driving this PR, which entails doing necessary background research as well as presenting its context and your thought process. If you are a new contributor, or do not know how to fulfill these requirements, we recommend that you familiarise yourself with scikit-learn's development conventions via other contribution types (e.g., reviewing PRs) before submitting code.

Scikit-learn maintainers cannot provide one-to-one guidance on this PR. However, if you ask focused, well-researched questions, a community member may be willing to help. 💬

If you substantially improve this PR within two weeks, a team member may remove the autoclose label and the PR stays open. Cosmetic changes or incomplete fixes will not be sufficient. Maintainers will assess improvements on their own schedule. Please do not ping (@) maintainers.

s3alfisc added 5 commits August 22, 2025 18:28

implement FN solver

1a3168d

Merge branch 'scikit-learn:main' into main

6c3f08a

cleanups

b689ec2

Merge branch 'main' of https://github.com/s3alfisc/scikit-learn into …

8c4fd48

…main

Merge branch 'scikit-learn:main' into main

31a024b

github-actions Bot added the module:linear_model label Aug 24, 2025

s3alfisc added 2 commits August 24, 2025 12:43

fix copy-paste error

36abc49

Merge branch 'main' of https://github.com/s3alfisc/scikit-learn into …

6045050

…main

s3alfisc marked this pull request as draft August 24, 2025 10:48

s3alfisc mentioned this pull request Aug 24, 2025

Frisch-Newton Interior Point Solver for Quantile Regression #31708

Open

ogrisel reviewed Aug 25, 2025

View reviewed changes

cleanup comments

9a5238d

Co-authored-by: Olivier Grisel <[email protected]>

lorentzenchr reviewed Aug 26, 2025

View reviewed changes

lorentzenchr added the autoclose PR automatically closed 14 days after setting the label label Sep 8, 2026

s3alfisc closed this Sep 8, 2026

		@@ -46,7 +46,11 @@ class QuantileRegressor(LinearModel, RegressorMixin, BaseEstimator):
		solver : {'highs-ds', 'highs-ipm', 'highs', 'interior-point', \
		'revised simplex'}, default='highs'

	# After rescaling alpha, the minimization problem is
	# min sum(pinball loss) + alpha * L1
	# Use linear programming formulation of quantile regression
	# min_x c x
	# A_eq x = b_eq
	# 0 <= x
	# x = (s0, s, t0, t, u, v) = slack variables >= 0
	# intercept = s0 - t0
	# coef = s - t
	# c = (0, alpha * 1_p, 0, alpha * 1_p, quantile * 1_n, (1-quantile) * 1_n)
	# residual = y - X@coef - intercept = u - v
	# A_eq = (1_n, X, -1_n, -X, diag(1_n), -diag(1_n))
	# b_eq = y
	# p = n_features
	# n = n_samples
	# 1_n = vector of length n with entries equal one
	# see https://stats.stackexchange.com/questions/384909/

	b=(1 - self.quantile) * X.T @ np.ones(n_indices),
	b=(1 - self.quantile) * X.sum(axis=0),

	if self.solver == "frisch-newton":
	if self.solver == "frisch-newton":


	"Set starting values for the Frisch-Newton algorithm."
	"""Set starting values for the Frisch-Newton algorithm."""

		return min(_bound(x, dx, backoff), _bound(s, ds, backoff))


		def _cold_start(A: np.ndarray, c: np.ndarray, q: float):

	def _cold_start(A: np.ndarray, c: np.ndarray, q: float):
	def _cold_start(A, c, q):


		mu_curr = _duality_gap(x=x, z=z, s=s, w=w)

		for _it in range(max_iter):

Uh oh!

Uh oh!

Conversation

s3alfisc commented Aug 24, 2025

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Uh oh!

github-actions Bot commented Aug 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ Linting issues

ruff check

ruff format

mypy

Uh oh!

ogrisel left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

s3alfisc commented Aug 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Aug 24, 2025 •

edited

Loading

`ruff check`

`ruff format`

`mypy`