DOC document the primal/dual switch in Ridge cholesky and sparse_cg solvers - #34893
DOC document the primal/dual switch in Ridge cholesky and sparse_cg solvers#34893itz-puneet wants to merge 2 commits into
Conversation
… solvers `cholesky` and `sparse_cg` both solve the primal problem when n_samples >= n_features, but switch to the dual problem when n_features > n_samples: they form the kernel K = X X' in place of the covariance matrix X' X, solve for the dual coefficients, and recover coef_ as X' @ dual_coef. This is unrelated to the `solver="auto"` selection table just above it, which only decides which of these solvers is picked; it does not describe what either of them actually does once picked. The two paths give the same coefficients, but their numerical conditioning differs, and cholesky's dual path has a memory footprint quadratic in n_samples since it forms K explicitly. sparse_cg's dual path is matrix-free and does not form K. Extends the `cholesky`/`sparse_cg` bullets in three places that carry the same solver list: the `ridge_regression` docstring, and the class docstrings of `Ridge` and `RidgeClassifier`. Adds a short paragraph to the Ridge section of the linear_model user guide, after the existing solver-selection table, so the distinction is discoverable from the narrative docs and not only the API reference. Co-Authored-By: Claude Sonnet 5 <[email protected]>
|
Thank you for opening your first pull request to scikit-learn! 🎉 To help get your contribution reviewed, please make sure that:
|
There was a problem hiding this comment.
🟡 Changes recommended
The new docstrings use an undefined “dual_coef” symbol and a coefficient recovery expression whose shape/orientation is misleading for multi-target outputs, so the formulas should be clarified before merge.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates scikit-learn’s Ridge documentation to explicitly describe that the cholesky and sparse_cg solvers switch between primal and dual formulations depending on whether n_samples >= n_features or not, aligning the public docs with the implementation in sklearn/linear_model/_ridge.py.
Changes:
- Expand the solver descriptions for
choleskyandsparse_cginridge_regression,Ridge, andRidgeClassifierto mention the primal/dual switch and its implications. - Add a short explanatory paragraph to the Ridge section of the user guide (
linear_model.rst) immediately after thesolver="auto"selection table.
This pull request includes code written with the assistance of AI.
The code has not yet been reviewed by a human.
File summaries
| File | Description |
|---|---|
| sklearn/linear_model/_ridge.py | Extends solver docstrings to explain primal vs dual behavior for cholesky/sparse_cg. |
| doc/modules/linear_model.rst | Adds a user-guide paragraph clarifying the primal/dual switch and memory/conditioning implications. |
Review details
Suppressed comments (2)
sklearn/linear_model/_ridge.py:1116
- In the Ridge 'cholesky' solver description,
dual_coefis not a public attribute and the expressionX' @ dual_coefhas the transposed shape relative tocoef_for multi-target problems. Use a generic symbol for the dual coefficients and make the transpose explicit (or otherwise clarify the orientation).
kernel ``K = X X'``, solves the dual problem for the dual
coefficients, and recovers ``coef_`` as ``X' @ dual_coef``. Both
sklearn/linear_model/_ridge.py:1490
- In the RidgeClassifier 'cholesky' solver description,
dual_coefis not a public attribute andX' @ dual_coefhas the transposed shape relative tocoef_for multi-target problems. Use a generic symbol for the dual coefficients and make the transpose explicit (or otherwise clarify the orientation).
kernel ``K = X X'``, solves the dual problem for the dual
coefficients, and recovers ``coef_`` as ``X' @ dual_coef``. Both
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…ients The solver descriptions referred to `dual_coef`, which is an internal local in `ridge_regression` rather than anything a reader can look up. `dual_coef_` is a public attribute on other estimators, such as `KernelRidge` and the SVMs, so the name suggested `Ridge` exposes something it does not. The formula also omitted the transpose. `X' c` has shape (n_features, n_targets), while the returned coefficients are (n_targets, n_features). Uses `c` for the dual coefficients, matching the notation already used in the `RidgeCV` Notes section (`w = H^-1 X' y = X' c where c = G^-1 y`), and states the orientation explicitly.
|
Closing as a duplicate. #34807 was already assigned to @jasperjonkhans on 3 September and their PR #34899 covers the same docstrings plus the |
Fixes #34807.
What does this implement/fix? Explain your changes.
choleskyandsparse_cgboth switch between the primal and dual formulation of the ridge problem depending on the shape ofX, but neither docstring said so — only the closed-form primal case was described.Confirmed against
_ridge.py: forn_features > n_samples,choleskyformsK = X X'insklearn/linear_model/_ridge.py:743-749, solves fordual_coefvia_solve_cholesky_kernel, and recoverscoef_ = X' @ dual_coef.sparse_cgdoes the analogous switch matrix-free in_solve_sparse_cg(lines 111-140) via aLinearOperator, without ever formingX' XorX X'explicitly. Checked numerically too:Ridge(solver="cholesky").coef_matches a hand-computed dual-formula solution to float64 precision (~3e-17), and the dual and primal formulas agree with each other to~2e-15on the same problem, confirming the two paths are equivalent as claimed.Extends the shared
cholesky/sparse_cgbullets in the three places that carry the same solver list (ridge_regression,Ridge,RidgeClassifier), and adds a short paragraph to thelinear_model.rstRidge section, right after the existingsolver="auto"selection table — that table only says which solver gets picked, not what either one does once picked, so it's the natural place for a reader to hit this next.Notation in the class docstrings (
K = X X',X' X) matches the convention already used inRidgeCV's Notes section in the same file; the.rstfile uses:math:X^T X`` since that's the convention already used elsewhere in that document.No changelog fragment: none of
major-feature/feature/efficiency/enhancement/fix/api/otherfit a pure docstring clarification with no behavior change. Happy to add one if a maintainer would rather have it underother.On overlap with this issue's discussion: @jasperjonkhans was told "go for it" on this issue on 2026-09-03. I'm aware of that and opened this anyway; apologies if it's unwelcome overlap — happy for a maintainer to close this in favor of their PR if it lands first, or to fold in any differences.
Introduce yourself
New contributor. I use scikit-learn for applied ML work; this came out of reading through the issue tracker for something concrete and well-scoped to verify carefully against source.
AI usage disclosure
I used AI assistance for:
This pull request includes code written with the assistance of AI.
The code has not yet been reviewed by a human.
Any other comments?
Verified:
python -m py_compile/ast.parseon the edited file, line lengths, RST math-role syntax checked against an already-published working example in the same file ((X^T X)^{-1} X^Tin the Classification section), and the numerical check described above. Did not run a fullmake htmlbuild of the documentation.