PRF Ridge - centering trick for cholesky solver - #34793
Conversation
… to svd
use_no_center_cholesky matched on solver in ("auto", "cholesky") but didn't
check the array namespace. With array API dispatch to a non-numpy namespace,
solver="auto" silently resolves to "svd" instead of "cholesky" (see
resolve_solver), which needs X to actually be centered. This left X
uncentered while running svd, causing
test_cross_val_predict_array_api_compliance[...-Ridge] failures on
array_api_strict and torch in CI.
| elif is_X_sparse and fit_intercept: | ||
| # when `fit_intercept=True`, `mean_variance_axis` will be called on X | ||
| # and it requires csr/csc format | ||
| return ["csr", "csc"] |
There was a problem hiding this comment.
Note to reviewers: this addition is needed because I now pass check_input=False to _preprocess_data which would have done this logic.
|
cc @antoinebaker, I was told you're the specialist of this kind of tricks 😉 |
|
I checked memory usage and I did see it saving 2GB of RAM of a X of 2GB (which is expected as it avoids a copy). I guess this is a nice improvement, as memory might be a limit more often than run time for this model that usually is very fast. Edit: Hum... I just had an OOM caused by Ridge on main while running benchmarks... 😄 |
ogrisel
left a comment
There was a problem hiding this comment.
The centering and validation optimizations look good to me but I don't understand the change related to the solvers.
Also please add a changelog entry.
|
cc @antoinebaker. BTW similar work should be done for RidgeCV as a followup to this PR:
|
Reference Issues/PRs
None, I started investigated for a speed gain while comparing speed/profiles with the scikit-learn-intelex (oneDAL) implementation.
What does this implement/fix? Explain your changes.
check_array(and to avoid a copy in the algebraic centering Cholesky path).Those relatively simple changes provide a speedups ranging from ~1.2x to 2x.
AI usage disclosure
I used AI assistance for:
Benchmarks
Benchmarks are run through the PR: probabl-ai/scikit-learn-benchmarks#31 (see bot comments with links to dashboards)
The benchmark cases are the same than the ones used to compare with scikit-learn-intelex, they are a mix of synthetic and real-world datasets.
On the intel-laptop, one case show a regression but it's a benchmark artifact: this case uses the dual path, which is not affected by this PR. And if you look at the granular timings, you'll see in both branches (main and this one) oscillate a lot.