starting to work on this features

please share me the closest piece of code to your problem statement

@lorentzenchr @ogrisel I assume that it would make sense to be able to control these low-level parameters sometimes. I am bit skeptical to add yet another parameter because it will be solver dependent. However, it could be handy to have a generic parameter to pass additional parameters, that could specific to the underlying solver.

Do you have any thoughts on this?

Could we please first merge #26721 (needs a 2nd reviewer) and then see if we still need control over maxfun. I'm not opposed to it, just first things first.

@lorentzenchr I'll review today.

@vascosmota Could you provide a minimal example?

I am working on creating a minimal example, but it is proving harder than expected. I will update as soon as I have one.

+1 for this as I stumbled over a case where I need to set a higher maxfun.

The question is a bit: Should we add a maxfun parameter to estimators using lbfgs, or should be add a solver_options argument?

Just started fixing this myself.

Design-wise, I would opt for passing an optimizer to fit and/or __init__. The idea is that the user has complete freedom to set up an optimizer of their choice and then they combine it with a model in order to do the fitting. However, this is not how scikit-learn has been put together.

The option that aligns most with the code that already exists is to just add a max_f_evals argument (or to just set maxfun to max_iter). Whether this or a general solver_options argument would be better seems (to me) to depend on a survey of the available optimizers to find out what other parameters users might want to control. However, reducing all the solver-related arguments to the arguments solver and solver_options seems like a good general solution short of passing an optimizer directly. This approach seems like large-scale design work, though, in that it would affect the constructor of every model that uses an optimizer for fitting. So, in that sense, are there short- and long-term solutions (small and big solutions) to be considered here? (Is there someone designated to make these calls?)

Taking the design work done by the Optim.jl people as a proxy for surveying the available SciPy optimizers myself, they have the following standard options: x_tol, f_tol, g_tol, f_calls_limit, g_calls_limit, h_calls_limit, allow_f_increases, iterations, store_trace, show_trace, extended_trace, trace_simplex, show_every, callback, time_limit. See https://julianlsolvers.github.io/Optim.jl/stable/#user/config/#general-options.

Example fixes:

• set maxfun from max_iter: https://github.com/scikit-learn/scikit-learn/compare/main...afbarnard:scikit-learn:issue-27484-fix-copy-max_iter?expand=1
• add a new max_f_evals argument: https://github.com/scikit-learn/scikit-learn/compare/main...afbarnard:scikit-learn:issue-27484-fix-plumb-arg?expand=1

(I've ignored making the same changes to LogisticRegressionCV for now.)