Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Fix exception in shape_solvers_for_html when objective column has all nan values - #565

Merged
tomMoral merged 1 commit into
benchopt:mainfrom
ogrisel:make-shape_solvers_for_html-robust-to-empty-df
Jun 22, 2023
Merged

Fix exception in shape_solvers_for_html when objective column has all nan values#565
tomMoral merged 1 commit into
benchopt:mainfrom
ogrisel:make-shape_solvers_for_html-robust-to-empty-df

Conversation

@ogrisel

@ogrisel ogrisel commented Jun 21, 2023

Copy link
Copy Markdown
Contributor

As observed in one case with @tomMoral this afternoon.

Here is the full traceback:

benchopt plot .
Processing outputs/benchopt_run_2023-06-21_17h02m11.parquet
/Users/ogrisel/code/benchopt/benchopt/plotting/generate_html.py:216: FutureWarning: The default value of numeric_only in DataFrameGroupBy.median is deprecated. In a future version, numeric_only will default to False. Either specify numeric_only or select only columns which should be valid for the function.
  groupby_stop_val_median = groupby_stop_val_median.median()
Traceback (most recent call last):
  File "/Users/ogrisel/mambaforge/envs/dev/bin/benchopt", line 8, in <module>
    sys.exit(benchopt())
             ^^^^^^^^^^
  File "/Users/ogrisel/mambaforge/envs/dev/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ogrisel/mambaforge/envs/dev/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/ogrisel/mambaforge/envs/dev/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ogrisel/mambaforge/envs/dev/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ogrisel/mambaforge/envs/dev/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ogrisel/code/benchopt/benchopt/cli/process_results.py", line 70, in plot
    plot_benchmark(result_filename, benchmark, kinds=kinds, display=display,
  File "/Users/ogrisel/code/benchopt/benchopt/plotting/__init__.py", line 47, in plot_benchmark
    plot_benchmark_html(fname, benchmark, kinds, display)
  File "/Users/ogrisel/code/benchopt/benchopt/plotting/generate_html.py", line 485, in plot_benchmark_html
    results = get_results(fnames, kinds, root_html, benchmark.name)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ogrisel/code/benchopt/benchopt/plotting/generate_html.py", line 111, in get_results
    result['json'] = json.dumps(shape_datasets_for_html(df))
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ogrisel/code/benchopt/benchopt/plotting/generate_html.py", line 160, in shape_datasets_for_html
    datasets_data[dataset] = shape_objectives_for_html(df, dataset)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ogrisel/code/benchopt/benchopt/plotting/generate_html.py", line 170, in shape_objectives_for_html
    objectives_data[objective] = shape_objectives_columns_for_html(
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ogrisel/code/benchopt/benchopt/plotting/generate_html.py", line 189, in shape_objectives_columns_for_html
    'solvers': shape_solvers_for_html(df_filtered, column),
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ogrisel/code/benchopt/benchopt/plotting/generate_html.py", line 233, in shape_solvers_for_html
    raise Exception(
Exception: Solver can be run using only one stopping strategy. Expected one stopping strategy but found

@ogrisel

ogrisel commented Jun 21, 2023

Copy link
Copy Markdown
Contributor Author

I believe this happened because I had a solver that would always raise an error, even with max_iter=0 (see scikit-learn/scikit-learn#26653).

@Badr-MOUFAD

Copy link
Copy Markdown
Member

Thanks @ogrisel for reporting this bug!
I confirm It solves the problem.

I was able to reproduce with a solver that instead results in an np.inf objective.
It seems weird that you get this problem for a solver that raises an error. Indeed, benchopt is supposed to drop it from the results file .parquet.

That being said, could you pls share the benchmark that reproduces that to investigate the problem further?

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

Merging #565 (bf58b8e) into main (64c658d) will decrease coverage by 0.04%.
The diff coverage is 0.00%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@            Coverage Diff             @@
##             main     #565      +/-   ##
==========================================
- Coverage   54.26%   54.23%   -0.04%     
==========================================
  Files          43       43              
  Lines        2823     2825       +2     
  Branches      516      517       +1     
==========================================
  Hits         1532     1532              
- Misses       1176     1178       +2     
  Partials      115      115              

@tomMoral tomMoral left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tomMoral
tomMoral merged commit 30316e3 into benchopt:main Jun 22, 2023
@tomMoral

Copy link
Copy Markdown
Member

This was on the benchopt/benchmark_logreg_l2 I think, with the new solver from the PR @ogrisel proposed.

@ogrisel

ogrisel commented Jun 22, 2023

Copy link
Copy Markdown
Contributor Author

I did a bunch of ctrl c manually and it was in a dev branch of scikit-learn so i don't think i would know the exact conditions to reproduce it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants