Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[MRG] Turns off memory_profiler in examples to stop circleci from hanging #16629

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 25 commits into from
Mar 9, 2020
Merged

[MRG] Turns off memory_profiler in examples to stop circleci from hanging #16629

merged 25 commits into from
Mar 9, 2020

Conversation

thomasjpfan
Copy link
Member

@thomasjpfan thomasjpfan commented Mar 4, 2020 •
edited
Loading

Reference Issues/PRs

Fixes #16628

What does this implement/fix? Explain your changes.

Update sphinx to try to get circleci to not timeout.

(Also removes azure pipeline configuration to be nice to the CI, while I test)

@thomasjpfan
Copy link
Member Author

Hmm removing the memory profiler resolve the issue

@adrinjalali
Copy link
Member

So do we just remove the memory profiler, or also upgrade sphinx? (I'm happy with bumping up the scphinx version)

@thomasjpfan
Copy link
Member Author

We may need to disable memory profiler. The gallery generation hangs for Sphinx 2.2-2.4.

@thomasjpfan thomasjpfan mentioned this pull request Mar 5, 2020
Open
@thomasjpfan thomasjpfan changed the title [NOMRG] Update sphinx to 2.3 to fix circleci timeout [NOMRG] Reduces the number of samples in plot_sparse_logistic_regression_mnist to prevent circleci from freezing Mar 6, 2020
@thomasjpfan thomasjpfan changed the title [NOMRG] Reduces the number of samples in plot_sparse_logistic_regression_mnist to prevent circleci from freezing [MRG] Turns off memory_profiler in examples to stop circleci from hanging Mar 8, 2020
@thomasjpfan
Copy link
Member Author

I have tried to remove the high memory usage examples and circleci still hangs. For now, this PR now just turns off the memory_profiler.

adrinjalali
adrinjalali approved these changes Mar 9, 2020
View reviewed changes
Copy link
Member

@adrinjalali adrinjalali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @thomasjpfan

How can we check the memory usage of different tests/documents locally to try and fix them?

rth
rth approved these changes Mar 9, 2020
View reviewed changes
Copy link
Member

@rth rth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Thomas!

@rth rth merged commit eb3f5df into scikit-learn:master Mar 9, 2020
@rth
Copy link
Member

rth commented Mar 9, 2020

Good to see master CI finally green!

@thomasjpfan
Copy link
Member Author

You can test using memory_profiler:

mprof run python examples/...

I tried completely removing the high memory examples and circleci still stalled. For reference the high memory examples are:

  1. examples/linear_model/plot_sgd_early_stopping.py
  2. examples/linear_model/plot_sparse_logistic_regression_mnist.py
  3. examples/linear_model/plot_poisson_regression_non_normal_loss.py
  4. examples/linear_model/plot_tweedie_regression_insurance_claims.py

In all cases, the original dataset loading lead to a bunch of memory.

As a follow up, I am planning to test if #16084 resolves the issue. (That PR is needs one more review to get merged)

@rth
Copy link
Member

rth commented Mar 9, 2020

Thanks for investigating. Yes, the last two examples load a somewhat large dataset and the openml loader needs improvement.

ashutosh1919 pushed a commit to ashutosh1919/scikit-learn that referenced this pull request Mar 13, 2020
@thomasjpfan @ashutosh1919
BLD Turns off memory_profiler in examples to fix CircleCI (scikit-lea…
68d1fef 
…rn#16629)
@thomasjpfan thomasjpfan mentioned this pull request Apr 5, 2020
Merged
gio8tisu pushed a commit to gio8tisu/scikit-learn that referenced this pull request May 15, 2020
@thomasjpfan @gio8tisu
BLD Turns off memory_profiler in examples to fix CircleCI (scikit-lea…
75e1936 
…rn#16629)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rth rth rth approved these changes

@adrinjalali adrinjalali adrinjalali approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Circleci has been timing out when building docs
3 participants
@thomasjpfan @adrinjalali @rth