[MRG+2] ENH Loop over candidates as outer loop in search #8322
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This encourages concurrent fits to be over *different datasets* so that fits over the same data subset are more likely to run in serial and hence generate cache hits where memoisation is used.
Codecov Report
@@ Coverage Diff @@
## master #8322 +/- ##
==========================================
+ Coverage 94.73% 94.75% +0.01%
==========================================
Files 342 342
Lines 60674 60801 +127
==========================================
+ Hits 57482 57609 +127
Misses 3192 3192
Continue to review full report at Codecov.
|
amueller
reviewed
May 17, 2017
| @@ -577,8 +577,8 @@ def fit(self, X, y=None, groups=None): | |||
| return_n_test_samples=True, | |||
| return_times=True, return_parameters=False, | |||
| error_score=self.error_score) | |||
| for train, test in cv.split(X, y, groups) | |||
| for parameters in candidate_params) | |||
| for parameters, (train, test) in product(candidate_params, | |||
There was a problem hiding this comment.
Nice. That reverts back to what it was in 0.18 (or pre 0.18?)
|
LGTM. Maybe quick benchmark? |
|
LGTM as well. I'd also like to see how strong this effect is. |
with import time, os
import numpy as np
from sklearn import datasets, model_selection, linear_model, pipeline, decomposition
bunch = datasets.fetch_20newsgroups_vectorized()
X = bunch.data
y = bunch.target
for cv in [3, 5]:
for grid_size in [2, 4]:
for n_jobs in [1, 3, 5, 7]:
os.system('rm -rf /tmp/mem')
pipe = pipeline.Pipeline([('decomp', decomposition.LatentDirichletAllocation(random_state=0, learning_method='batch')),
('clf', linear_model.LogisticRegression(random_state=0))],
memory='/tmp/mem')
search = model_selection.GridSearchCV(pipe,
{'clf__C': np.logspace(-2, 2, grid_size)},
cv=cv, n_jobs=n_jobs)
start = time.time()
search.fit(X, y)
print(cv, grid_size, n_jobs, time.time() - start) |
|
cv grid_size n_jobs master cv_inner
Quite convincing! Thanks for the benchmarks.
|
|
LGTM, merging this one, thanks a lot @jnothman! |
Sundrique
pushed a commit
to Sundrique/scikit-learn
that referenced
this pull request
Jun 14, 2017
…n#8322) This encourages concurrent fits to be over *different datasets* so that fits over the same data subset are more likely to run in serial and hence generate cache hits where memoisation is used.
dmohns
pushed a commit
to dmohns/scikit-learn
that referenced
this pull request
Aug 7, 2017
…n#8322) This encourages concurrent fits to be over *different datasets* so that fits over the same data subset are more likely to run in serial and hence generate cache hits where memoisation is used.
dmohns
pushed a commit
to dmohns/scikit-learn
that referenced
this pull request
Aug 7, 2017
…n#8322) This encourages concurrent fits to be over *different datasets* so that fits over the same data subset are more likely to run in serial and hence generate cache hits where memoisation is used.
NelleV
pushed a commit
to NelleV/scikit-learn
that referenced
this pull request
Aug 11, 2017
…n#8322) This encourages concurrent fits to be over *different datasets* so that fits over the same data subset are more likely to run in serial and hence generate cache hits where memoisation is used.
paulha
pushed a commit
to paulha/scikit-learn
that referenced
this pull request
Aug 19, 2017
…n#8322) This encourages concurrent fits to be over *different datasets* so that fits over the same data subset are more likely to run in serial and hence generate cache hits where memoisation is used.
AishwaryaRK
pushed a commit
to AishwaryaRK/scikit-learn
that referenced
this pull request
Aug 29, 2017
…n#8322) This encourages concurrent fits to be over *different datasets* so that fits over the same data subset are more likely to run in serial and hence generate cache hits where memoisation is used.
maskani-moh
pushed a commit
to maskani-moh/scikit-learn
that referenced
this pull request
Nov 15, 2017
…n#8322) This encourages concurrent fits to be over *different datasets* so that fits over the same data subset are more likely to run in serial and hence generate cache hits where memoisation is used.
jwjohnson314
pushed a commit
to jwjohnson314/scikit-learn
that referenced
this pull request
Dec 18, 2017
…n#8322) This encourages concurrent fits to be over *different datasets* so that fits over the same data subset are more likely to run in serial and hence generate cache hits where memoisation is used.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #8830
As suggested in #7990 (review).
This encourages concurrent fits to be over different datasets so that fits over the same data subset are more likely to run in serial and hence generate cache hits where memoisation is used.
This requires storing all CV splits in memory, consuming
n_samples * n_splitsints worth of memory; the previous version generated these dynamically.