Thanks to visit codestin.com
Credit goes to github.com

Skip to content

PERF Use joblib.Parallel backend='threading' for sparse_encode when called with algorithm='lasso_cd' #4896

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ogrisel opened this issue Jun 25, 2015 · 8 comments
Closed

PERF Use joblib.Parallel backend='threading' for sparse_encode when called with algorithm='lasso_cd' #4896

ogrisel opened this issue Jun 25, 2015 · 8 comments
Labels
Enhancement

Comments

@ogrisel
Copy link
Member

ogrisel commented Jun 25, 2015

Since we released the GIL in most of the Cython coordinate descent solver (#3102), we could now sparse encode in parallel efficiently with threads when using that solver.

Making this change in the code of sparse_encode should be straightforward and the tests should stay the same but accepting a PR for that will require running some benchmarks to check that switching to the threading backend improves memory usage, reduces scheduling overhead and therefore should slightly improve overall sparse encoding speed.

Note: Using threading for the LARS solver might not be efficiently parallelizable with threads. The LARS solver is primarily written in Python / NumPy although we should check as numpy releases the GIL often.
@ogrisel ogrisel added Easy Well-defined and straightforward way to resolve Enhancement labels Jun 25, 2015
@ogrisel
Copy link
Member Author

ogrisel commented Jun 25, 2015

@arthurmensch you might be interested in this.

@arthurmensch
Copy link
Contributor

Profiling sparse_encode with lasso_cd, using joblib.Parallel and multiprocessing.pool.threadPool, we observe that 80% of computation time is still spent in acquire... It might be due to _sparse_encode which acquire the GIL, maybe we could directly implement a n_jobs option within Lasso

@ogrisel
Copy link
Member Author

ogrisel commented Jun 25, 2015

If this is the case it means that the Cython code is actually not in the critical part. Upon further inspection of some profiling results with %lprun -f Lasso.path sparse_encode(some_data...) it seems that redundant input validation checks are dominating the sparse encoding time. We should move those calls to check_array out of the inner loop by adding a check_input=True optional argument to the Lasso.path method and pass check_input=False when it's called in a sparse_encode context.

@ogrisel
Copy link
Member Author

ogrisel commented Jun 25, 2015

Removing the "Easy" tag, as it's not such an easy issue anymore...

@ogrisel ogrisel removed the Easy Well-defined and straightforward way to resolve label Jun 25, 2015
@arthurmensch
Copy link
Contributor

On plot_faces_decomposition, I have a 2,5x improvement in dict_learning_online performance using cd and bypassing checks. Using a parallel pool becomes useful for reasonably large number of feature and batch size. More to come

@arthurmensch
Copy link
Contributor

This is outdated and should be close

@amueller
Copy link
Member

@ogrisel close?

@rth
Copy link
Member

rth commented Jun 20, 2019

This is outdated and should be closed

Closing.

@rth rth closed this as completed Jun 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ogrisel @amueller @rth @arthurmensch