[MRG] ENH: MultiTaskElasticNet (and Lasso) CV #2598

MechCoder · 2013-11-19T10:11:21Z

New MultiTaskElasticNet and Lasso CV.

Do look at the discussions in
#2590
#2402
it was finally decided that having a separate MultiTaskElasticNetCV, would be more useful than multiple usage of ElasticNetCV across n_tasks

MechCoder · 2013-11-19T10:22:00Z

cc @agramfort

agramfort · 2013-11-19T15:39:05Z

sklearn/linear_model/coordinate_descent.py

MultiTaskElasticNetCV should work with all task jointly. It does not make sense to loop over tasks.

MechCoder · 2013-11-20T08:00:39Z

Hi, @agramfort I have my exams coming up, and would be busy for the next 15 - 20 days, and I'm afraid I cannot contribute much to this right now. Is there any deadline before which I should get this merged?

agramfort · 2013-11-20T08:19:48Z

no deadline. Let's finish this after your exams.

MechCoder · 2013-11-22T20:29:42Z

@agramfort : I had some free time and I managed to make a few changes, I also added a enet_multitask_path which would give the alphas for all the multitasks directly. Do have a look.

agramfort · 2013-11-24T20:22:18Z

sklearn/linear_model/coordinate_descent.py

this seems wrong. Can't you avoid the for loop?

MechCoder · 2013-12-08T20:39:51Z

@agramfort : Would you be able to have a look at the latest commits? I have resumed work on this.

MechCoder · 2013-12-09T06:43:33Z

ping @agramfort

MechCoder · 2013-12-10T05:48:53Z

There are some test failures. I shall attempt to rectify them first

MechCoder · 2013-12-14T17:36:32Z

@agramfort : Could you please review this for me? There seems to be only one test failure, due to safe_sparse_dot of a 3-D np array. I hope I'm not troubling you too much, but I just want to finish work on this one.

vene · 2013-12-14T18:04:45Z

I think it would be interesting to have the non_negative option in order to use this as a non-negative least squares algorithm and therefore as a NMF backend. I wonder, since this is a coordinate descent solver, is this the same as coordinate descent NMF? @mblondel can you illuminate me as always? :)

coveralls · 2013-12-15T07:27:20Z

Coverage remained the same when pulling 2853022 on Manoj-Kumar-S:MultiENetCV into aa8139b on scikit-learn:master.

mblondel · 2013-12-15T09:20:44Z

I think that should work but only for dense data (for sparse data, MultiTaskElasticNet would need to support sparse Y). There is nothing fancy about coordinate descent for NMF but a dedicated implementation can make things much faster (e.g., by maintaining temporary matrices for computing the gradient faster).

MechCoder · 2013-12-15T12:39:45Z

@agramfort : I've taken care of all test failures. Any updates on this?

agramfort · 2013-12-15T12:42:46Z

I am looking at it now

agramfort · 2013-12-15T12:46:08Z

sklearn/linear_model/coordinate_descent.py

to me you don't need this param. If y.ndim == 2 then you're in a multi_output case

agramfort · 2013-12-15T12:53:46Z

please start to add tests. I'll continue reviewing afterwards. thanks

MechCoder · 2013-12-15T17:30:56Z

@agramfort : Thanks for the reviews 👍

coveralls · 2013-12-16T18:30:51Z

Coverage remained the same when pulling 866d9e3 on Manoj-Kumar-S:MultiENetCV into aa8139b on scikit-learn:master.

MechCoder · 2013-12-17T20:01:44Z

@agramfort : I've added a few tests. I believe my work for this PR is done. Do you see any major changes to be made?

MechCoder · 2013-12-17T20:05:57Z

@agramfort : Do I need to add docs to the .rst file as well?

coveralls · 2013-12-17T20:15:11Z

Coverage remained the same when pulling 9a60b31 on Manoj-Kumar-S:MultiENetCV into aa8139b on scikit-learn:master.

agramfort · 2014-01-18T21:41:59Z

Ok fair enough

coveralls · 2014-01-18T21:42:26Z

Coverage remained the same when pulling 45d8431 on Manoj-Kumar-S:MultiENetCV into adea283 on scikit-learn:master.

MechCoder · 2014-01-18T21:42:53Z

Tests pass :)

vene · 2014-01-18T21:46:50Z

In test_common.py I remember adding a separate conditional branch for estimators that need 2d Ys (specifically it was for PLS & CCA). Couldn't you get this estimator to follow that code path? Or am I misreading the latest comments?

coveralls · 2014-01-19T05:58:55Z

Coverage remained the same when pulling 1ed9cbf on Manoj-Kumar-S:MultiENetCV into adea283 on scikit-learn:master.

MechCoder · 2014-01-19T06:01:51Z

@vene , @agramfort Well. I personally thought even if y is 1-D , it should be converted to 2-D y for testing. Please tell me if the latest commit looks good.

coveralls · 2014-01-19T06:21:24Z

Coverage remained the same when pulling 35a5a7a on Manoj-Kumar-S:MultiENetCV into 877d892 on scikit-learn:master.

agramfort · 2014-01-19T08:20:20Z

sklearn/linear_model/tests/test_coordinate_descent.py

!!!

l1_ratio does not make sense for the Lasso ! how can this work?

Also when fitting as MultiTaskLassoCV I see that it has the l1_ratio + l1_ratio_ attributes. We should avoid this:

In [38]: c = MultiTaskLassoCV().fit(X, Y) In [39]: c.l1 c.l1_ratio c.l1_ratio_

MechCoder · 2014-01-19T12:43:54Z

@agramfort : I've fixed it by removing the l1_ratio parameter given in the init. It inherits from LinearModelCV and l1_ratio is now set internally in LinearModelCV

coveralls · 2014-01-19T12:50:55Z

Coverage remained the same when pulling b757d30 on Manoj-Kumar-S:MultiENetCV into 877d892 on scikit-learn:master.

agramfort · 2014-01-19T13:21:40Z

+1 for merge on my side. Thanks @manoj-kumar-s

an external review maybe @GaelVaroquaux @fabianp @mblondel @ogrisel @jnothman ?

MechCoder · 2014-01-20T06:40:57Z

@agramfort . Doesn't the earlier +1 by @jnothman still hold (#2598 (comment)) ;)

MechCoder · 2014-01-20T14:56:16Z

@agramfort Can this be merged? @jnothman I understand that you are busy, but all that i need is a +1. :)

jnothman · 2014-01-20T20:17:50Z

It's fine, Manoy. But three reminders in 12 hours really isn't. I'll merge via squash soon, but everything needs its time and space.

MechCoder · 2014-01-20T20:22:58Z

I'm sorry. I just got a bit hyped because this was my first big PR. I promise to be really patient.

jnothman · 2014-01-20T21:42:03Z

Thanks and congratulations, Manoj! It's merged as a917a02.

MechCoder · 2014-01-21T08:09:04Z

Thanks a lot guys :)

@agramfort : Really quick question. Hope you wouldn't mind.
How do I contact you via email (regarding GSoC) ? You seem to have a list of email id's. I don't know which one to write to.

[email protected]
[email protected]
[email protected]

GaelVaroquaux · 2014-01-21T08:55:37Z

On Tue, Jan 21, 2014 at 12:09:07AM -0800, Manoj Kumar wrote:

Thanks a lot guys :)

@agramfort : Really quick question. Hope you wouldn't mind.
How do I contact you via email (regarding GSoC) ? You seem to have a list of
email id's. I don't know which one to write to.

Don't contact anybody privately, use the mailing list, but a very
explicite title.

MechCoder · 2014-01-21T09:30:24Z

@GaelVaroquaux : Ok. Thanks. Umm.. I had written to the the mailing list regarding GMM's for which you were listed as a mentor around 4 days back ( http://sourceforge.net/mailarchive/message.php?msg_id=31860906 ) . I think you might have somehow missed it. Would you be able to reply back and guide me a bit?

agramfort · 2014-01-21T09:46:13Z

@manoj-kumar-s since you seem to have a lot of time :)

maybe you can have a look at the bug of LassoCV and ElasticNetCV that don't agree on results if X is sparse or dense in case fit_intercept=True

MechCoder · 2014-01-21T09:59:19Z

@agramfort Is it this one? #2526
I'll definitely have a look tonight or tomorrow... Running into some college work...

Btw, It would be nice if someone could guide me regarding GMM's on the mailing list. Err. Do I need to be a bit more patient?

agramfort · 2014-01-21T10:05:22Z

@agramfort https://github.com/agramfort Is it this one? #2526 #2526

yes

I'll definitely have a look tonight or tomorrow... Running into some

college work...

Btw, It would be nice if someone could guide me regarding GMM's on the
mailing list. Err. Do I need to be a bit more patient?

yes. Continue to send PRs. The more you know the code base the more you
will find weak spots for a good GSOC proposal.

GaelVaroquaux · 2014-01-21T10:27:28Z

@GaelVaroquaux : Ok. Thanks. Umm.. I had written to the the mailing list
regarding GMM's for which you were listed as a mentor around 4 days back (
http://sourceforge.net/mailarchive/message.php?msg_id=31860906 ) . I think you
might have somehow missed it.

I haven't missed it. I have a huge stack of things to do, and this hasn't
gone up high enough the priority list.

I am sorry, but we all get a lot of requests on many fronts, and
scikit-learn is for most of us a tiny fraction of our activity.

MechCoder · 2014-01-21T12:08:30Z

@GaelVaroquaux : Ok, I will continue trying to fix bugs, till you are free enough. Is that ok?

agramfort reviewed Nov 19, 2013
View reviewed changes

agramfort reviewed Nov 24, 2013
View reviewed changes

sklearn/linear_model/coordinate_descent.py Outdated

Copy link

Member

agramfort Nov 24, 2013

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems wrong. Can't you avoid the for loop?

agramfort reviewed Dec 15, 2013
View reviewed changes

agramfort mentioned this pull request Dec 19, 2013

[DOC] LassoCV.fit() taking multiple targets? #2678

Closed

FIX: Multi-output Estimator raises Error for mono-output y

35a5a7a

agramfort reviewed Jan 19, 2014
View reviewed changes

FIX: Remove l1_ratio and l1_ratio_ attributes for MTaskLassoCV

b757d30

jnothman closed this Jan 20, 2014

MechCoder deleted the MultiENetCV branch January 21, 2014 04:19

MechCoder mentioned this pull request Jul 3, 2014

[MRG] Logistic Regression CV #2862

Merged

5 tasks

Uh oh!

[MRG] ENH: MultiTaskElasticNet (and Lasso) CV #2598

[MRG] ENH: MultiTaskElasticNet (and Lasso) CV #2598

Uh oh!

Conversation

MechCoder commented Nov 19, 2013

Uh oh!

MechCoder commented Nov 19, 2013

Uh oh!

agramfort Nov 19, 2013

Choose a reason for hiding this comment

Uh oh!

MechCoder commented Nov 20, 2013

Uh oh!

agramfort commented Nov 20, 2013

Uh oh!

MechCoder commented Nov 22, 2013

Uh oh!

agramfort Nov 24, 2013

Choose a reason for hiding this comment

Uh oh!

MechCoder commented Dec 8, 2013

Uh oh!

MechCoder commented Dec 9, 2013

Uh oh!

MechCoder commented Dec 10, 2013

Uh oh!

MechCoder commented Dec 14, 2013

Uh oh!

vene commented Dec 14, 2013

Uh oh!

coveralls commented Dec 15, 2013

Uh oh!

mblondel commented Dec 15, 2013

Uh oh!

MechCoder commented Dec 15, 2013

Uh oh!

agramfort commented Dec 15, 2013

Uh oh!

agramfort Dec 15, 2013

Choose a reason for hiding this comment

Uh oh!

agramfort commented Dec 15, 2013

Uh oh!

MechCoder commented Dec 15, 2013

Uh oh!

coveralls commented Dec 16, 2013

Uh oh!

MechCoder commented Dec 17, 2013

Uh oh!

MechCoder commented Dec 17, 2013

Uh oh!

coveralls commented Dec 17, 2013

Uh oh!

agramfort commented Jan 18, 2014

Uh oh!

coveralls commented Jan 18, 2014

Uh oh!

MechCoder commented Jan 18, 2014

Uh oh!

vene commented Jan 18, 2014

Uh oh!

coveralls commented Jan 19, 2014

Uh oh!

MechCoder commented Jan 19, 2014

Uh oh!

coveralls commented Jan 19, 2014

Uh oh!

agramfort Jan 19, 2014

Choose a reason for hiding this comment

Uh oh!

agramfort Jan 19, 2014

Choose a reason for hiding this comment

Uh oh!

MechCoder commented Jan 19, 2014

Uh oh!

coveralls commented Jan 19, 2014

Uh oh!

agramfort commented Jan 19, 2014

Uh oh!