Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Example of nested cross-validation #5589

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jnothman opened this issue Oct 24, 2015 · 13 comments · Fixed by #7111
Closed

Example of nested cross-validation #5589

jnothman opened this issue Oct 24, 2015 · 13 comments · Fixed by #7111
Labels
Documentation Easy Well-defined and straightforward way to resolve

Comments

@jnothman
Copy link
Member

We recently merged a change to cross-validation generators that allows, e.g. KFold to be performed within the training of an outer KFold operation. This should be illustrated with an example. Be creative!
@jnothman jnothman added Easy Well-defined and straightforward way to resolve Documentation Moderate Anything that requires some knowledge of conventions and best practices Need Contributor and removed Moderate Anything that requires some knowledge of conventions and best practices labels Oct 24, 2015
@glouppe
Copy link
Contributor

glouppe commented Oct 25, 2015

This should illustrate why model selection and model evaluation should remain separated, and also explain why using tuned CV scores to report generalization estimates is wrong.

@GaelVaroquaux
Copy link
Member

GaelVaroquaux commented Oct 25, 2015 via email

@raghavrv
Copy link
Member

I am working on this and will send a PR shortly (in a day or two) (I am never saying in a day or two again :@)! Thanks for the link Gael!

@Sandy4321
Copy link

cool looking forward

On Sun, Oct 25, 2015 at 4:46 PM, Raghav R V [email protected]
wrote:

I am working on this and will send a PR shortly!

—
Reply to this email directly or view it on GitHub
#5589 (comment)
.

@amueller
Copy link
Member

@rvraghav93 are you still working on this or do you want to hand it over to someone else?

@raghavrv
Copy link
Member

If I don't send a PR within 2 days I'll notify you to add the "Need Cont." tag!

(Sorry for acting like a kid unwilling to give up any of his toys despite not playing with them :P)

@raghavrv
Copy link
Member

BTW this one is a duplicate of #5641 (or the other way around) could you close that?

@amueller amueller mentioned this issue Dec 11, 2015
Closed
@Xndr7
Copy link

Xndr7 commented Apr 25, 2016

I am looking to contribute to this project and, as a beginner, is this a good issue to start with? or has it been solved already?
Cheers!:)

@raghavrv
Copy link
Member

raghavrv commented Apr 25, 2016 •
edited
Loading

@Xndr7 Yes please go ahead! Ping me for any help!

@Xndr7
Copy link

Xndr7 commented Apr 26, 2016

@rvraghav93 since this is my first contribution here, if you could guide me about the procedure with respect to this issue and also a few hints as to how to proceed, it would be much appreciated.
PS:I have read the docs related to contributing and documentation and also have seen the link Gael posted above.

@raghavrv
Copy link
Member

Apologies for the delay in response!

You need to make an example where you show how nested cross-validation is done using our new model_selection's cross-validation objects. (which are from 0.18 not tied to data and hence can be passed to grid search/cross_val_score without having to worry about the size of data used in the inner loops).

I'm not sure if this would genuinely help you or just be considered a shameless plug, but maybe you could read more about it from my blog post here - http://rvraghav93.blogspot.fr/2015/08/gsoc-2015-new-cross-validation.html

@raghavrv raghavrv mentioned this issue Jun 1, 2016
Merged
10 tasks
@mlliou112
Copy link
Contributor

Is this still being worked on? If not, I'd like to try picking this up!

@raghavrv
Copy link
Member

Please go ahead!

@mlliou112 mlliou112 mentioned this issue Jul 29, 2016
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Documentation Easy Well-defined and straightforward way to resolve
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[MRG+1] DOC: Added Nested Cross Validation Example mlliou112/scikit-learn
9 participants
@jnothman @GaelVaroquaux @amueller @glouppe @mlliou112 @raghavrv @TomDLT @Sandy4321 @Xndr7