Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[STALLED] Optional verbosity for Pipeline #5321

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

samzhang111
Copy link

Adds named verbosity argument in Pipeline constructor.

After each step, verbose pipelines print to standard output lines like:

[Pipeline] <step name>, <action> (fit or transform), <duration>
#5298

@jnothman
Copy link
Member

Apart from the use of closures breaking pickling, modifying an estimator because of the context it's used in could be pretty nasty to the user... You'll need to rethink the design.

@samzhang111
Copy link
Author

I wanted to avoid putting timing code in each method, and I wanted fit_transform to break out separate log messages for fit and transform.

Breaking pickling seems unacceptable though. I can make a helper function that takes the fit/transform method as an argument and prints timing info (ugly, unfortunately), or just write timers into each method. What do you prefer?

@jnothman
Copy link
Member

I suspect that usually we want timing information for all fit calls, and
you should focus on that initially. If we really think that transform calls
at prediction time are useful to time separately, we could do so only under
a high verbosity setting. I wouldn't object to the timing incorporating
both fit and transform in the _pre_transform helper.

On 27 September 2015 at 08:26, Sam Zhang [email protected] wrote:

I wanted to avoid putting timing code in each method, but breaking
pickling is unacceptable. I can make a helper function that handles the
call to fit/transform and prints timing info, or just break it out into
each method.

(broke docstrings)


Reply to this email directly or view it on GitHub
#5321 (comment)
.

@samzhang111
Copy link
Author

I've taken those suggestions into account and left prediction-time calls untimed for now.

if self.verbose:
elapsed = time.time() - start_time
time_str = logger.short_format_time(elapsed)
print('[Pipeline] %s ... %s' % (name, time_str))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's be a bit more informative and say (step 1 of 10) for instance.

@jnothman
Copy link
Member

LGTM! Please make that little tweak to informativeness, and add an entry to whats_new.rst

@amueller
Copy link
Member

LGTM apart from @jnothman's comments

@GaelVaroquaux GaelVaroquaux changed the title Optional verbosity for Pipeline [MRG+1] Optional verbosity for Pipeline Oct 13, 2015
@samzhang111 samzhang111 force-pushed the verbose-pipeline-5298 branch 2 times, most recently from 1a8fa35 to 404ee4c Compare October 23, 2015 06:19
@amueller
Copy link
Member

could you please rebase?

Adds named verbosity argument in Pipeline constructor.

After each step, verbose pipelines print to standard output lines like:
> [Pipeline] <step name>, <action> (fit or transform), <duration>
@samzhang111 samzhang111 force-pushed the verbose-pipeline-5298 branch from 404ee4c to a42026c Compare December 10, 2015 16:33
@amueller
Copy link
Member

hm didn't work :-/

@ClimbsRocks
Copy link
Contributor

@samzhang111 Thanks for picking this up! I was looking for exactly this functionality. If you're still interested in getting this merged, I'd love to be an early user :)

@jnothman
Copy link
Member

jnothman commented Aug 8, 2016

I was almost ready to merge at #7163 when I decided to check the output. Cross-validation's _fit_and_score helper ensures all line lengths are the same when reporting times. I think we should conform here. Indeed, a utility:

def _log_time(left_text, right_text, time, width=70, file=sys.stdout):
    pass

would be appropriate.

@samzhang111, are you still around and interested in working on this?

@amueller
Copy link
Member

@samzhang111 I guess that's a "no"?

@samzhang111
Copy link
Author

Sorry, I have been on vacation. If someone would like to do it quickly,
please. Otherwise I can later this weekend.

On Fri, Aug 26, 2016 at 1:07 PM, Andreas Mueller [email protected]
wrote:

@samzhang111 https://github.com/samzhang111 I guess that's a "no"?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#5321 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACq6obRF-4IGVE-XcAmXrTcPVvJxaPSJks5qjx09gaJpZM4GEYmS
.

@jnothman
Copy link
Member

jnothman commented Nov 1, 2016

@samzhang111, I'm going to open this to another contributor to complete. Let me know if you're pushing on with it.

@raghavrv raghavrv changed the title [MRG+1] Optional verbosity for Pipeline [STALLED] Optional verbosity for Pipeline Dec 2, 2016
@jnothman jnothman added the Easy Well-defined and straightforward way to resolve label Feb 23, 2017
@jimmywan
Copy link
Contributor

jimmywan commented May 9, 2018

Close in favor of #10435 just to avoid clutter?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Easy Well-defined and straightforward way to resolve Stalled
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants