POC: Signal progress via progressr #305
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #138
Closes #263
This implements the ideas in futureverse/progressr#85, all of the arguments there are still valid, and the global calling handler part of progressr makes this way easier to swallow as well.
TODO!in docs.progress = TRUE(for advanced cases only)handle_cli()slowness? It is very bad! Do notforcecli to refresh the display progressr#190@HenrikBengtsson this is an experimental ideal that emits
progressr::progressor()backed progress updates after each.finvocation when.progress = TRUE. We create aprogressr::progressor()in the main R process which has as many steps as there are elements to iterate over, and that is then exported to all of the workers. The workers just callp()after each.fcall.I know we've had various conversations about where the user vs developer split is on progress bars. I feel like this is actually a pretty reasonable solution, but I'm very interested to hear your thoughts!
with_progress()or (my preference)handlers(global = TRUE).handlers(handler_cli())and friends.progressr::progressor()2) call it at some interval.progress = FALSE/TRUEflag of their own, which their users should opt into for progression updated. I feel this is particularly important because progression signaling is definitely not "free" and should not really be used with short running tasks.I noticed that
handler_cli()is extremely slow. I see futureverse/progressr#167 and would love to see if we can get to the root of that. The progress bar in purrr doesn't seem to have this issue and is backed by cli. Something weird must be happening here.I imagine user scripts to look something like this