-
Notifications
You must be signed in to change notification settings - Fork 76
Description
Prework
- Read and agree to the code of conduct and contributing guidelines.
- If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
- New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.
- Format your code according to the tidyverse style guide.
Background
Several people have requested this feature, most recently @jennysjaarda in #522, and also @kendonB and @MilesMcBain some time ago.
targets normally traverses the entire pipeline and checks every single target even when you know in advance everything should be up to date. As @jennysjaarda pointed out, skipping lots of targets adds overhead to initialization times. In the example below, it takes a long time to get to the finalize target. If you iterate a lot on finalize, you still have to skip branches, and the delay due to checking targets is annoying.
# _targets.R
library(targets)
list(
tar_target(index, seq_len(1000)),
tar_target(branches, index, pattern = map(index)),
tar_target(finalize, sum(branches))
)# R console
system.time(tar_make())
#> ✓ skip target index
#> ✓ skip branch branches_77ba9815
#> ✓ skip branch branches_4b7abfa4
#> ✓ skip branch branches_f5a3cbc2
#> ✓ skip branch branches_b0df9eb8
#> ...
#> user system elapsed
#> 5.114 0.650 5.134 Proposal
I propose a shortcut argument to tar_make() and related functions so that we can jump right to the targets you mention in names. We bootstrap upstream dependencies from the metadata so required data can be loaded if needed.
> system.time(tar_make(names = all_of("finalize"), shortcut = TRUE))
✓ skip target finalize
✓ skip pipeline
#> user system elapsed
#> 0.984 0.133 1.196 # Some overhead due to callr cannot be helped.For the longest time, I thought this feature would break everything. But this time I could not let it go, and I figured out how to do it both reliably and efficiently. PR forthcoming.