Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Shortcutting #523

@wlandau

Description

@wlandau

Prework

  • Read and agree to the code of conduct and contributing guidelines.
  • If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
  • New features take time and effort to create, and they take even more effort to maintain. So if the purpose of the feature is to resolve a struggle you are encountering personally, please consider first posting a "trouble" or "other" issue so we can discuss your use case and search for existing solutions first.
  • Format your code according to the tidyverse style guide.

Background

Several people have requested this feature, most recently @jennysjaarda in #522, and also @kendonB and @MilesMcBain some time ago.

targets normally traverses the entire pipeline and checks every single target even when you know in advance everything should be up to date. As @jennysjaarda pointed out, skipping lots of targets adds overhead to initialization times. In the example below, it takes a long time to get to the finalize target. If you iterate a lot on finalize, you still have to skip branches, and the delay due to checking targets is annoying.

# _targets.R
library(targets)
list(
  tar_target(index, seq_len(1000)),
  tar_target(branches, index, pattern = map(index)),
  tar_target(finalize, sum(branches))
)
# R console
system.time(tar_make())
#> ✓ skip target index
#> ✓ skip branch branches_77ba9815
#> ✓ skip branch branches_4b7abfa4
#> ✓ skip branch branches_f5a3cbc2
#> ✓ skip branch branches_b0df9eb8
#> ...
#>    user  system elapsed 
#>   5.114   0.650   5.134 

Proposal

I propose a shortcut argument to tar_make() and related functions so that we can jump right to the targets you mention in names. We bootstrap upstream dependencies from the metadata so required data can be loaded if needed.

> system.time(tar_make(names = all_of("finalize"), shortcut = TRUE))
✓ skip target finalizeskip pipeline
#>   user  system elapsed 
#>  0.984   0.133   1.196 # Some overhead due to callr cannot be helped.

For the longest time, I thought this feature would break everything. But this time I could not let it go, and I figured out how to do it both reliably and efficiently. PR forthcoming.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions