Add new groups to data without invalidating downstream targets? #508

jennysjaarda · 2021-06-12T21:11:41Z

jennysjaarda
Jun 12, 2021

I was wondering what you would recommend if new groups are added to an upstream data set so that only the new groups are rerun. For example, borrowing from the example from tar_group_by, we have:

# _targets.R file:
library(targets)
library(tarchetypes)
produce_data <- function() {
  expand.grid(var1 = c("a", "b"), var2 = c("c", "d"), rep = c(1, 2, 3))
}
list(
  tar_group_by(data, produce_data(), var1, var2),
  tar_target(result, analyze_fn(data), pattern = map(data))


)

if later new groups are added to the dataset:

produce_data <- function() {
  expand.grid(var1 = c("a", "b", "c"), var2 = c("c", "d"), rep = c(1, 2, 3))
}

Now the data target is outdated (understandably) but every result branch is also rebuilt. I guess targets doesn't know the contents of the data, so what would be the best way around this case? Best to be mapping over var1 directly and have that as input into the function?

wlandau · 2021-06-13T14:50:11Z

wlandau
Jun 13, 2021
Maintainer

768afad should work. Please give it a try.

At first I thought this was just an instance of #507, then I realized expand.grid() creates an out.attrs attributes that depends on the whole data frame but gets propagated to each group/slice. 768afad removes the attribute. Another workaround would have been to install d6bac3a and use tidyr::expand_grid() instead of base::expand.grid().

1 reply

wlandau Jun 13, 2021
Maintainer

I would also set stringsAsFactors = FALSE because the adding a new slice with a new factor level could change the factor levels of the other slices and invalidate everything.

jennysjaarda · 2021-06-13T16:11:34Z

jennysjaarda
Jun 13, 2021
Author

I just tried to install and run again, but I got a strange error:

Error : 'switch_config' is not an exported object from 'namespace:targets'

2 replies

wlandau Jun 13, 2021
Maintainer

Would you post a reprex? Not sure where that might come from. Development targets handles tar_config_set() and _targets.yaml differently.

wlandau Jun 13, 2021
Maintainer

switch_config() should not be invoked at all. Maybe just restart your R session?

jennysjaarda · 2021-06-13T17:45:50Z

jennysjaarda
Jun 13, 2021
Author

Sorry yes I guess it was just something with my R session as I couldn't reproduce with reprex. I still haven't managed to solve the original question with the update though. Will try and get a reprex for that.

1 reply

wlandau Jun 14, 2021
Maintainer

The update changes how row groups are hashed, which automatically invalidates all the branches from an expand.grid() call. However, that invalidation due to the update only happens once per old project. So if expand.grid() was the only factor at play, then I think subsequent runs should skip the correct slices if previous tar_make() also used the update.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add new groups to data without invalidating downstream targets? #508

Uh oh!

{{title}}

Uh oh!

Replies: 3 comments 4 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Add new groups to data without invalidating downstream targets? #508

Uh oh!

jennysjaarda Jun 12, 2021

Replies: 3 comments · 4 replies

Uh oh!

wlandau Jun 13, 2021 Maintainer

Uh oh!

wlandau Jun 13, 2021 Maintainer

Uh oh!

jennysjaarda Jun 13, 2021 Author

Uh oh!

wlandau Jun 13, 2021 Maintainer

Uh oh!

wlandau Jun 13, 2021 Maintainer

Uh oh!

jennysjaarda Jun 13, 2021 Author

Uh oh!

wlandau Jun 14, 2021 Maintainer

jennysjaarda
Jun 12, 2021

Replies: 3 comments 4 replies

wlandau
Jun 13, 2021
Maintainer

wlandau Jun 13, 2021
Maintainer

jennysjaarda
Jun 13, 2021
Author

wlandau Jun 13, 2021
Maintainer

wlandau Jun 13, 2021
Maintainer

jennysjaarda
Jun 13, 2021
Author

wlandau Jun 14, 2021
Maintainer