Prework

• Read and agree to the code of conduct and contributing guidelines.
• Confirm that your issue is most likely a genuine bug in the targets package itself and not a user error, a known limitation, or a bug in another package that targets depends on. For miscellaneous troubleshooting, please post to the discussions instead.
• If there is already a relevant issue, whether open or closed, comment on the existing thread instead of posting a new issue.
• Post a minimal reproducible example like this one so the maintainer can troubleshoot the problems you identify. A reproducible example is:
  • Runnable: post enough R code and data so any onlooker can create the error on their own computer.
  • Minimal: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand.
  • Readable: format your code according to the tidyverse style guide.

Description

On Windows, if you call tar_make(callr_function = callr::r_bg) and then call tar_poll(), both processes will fight over _targets/meta/progress and the pipeline will lose out.

Error in file(file, ifelse(append, "a", "w")) : 
  cannot open the connection
In addition: Warning message:
In file(file, ifelse(append, "a", "w")) :
  cannot open file '_targets/meta/progress': Permission denied

It is a tricky race condition. I think the only way around it is for the pipeline to keep trying to write that line until it does not hit an error. Partial lines are unlikely because the error happens while opening the connection, and duplicate lines are okay because targets just keeps the most recent one if there are duplicates.

The upcoming patch will also apply to the metadata automatically, which should catch the rare race condition in tar_visnetwork() etc. if it happens. (I did not notice because I rarely use Windows.)

Reproducible example

At first, you will see the pipeline start okay. Then, the "since" column in tar_poll() will increase without bound even though there are targets remaining, which indicates that the pipeline stopped too soon. The error log file will show the error mentioned above.

library(targets)
tar_script(
  list(
    tar_target(n_random, rep(1, 1500)),
    tar_target(random, Sys.sleep(0.1), pattern = map(n_random))
  )
)
px <- tar_make(
  callr_function = callr::r_bg,
  callr_arguments = list(
    stdout = "C:/Users/landau/Desktop/out.txt",
    stderr = "C:/Users/landau/Desktop/err.txt"
  )
)
tar_poll()

• Post a minimal reproducible example so the maintainer can troubleshoot the problems you identify. A reproducible example is:
  • Runnable: post enough R code and data so any onlooker can create the error on their own computer.
  • Minimal: reduce runtime wherever possible and remove complicated details that are irrelevant to the issue at hand.
  • Readable: format your code according to the tidyverse style guide.

Expected result

What should have happened? Please be as specific as possible.

Diagnostic information

• A reproducible example.
• Session info, available through sessionInfo() or reprex(si = TRUE).
• A stack trace from traceback() or rlang::trace_back().
• The SHA-1 hash of the GitHub commit of targets currently installed. packageDescription("targets")$GithubSHA1 shows you this.