Describe the bug
Disclaimer: I'm choosing to file this as "bug" because it (subjectively) fails "principle of least surprise" and I was unable to find documentation to explain (or hint at) the behavior.

I am simply trying to capture both test results and the outputs together. My desire was to capture both in sibling directories (i.e., a common parent). With no (obvious) documentation stating otherwise, I guessed/assumed creating the common parent directory would be sufficient and the two children would be automatically created. That resulted in an error. I then proceeded to pre-create both children directories. That also resulted in an error.

To Reproduce
Steps to reproduce the behavior:

I thought it might be clearer to assert the undesirable behavior and allow the test names to tell the story. So in this case, a passing test is actually a negative thing 😄 .

Contents of wut.bats:

#!/usr/bin/env bats

setup() {
  export DUMMY_TEST="$BATS_TEST_TMPDIR/test.bats"
  export STUFF_ROOT="$BATS_TEST_TMPDIR/stuff"
  export REPORT_DIR="$STUFF_ROOT/report"
  export OUTPUT_DIR="$STUFF_ROOT/output"

  cat <<'EOT' | sed 's/^ATtest/@test/' > "$DUMMY_TEST"
ATtest "simple" {
  echo "hi"
}
EOT
  mkdir "$STUFF_ROOT"
}

@test "no pre-created dirs fails (reasonable)" {
  run bats --report-formatter junit --output "$REPORT_DIR" --gather-test-outputs-in "$OUTPUT_DIR" "$DUMMY_TEST"
  [ "$status" -ne 0 ]
  echo "$output" | grep "Output path $REPORT_DIR is not writeable"
}

@test "pre-created report and output dir fails (unexpected)" {
  mkdir "$OUTPUT_DIR" && mkdir "$REPORT_DIR"
  run bats --report-formatter junit --output "$REPORT_DIR" --gather-test-outputs-in "$OUTPUT_DIR" "$DUMMY_TEST"
  [ "$status" -ne 0 ]
  echo "$output" | grep "can't create directory '$OUTPUT_DIR': File exists"
}

@test "pre-created report dir (only) succeeds (inconsistent)" {
  mkdir "$REPORT_DIR"
  bats --report-formatter junit --output "$REPORT_DIR" --gather-test-outputs-in "$OUTPUT_DIR" "$DUMMY_TEST"
}

@test "pre-created output dir (only) fails (inconsistent)" {
  mkdir "$OUTPUT_DIR"
  run bats --report-formatter junit --output "$REPORT_DIR" --gather-test-outputs-in "$OUTPUT_DIR" "$DUMMY_TEST"
  [ "$status" -ne 0 ]
  echo "$output" | grep "can't create directory '$OUTPUT_DIR': File exists"
}

Incidentally, this actually shows a second pseudo-bug: using the @test marker in a heredoc still gets picked up as a test. The sed hack was to avoid getting an "expected X tests but ran Y" warning. I'm assuming this is an extremely atypical pattern, so didn't think it warranted filing an issue. But I'd be happy to file a report if you feel it should be addressed.

Result (current and previous version):

$ docker run --rm -it -v $(pwd):/code:ro  bats/bats:1.7.0 wut.bats
wut.bats
 ✓ no pre-created dirs fails (reasonable)
 ✓ pre-created report and output dir fails (unexpected)
 ✓ pre-created report dir (only) succeeds (inconsistent)
 ✓ pre-created output dir (only) fails (inconsistent)

4 tests, 0 failures

$ docker run --rm -it -v $(pwd):/code:ro  bats/bats:1.6.0 wut.bats
 ✓ no pre-created dirs fails (reasonable)
 ✓ pre-created report and output dir fails (unexpected)
 ✓ pre-created report dir (only) succeeds (inconsistent)
 ✓ pre-created output dir (only) fails (inconsistent)

4 tests, 0 failures

Expected behavior
I would expect one of the following:

1. both directories required to be already created
2. requiring that neither directory exist
3. documentation explicitly stating the prerequisites (ideally, with an explanation for the discrepancy, if it's by design)

Environment (please complete the following information):

• Bats Version: 1.6.0 and 1.7.0 (but likely every version from when those features were first introduced)
• OS: Linux (my initial encounter, as well as my reproduction, were in containerized environments)
• Bash version: 5.1.16 (version present in the bats docker image)

Additional context
I am relatively new to this tool and have yet to explore all its features. Maybe, with extended use, it would be obvious why it behaves the way it does. If I was confident that it was a bug and that there was a "correct' answer, I likely would have contributed a PR alongside this report. But I'm not, so I didn't.