-
Couldn't load subscription status.
- Fork 450
Show errors when Bash EXIT trap fires without ERR #82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Closes #72 and #81. Both of those issues deal with the `bats_error_trap` executing with an incorrect value for `$?`. Bats exited with an error, but provided no stack trace information to pinpoint its source. * #72 deals with the inconsistent failure-mode behavior of the `source` builtin between Bash versions. I believe it's feasible to update `load` with robust error checking around `source` and strongly recommend its use in Bats test files instead of using `source` directly. I've already opened #80 to track this. * #81 deals with inconsistent failure-mode behavior of `set -u`, particularly when code within a Bats test file itself references an unbound variable. With credit to sstephenson, Bats was smart enough to know and to report that an error occurred, even when it couldn't tell exactly where the error came from. Building on that existing mechanism, this change produces output for these failure cases even when `bats_error_trap` isn't called with the correct value for `$?`. It passes all the existing tests under Bash 3.2.57(1)-release and 4.4.19(1)-release. I also timed the original suites with and without the change to ensure the runtime cost was negligible. A couple more notes: * This change obsoletes the suggestion in #81 that a test case to validate testing code running under `set -u` is necessary. `run()` disables `set -e` for the code under test to begin with, sidestepping the problem with `set -eu` interactions in Bash versions prior to 4.1.0. * I've noticed that this mechanism does _not_ work when the problematic behavior is in `teardown()`. The change preserves existing behavior, the remedy for that issue requires further thinking (and a new issue).
This enables Bats to report errors in `teardown()` due to one of its commands triggering the EXIT trap without setting `$?` properly. This is accomplished by calling `bats_teardown_trap` manually whenever the test case (including `setup()`) runs to completion, i.e. whenever the test case doesn't trigger an error that triggers the EXIT trap. This will break down when both the test case (including `setup()`) _and_ `teardown()` trigger errors that both trigger the EXIT trap, because a running EXIT trap cannot trigger another EXIT trap by design. In this case, `bats_exit_trap` will not get called, truncating the test output.
|
Just pushed a new commit that will report similar errors in |
This is actually a more reliable way to check for an error than checking the value of `$?`, since `$?` can sometimes be zero even after an error occurred. The updated test cases failed before the code change was applied, and pass after. Also updated the comment to try to be clearer about the fact that the test will generate no failure output when an EXIT trigger happens while `bats_teardown_trap` executes as the EXIT trap.
6132424 to
04ac77e
Compare
Using `"${status:-1}"` was left over from before I realized that
`$status` is always defined, even when it's zero, in which case the
expression would resolve to zero.
|
@jasonkarns @sublimino and @btamayo Life pulled me away from this and other projects for a while, but it seems I've regained the bandwidth to start incrementally moving things forward again. If at least one of y'all could review this, I'd appreciate it. I think it can safely sneak in before our "release"—which I'd also like to finally get across the finish line soon. |
|
Gang, I'm gonna go ahead and merge this. If there's any issues after-the-fact, I'm happy to file a new issue and address them later. |
Closes #72 and #81. Both of those issues deal with the
bats_error_trapexecuting with an incorrect value for$?. Bats exited with an error, but provided no stack trace information to pinpoint its source.Failure in setup function causes test to fail without displaying errors #72 deals with the inconsistent failure-mode behavior of the
sourcebuiltin between Bash versions. I believe it's feasible to updateloadwith robust error checking aroundsourceand strongly recommend its use in Bats test files instead of usingsourcedirectly. I've already opened Work aroundsourcebug when run underset -ein Bash 3.2.57(1)-release #80 to track this.silent failure when accessing unbound variable with set -u #81 deals with inconsistent failure-mode behavior of
set -u, particularly when code within a Bats test file itself references an unbound variable.With credit to sstephenson, Bats was smart enough to know and to report that an error occurred, even when it couldn't tell exactly where the error came from.
Building on that existing mechanism, this change produces output for these failure cases even when
bats_error_trapisn't called with the correct value for$?. It passes all the existing tests under Bash 3.2.57(1)-release and 4.4.19(1)-release. I also timed the original suites with and without the change to ensure the runtime cost was negligible.A couple more notes:
This change obsoletes the suggestion in silent failure when accessing unbound variable with set -u #81 that a test case to validate testing code running under
set -uis necessary.run()disablesset -efor the code under test to begin with, sidestepping the problem withset -euinteractions in Bash versions prior to 4.1.0.I've noticed that this mechanism does not work when the problematic behavior is in
teardown(). The change preserves existing behavior, the remedy for that issue requires further thinking (and a new issue).cc: @cpmills1975 @tterranigma @eedwards-sk