Unbreak test suites with multiple files loading common constants #904

debarshiray · 2024-04-19T18:51:24Z

Test suites with a helper file with common constants that is loaded by
multiple files stopped working with:

  $ bats /some/path
  /some/path/test_helper.bash: line 1: A_CONSTANT: readonly variable
  Error while sourcing library loader at '/some/path/test_helper.bash'

If the helper file had anything else other than the common constants,
then the tests would run, but the spew about the constants would still
remain:

  $ bats --tap /some/path
  /some/path/test_helper.bash: line 1: A_CONSTANT: readonly variable
  1..2
  ok 1 first
  ok 2 second

Fallout from 9d5ecdb

I have reviewed the Contributor Guidelines.
I have reviewed the Code of Conduct and agree to abide by it

Some editors like GNU Emacs insist on adding the newline at end of file, if it's missing, which leads to noise when looking at the actual code changes. bats-core#904

Test suites with a helper file with common constants that is loaded by multiple files stopped working with: $ bats /some/path /some/path/test_helper.bash: line 1: A_CONSTANT: readonly variable Error while sourcing library loader at '/some/path/test_helper.bash' If the helper file had anything else other than the common constants, then the tests would run, but the spew about the constants would still remain: $ bats --tap /some/path /some/path/test_helper.bash: line 1: A_CONSTANT: readonly variable 1..2 ok 1 first ok 2 second Fallout from 9d5ecdb bats-core#904

bats-core#904

debarshiray · 2024-04-19T18:53:08Z

This goes on top of #902

martin-schulze-vireso

Thanks for your report and suggestion. I believe that we should not ignore load errors during the test registration, that will only lead to hard to diagnose problems.

IMHO, the correct way forward should be to separate the evaluationen of individual test files during test registration in the same way as is done for test execution.

debarshiray · 2024-04-21T17:29:00Z

Thanks for your report and suggestion. I believe that we should not ignore load errors during the test registration, that will only lead to hard to diagnose problems.

Yeah, I can imagine.

IMHO, the correct way forward should be to separate the evaluationen of individual test files during test registration in the same way as is done for test execution.

I don't understand the Bats code well enough to know how exactly that should be done. I can try it, if you give me some pointers, or I don't mind if you do it yourself and I hope that the tests would still be useful in that case. :)

debarshiray · 2024-04-21T17:29:12Z

Thanks for the review!

Test suites with a helper file with common constants that is loaded by multiple files stopped working with: $ bats /some/path /some/path/test_helper.bash: line 1: A_CONSTANT: readonly variable Error while sourcing library loader at '/some/path/test_helper.bash' If the helper file had anything else other than the common constants, then the tests would run, but the spew about the constants would still remain: $ bats --tap /some/path /some/path/test_helper.bash: line 1: A_CONSTANT: readonly variable 1..2 ok 1 first ok 2 second Fallout from 9d5ecdb bats-core#904

bats-core#904

debarshiray · 2024-04-22T16:11:14Z

Tried to fix the changelog and shellcheck tests.

Ansible's built-in 'package' module doesn't show any details when installing the RPMs. All that can be seen is: TASK [Install RPM packages] fedora-rawhide | changed Therefore, there's no way to know what version of the packages got installed. In this case, not knowing the Bats version being used by the CI makes it difficult to know why the tests are generating this spew on Fedora Rawhide [1]: TASK [Run system tests] test/system/libs/helpers.bash: line 7: TEMP_BASE_DIR: readonly variable test/system/libs/helpers.bash: line 8: TEMP_STORAGE_DIR: readonly variable test/system/libs/helpers.bash: line 10: IMAGE_CACHE_DIR: readonly variable test/system/libs/helpers.bash: line 11: ROOTLESS_PODMAN_STORE_DIR: readonly variable test/system/libs/helpers.bash: line 12: ROOTLESS_PODMAN_RUNROOT_DIR: readonly variable test/system/libs/helpers.bash: line 13: PODMAN_STORE_CONFIG_FILE: readonly variable test/system/libs/helpers.bash: line 14: DOCKER_REG_ROOT: readonly variable test/system/libs/helpers.bash: line 15: DOCKER_REG_CERTS_DIR: readonly variable test/system/libs/helpers.bash: line 16: DOCKER_REG_AUTH_DIR: readonly variable test/system/libs/helpers.bash: line 17: DOCKER_REG_URI: readonly variable test/system/libs/helpers.bash: line 18: DOCKER_REG_NAME: readonly variable test/system/libs/helpers.bash: line 21: PODMAN: readonly variable test/system/libs/helpers.bash: line 22: TOOLBX: readonly variable test/system/libs/helpers.bash: line 23: SKOPEO: readonly variable ... fedora-rawhide | 1..340 [1] bats-core/bats-core#904

Ansible's built-in 'package' module doesn't show any details when installing the RPMs. All that can be seen is: TASK [Install RPM packages] fedora-rawhide | changed Therefore, there's no way to know what version of the packages got installed. In this case, not knowing the Bats version being used by the CI makes it difficult to know why the tests are generating this spew on Fedora Rawhide [1]: TASK [Run system tests] test/system/libs/helpers.bash: line 7: TEMP_BASE_DIR: readonly variable test/system/libs/helpers.bash: line 8: TEMP_STORAGE_DIR: readonly variable test/system/libs/helpers.bash: line 10: IMAGE_CACHE_DIR: readonly variable test/system/libs/helpers.bash: line 11: ROOTLESS_PODMAN_STORE_DIR: readonly variable test/system/libs/helpers.bash: line 12: ROOTLESS_PODMAN_RUNROOT_DIR: readonly variable test/system/libs/helpers.bash: line 13: PODMAN_STORE_CONFIG_FILE: readonly variable test/system/libs/helpers.bash: line 14: DOCKER_REG_ROOT: readonly variable test/system/libs/helpers.bash: line 15: DOCKER_REG_CERTS_DIR: readonly variable test/system/libs/helpers.bash: line 16: DOCKER_REG_AUTH_DIR: readonly variable test/system/libs/helpers.bash: line 17: DOCKER_REG_URI: readonly variable test/system/libs/helpers.bash: line 18: DOCKER_REG_NAME: readonly variable test/system/libs/helpers.bash: line 21: PODMAN: readonly variable test/system/libs/helpers.bash: line 22: TOOLBX: readonly variable test/system/libs/helpers.bash: line 23: SKOPEO: readonly variable ... fedora-rawhide | 1..340 [1] bats-core/bats-core#904 containers#1482

martin-schulze-vireso · 2024-05-04T23:37:29Z

I underestimated the complexity. The code for counting tests and duplicates relied on having everything evaluated in the same shell. This will need a rework but I am out of time for today.

debarshiray · 2024-05-06T15:27:39Z

I underestimated the complexity. The code for counting tests and duplicates relied on having everything evaluated in the same shell. This will need a rework but I am out of time for today.

Thanks for working on this, @martin-schulze-vireso !

Some editors like GNU Emacs insist on adding the newline at end of file, if it's missing, which leads to noise when looking at the actual code changes. bats-core#904

Test suites with a helper file with common constants that is loaded by multiple files stopped working with: $ bats /some/path /some/path/test_helper.bash: line 1: A_CONSTANT: readonly variable Error while sourcing library loader at '/some/path/test_helper.bash' If the helper file had anything else other than the common constants, then the tests would run, but the spew about the constants would still remain: $ bats --tap /some/path /some/path/test_helper.bash: line 1: A_CONSTANT: readonly variable 1..2 ok 1 first ok 2 second Fallout from 9d5ecdb bats-core#904

bats-core#904

… interactions

Failures are due to scoping issues in traps that only happen when $() is used: bats-gather-tests:280, start of bats_gather_tests_source_exit_trap: local: can only be used in a function tracing.bash:301, end of bats_debug_trap: pop_scope: head of shell_variables not a temporary environment scope

Windows and Linux go a newer version ( v18.20.3) with npm 10.7 (instead of 10.5)

martin-schulze-vireso · 2024-05-31T05:35:41Z

Thanks for your contribution.

… with Bats 1.10.0 Fixed in latest upstream version by bats-core/bats-core#904 This reverts commit 2037f85.

debarshiray requested a review from a team as a code owner April 19, 2024 18:51

debarshiray added a commit to debarshiray/bats-core that referenced this pull request Apr 19, 2024

Test test suites with multiple files loading common constants

745b619

bats-core#904

debarshiray force-pushed the wip/rishi/unbreak-suite-load-readonly branch from b94660b to 745b619 Compare April 19, 2024 18:52

debarshiray mentioned this pull request Apr 19, 2024

dynamic test registration #349

Merged

2 tasks

martin-schulze-vireso requested changes Apr 20, 2024

View reviewed changes

debarshiray added a commit to debarshiray/bats-core that referenced this pull request Apr 22, 2024

Test test suites with multiple files loading common constants

f94e695

bats-core#904

debarshiray force-pushed the wip/rishi/unbreak-suite-load-readonly branch from 745b619 to f94e695 Compare April 22, 2024 16:10

debarshiray mentioned this pull request Apr 24, 2024

.zuul: Enable testing on Fedora 40 containers/toolbox#1468

Merged

debarshiray mentioned this pull request Apr 30, 2024

playbooks: Show the Bats version containers/toolbox#1482

Merged

martin-schulze-vireso force-pushed the wip/rishi/unbreak-suite-load-readonly branch 2 times, most recently from 23f55be to 85f73c2 Compare May 6, 2024 22:50

debarshiray and others added 6 commits May 29, 2024 22:13

Add newline at end of file

1061858

Some editors like GNU Emacs insist on adding the newline at end of file, if it's missing, which leads to noise when looking at the actual code changes. bats-core#904

Test test suites with multiple files loading common constants

f94763e

bats-core#904

Evaluate each test file in own shell during gather-tests to avoid bad…

5c52dc3

… interactions

Simplify tracing code

34833b5

Adapt to differing source error behavior between bash versions

c1cc818

martin-schulze-vireso force-pushed the wip/rishi/unbreak-suite-load-readonly branch from 240f074 to c1cc818 Compare May 29, 2024 20:15

Centrallize RELATIVE_BATS_ROOT in test_helper.bash fixture()

4ac00c5

martin-schulze-vireso added 2 commits May 30, 2024 22:09

Reduce noise in tests

e3266cd

martin-schulze-vireso force-pushed the wip/rishi/unbreak-suite-load-readonly branch 3 times, most recently from 71820cc to d3b2e27 Compare May 30, 2024 20:52

CI: Split npm commands

f27d259

martin-schulze-vireso force-pushed the wip/rishi/unbreak-suite-load-readonly branch from d3b2e27 to ec885f1 Compare May 30, 2024 20:58

CI: Fix bats not found when installed via npm

e406c01

Windows and Linux go a newer version ( v18.20.3) with npm 10.7 (instead of 10.5)

martin-schulze-vireso force-pushed the wip/rishi/unbreak-suite-load-readonly branch from ec885f1 to e406c01 Compare May 31, 2024 05:00

martin-schulze-vireso approved these changes May 31, 2024

View reviewed changes

martin-schulze-vireso merged commit 0ae17b2 into bats-core:master May 31, 2024

debarshiray deleted the wip/rishi/unbreak-suite-load-readonly branch June 5, 2024 12:18

inkarkat added a commit to inkarkat/executionMarker that referenced this pull request Jul 31, 2024

Revert: TESTSFIX: Adapt: Cannot define readonly test-scoped variables…

0dd8c8b

… with Bats 1.10.0 Fixed in latest upstream version by bats-core/bats-core#904 This reverts commit 2037f85.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Unbreak test suites with multiple files loading common constants #904

Unbreak test suites with multiple files loading common constants #904

Uh oh!

debarshiray commented Apr 19, 2024

Uh oh!

debarshiray commented Apr 19, 2024

Uh oh!

martin-schulze-vireso left a comment

Uh oh!

debarshiray commented Apr 21, 2024

Uh oh!

debarshiray commented Apr 21, 2024

Uh oh!

debarshiray commented Apr 22, 2024

Uh oh!

martin-schulze-vireso commented May 4, 2024

Uh oh!

debarshiray commented May 6, 2024

Uh oh!

martin-schulze-vireso commented May 31, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Unbreak test suites with multiple files loading common constants #904

Unbreak test suites with multiple files loading common constants #904

Uh oh!

Conversation

debarshiray commented Apr 19, 2024

Uh oh!

debarshiray commented Apr 19, 2024

Uh oh!

martin-schulze-vireso left a comment

Choose a reason for hiding this comment

Uh oh!

debarshiray commented Apr 21, 2024

Uh oh!

debarshiray commented Apr 21, 2024

Uh oh!

debarshiray commented Apr 22, 2024

Uh oh!

martin-schulze-vireso commented May 4, 2024

Uh oh!

debarshiray commented May 6, 2024

Uh oh!

martin-schulze-vireso commented May 31, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants