dtcw Automated Tests

This directory contains automated tests for the dtcw Bash wrapper script. The tests use bats-core, a Bash automated testing system.

Documentation for bats-core can be found at https://bats-core.readthedocs.io/. Detailed documentation to the helper libraries can be found in their git repositories:

• bats-support

• bats-assert

• bats-file

• bats-mock

The test framework and the libraries are provided with git submodules. To update the content of the submodules execute from the main project directory:

➜ git submodule update --init --recursive

End-to-end test cases are tagged with e2e. Those test cases download external packages with curl or sdk and change the system. It is self-evident that those tests have a long runtime.

Warning

Tests tagged with e2e have to be executed in a container to ensure a deterministic test setup and to prevent a pollution of the system with test artifacts.

Test Execution

Warning

The test teardown routines delete $HOME/.doctoolchain and thus possible local installations. To be on the save side install a container runtime and run the tests in a container as explained in Execute End-to-end Tests.

The test suite is executed with the following command from the docToolchain project root directory,

# Run all tests except end-to-end tests tagged with 'e2e'
➜ ./test/bats/bin/bats --filter-tags \!e2e test

# Run all tests of a test suite file
➜ ./test/bats/bin/bats test/pristine_environment.bats

# Only run tests which match a regular expression
➜ ./test/bats/bin/bats -f "show version info" test

Additional information about how you can tweak test execution start the container with the --help option

➜ ./test/bats/bin/bats --help

Execute End-to-end Tests

Pre-requisites: installed container runtime (Docker or Podman).

As mentioned before, end-to-end tests have to be executed in a container image. We use the offical Debian Image to run the test suite.

Note	bats-core provides a container images which is based on musl libc. The musl C library causes problems with Java. So we use a Debian based test container with a standard glibc.

# From the docToolchain project root
➜ docker run --rm -it -v "${PWD}/dtcw:/code/dtcw" -v "${PWD}/test:/code/test" -w "/code" debian:stable-slim ./test/bats/bin/bats test

The volume bind mounts make dtcw and the test code available at /code. This setup provides a clean project setup and avoids possible pitfalls to access resources from the docToolchain project directory.

# In the container we have a pristine project which uses the docToolchain
code
├── dtcw
└── test
    ├── pristine_environment.bats
    └── README.adoc

You probably want to create an alias to run the test suite

➜ alias tdtcw='docker run --rm -it -v "${PWD}/dtcw:/code/dtcw" -v "${PWD}/test:/code/test" -w "/code" debian:stable-slim ./test/bats/bin/bats'
➜ tdtcw test

Debugging

To access the test environment in the container provide the bind mounts without executing any command.

➜ docker run --rm -it -v "${PWD}/dtcw:/code/dtcw" -v "${PWD}/test:/code/test" -w "/code" debian:stable-slim
bash-5.2#

TODO

Increase Test Coverage

Test Code

• Skip e2e tests if they are not executed in a container.

dtcw

• Provide detailed information which Java is downloaded/installed.

• Can we make Gradle less noisy by default?

• Provide --verbose flag