-
Notifications
You must be signed in to change notification settings - Fork 150
Meta: Add bashunit tests to CI workflow and implement a first test case #464
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome!
@cjappl @sandr01d @wfxr Looking forward to your opinions on this. question: I updated the PR template to include a checkbox for having implemented test cases. Should we add some documentation about how to do this somewhere? If so, where would be a good place for this? We do not have a "Contributing" section or file anywhere, yet, but this might seem like the place for stuff like this. WDYT? |
Thanks for looking into this @carlfriedrich! bashunit looks promising, the only downsides I see with it is that it does not seem to be as widely used as other testing frameworks and doesn't have any distribution packages. I think that the main benefit we could gain from unit tests is to make sure that commands that are processing input (mostly output from |
4cd1cef
to
a1db28a
Compare
@sandr01d Thanks for your feedback.
I don't see that as an actual downside, as long as the product is robust and the project is well maintained (which both is the case IMO).
I agree and have implemented a test case for one of these filename processing functions. We do not need any mocking at all for this kind of test, as long as the code to be tested is a self-contained function with defined inputs and outputs. We already have extracted lots of code into this kind of small helper functions, and we should continue to do so in order to make them unit-testable. During implementation of this specific test case I actually stumbled upon some more code that could be extracted, so I did so and pushed that in a second commit. The good thing: since this specific code is covered by a unit test now, I am not afraid at all that this refactoring might break anything. :-) Let me know if this example test is convincing enough for you in order to move forward with bashunit. If you have already experience with another bash testing framework, though, I would like to hear your opinion on that and what might be better than with bashunit. |
One more thing to notice: I had to mock Even better IMO would be a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks very neat!
question: I updated the PR template to include a checkbox for having implemented test cases. Should we add some documentation about how to do this somewhere? If so, where would be a good place for this? We do not have a "Contributing" section or file anywhere, yet, but this might seem like the place for stuff like this. WDYT?
Forgot to reply to this earlier. Yes, good idea. Adding a CONTRIBUTING.md file would be best IMO.
We already have extracted lots of code into this kind of small helper functions, and we should continue to do so in order to make them unit-testable.
Yes, absolutely 🚀
Let me know if this example test is convincing enough for you in order to move forward with bashunit. If you have already experience with another bash testing framework, though, I would like to hear your opinion on that and what might be better than with bashunit.
You convinced me. I looked into bats-core previously, but it falls short to bashunit IMO, especially in regards to those really handy mocking features.
Once the (very minor) changes I've proposed is addressed I think we could go ahead and merge this PR. It's probably best to introduce the rest of the unit tests in several smaller PRs. WDYT?
a1db28a
to
981b471
Compare
4b055a7
to
60dcafb
Compare
I decided that it's better to do this upfront and added a dedicated refactoring commit before the test implementation. This made the
I have opened an issue for that: #466
Thanks for your feedback. Just for completeness, I also evaluated
In contrast I did not find any disadvantage in bashunit (except being much less popular, but as I already stated I don't think that's a valid argument).
Yes, this PR was intended to only set up unit testing initially. Test cases should be gradually added, at best in conjunction with production code changes because that's when we're manually testing anyway. |
This is a prequisite for being able to source the script in order to implement unit tests for individual functions.
In order to run unit tests for individual functions we have to be able to source the script without triggering the main execution flow.
…xr#464) Update CI to install latest bashunit beta version instead of fixed 0.23.0 version because we need the new "data_set" function for parameterized tests.
60dcafb
to
b8f7371
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd say that this is ready to be merged. Thank you @carlfriedrich!
…xr#464) Update CI to install latest bashunit beta version instead of fixed 0.23.0 version because we need the new "data_set" function for parameterized tests.
b8f7371
to
d646d39
Compare
This is a prequisite for being able to source the script in order to implement unit tests for individual functions.
In order to run unit tests for individual functions we have to be able to source the script without triggering the main execution flow.
Update CI to install latest bashunit beta version instead of fixed 0.23.0 version because we need the new "data_set" function for parameterized tests.
This PR is an attempt to close #363. It adds the bash unit test framework
bashunit
to the CI pipeline, along with a first simple test file.Some background information: for another project I had to evaluate bash unit test frameworks (which was easier than I thought, as someone already created a really good in-depth comparison here, which I took as a starting point).
bashunit
was the clear winner. It is feature-rich and yet simple, user-friendly and has a really good documentation.Check list
Description
Type of change
Test environment