Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

carlfriedrich
Copy link
Collaborator

This PR is an attempt to close #363. It adds the bash unit test framework bashunit to the CI pipeline, along with a first simple test file.

Some background information: for another project I had to evaluate bash unit test frameworks (which was easier than I thought, as someone already created a really good in-depth comparison here, which I took as a starting point). bashunit was the clear winner. It is feature-rich and yet simple, user-friendly and has a really good documentation.

Check list

  • I have performed a self-review of my code
  • I have commented my code in hard-to-understand areas
  • I have made corresponding changes to the documentation

Description

Type of change

  • Bug fix
  • New feature
  • Refactor
  • Breaking change
  • Test
  • Documentation change

Test environment

  • Shell
    • bash
    • zsh
    • fish
  • OS
    • Linux
    • Mac OS X
    • Windows
    • Others:

Copy link
Collaborator

@cjappl cjappl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

@carlfriedrich
Copy link
Collaborator Author

@cjappl @sandr01d @wfxr Looking forward to your opinions on this.

question: I updated the PR template to include a checkbox for having implemented test cases. Should we add some documentation about how to do this somewhere? If so, where would be a good place for this? We do not have a "Contributing" section or file anywhere, yet, but this might seem like the place for stuff like this. WDYT?

@sandr01d
Copy link
Collaborator

sandr01d commented Sep 2, 2025

Thanks for looking into this @carlfriedrich!

bashunit looks promising, the only downsides I see with it is that it does not seem to be as widely used as other testing frameworks and doesn't have any distribution packages.

I think that the main benefit we could gain from unit tests is to make sure that commands that are processing input (mostly output from git and file names using commands like sed) work across platforms and continue to do so. I think it should be possible to achieve this with the mock feature you already use (basically simulate fzf usage by printing out the n-th line of the input). I'd like to have at least one such test implemented before moving on, so we can see if bashunit is up for the task. Would you be fine with implementing such a test?

@carlfriedrich
Copy link
Collaborator Author

@sandr01d Thanks for your feedback.

bashunit looks promising, the only downsides I see with it is that it does not seem to be as widely used as other testing frameworks and doesn't have any distribution packages.

I don't see that as an actual downside, as long as the product is robust and the project is well maintained (which both is the case IMO).

I think that the main benefit we could gain from unit tests is to make sure that commands that are processing input (mostly output from git and file names using commands like sed) work across platforms and continue to do so.

I agree and have implemented a test case for one of these filename processing functions. We do not need any mocking at all for this kind of test, as long as the code to be tested is a self-contained function with defined inputs and outputs.

We already have extracted lots of code into this kind of small helper functions, and we should continue to do so in order to make them unit-testable. During implementation of this specific test case I actually stumbled upon some more code that could be extracted, so I did so and pushed that in a second commit. The good thing: since this specific code is covered by a unit test now, I am not afraid at all that this refactoring might break anything. :-)

Let me know if this example test is convincing enough for you in order to move forward with bashunit. If you have already experience with another bash testing framework, though, I would like to hear your opinion on that and what might be better than with bashunit.

@carlfriedrich
Copy link
Collaborator Author

carlfriedrich commented Sep 4, 2025

One more thing to notice: I had to mock fzf to provide a valid version number before sourcing git-forgit now, since the version check is still executed when sourced. In my opinion we could move this code to the end of the script, below the new check whether the script is sourced, then we do not need the mocking.

Even better IMO would be a main function which encapsulates all the code that is executed when git-forgit is run (and not sourced), so that we do not have any code executed on the top level of the script, except for the source check and a call to main. This might be beyond the scope of this PR, though, so I would suggest to do this in a follow-up. Let me know what you think.

Copy link
Collaborator

@sandr01d sandr01d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very neat!

question: I updated the PR template to include a checkbox for having implemented test cases. Should we add some documentation about how to do this somewhere? If so, where would be a good place for this? We do not have a "Contributing" section or file anywhere, yet, but this might seem like the place for stuff like this. WDYT?

Forgot to reply to this earlier. Yes, good idea. Adding a CONTRIBUTING.md file would be best IMO.

We already have extracted lots of code into this kind of small helper functions, and we should continue to do so in order to make them unit-testable.

Yes, absolutely 🚀

Let me know if this example test is convincing enough for you in order to move forward with bashunit. If you have already experience with another bash testing framework, though, I would like to hear your opinion on that and what might be better than with bashunit.

You convinced me. I looked into bats-core previously, but it falls short to bashunit IMO, especially in regards to those really handy mocking features.

Once the (very minor) changes I've proposed is addressed I think we could go ahead and merge this PR. It's probably best to introduce the rest of the unit tests in several smaller PRs. WDYT?

@carlfriedrich
Copy link
Collaborator Author

One more thing to notice: I had to mock fzf to provide a valid version number before sourcing git-forgit now, since the version check is still executed when sourced. In my opinion we could move this code to the end of the script, below the new check whether the script is sourced, then we do not need the mocking.

Even better IMO would be a main function which encapsulates all the code that is executed when git-forgit is run (and not sourced), so that we do not have any code executed on the top level of the script, except for the source check and a call to main. This might be beyond the scope of this PR, though, so I would suggest to do this in a follow-up. Let me know what you think.

I decided that it's better to do this upfront and added a dedicated refactoring commit before the test implementation. This made the fzf mock just to pass the version check obsolete.

Forgot to reply to this earlier. Yes, good idea. Adding a CONTRIBUTING.md file would be best IMO.

I have opened an issue for that: #466

You convinced me. I looked into bats-core previously, but it falls short to bashunit IMO, especially in regards to those really handy mocking features.

Thanks for your feedback. Just for completeness, I also evaluated bats and - while it seems to be the most popular bash testing framework - I saw these disadvantages:

  • Much bigger distribution (several folders and files) than bashunit (just a single self-contained bash script)
  • No assert functions available in bats-core, only via extension (so you have to make the distribution even bigger to work with it)
  • Asserts are not intuitivley named (at least for my liking) and some have to be called with arguments. Makes the test code way harder to read than bashunit tests.
  • Test cases are not pure bash syntax, file extension is .bats which makes shellcheck integration difficult
  • No explicit mock functionality
  • Scripts containing set -e may behave differently during unit test compared to out of the test. We don't have set -e in forgit, but maybe we will in the future, and that sounds like a really bad potential gotcha.

In contrast I did not find any disadvantage in bashunit (except being much less popular, but as I already stated I don't think that's a valid argument).

Once the (very minor) changes I've proposed is addressed I think we could go ahead and merge this PR. It's probably best to introduce the rest of the unit tests in several smaller PRs. WDYT?

Yes, this PR was intended to only set up unit testing initially. Test cases should be gradually added, at best in conjunction with production code changes because that's when we're manually testing anyway.

This is a prequisite for being able to source the script in order to
implement unit tests for individual functions.
In order to run unit tests for individual functions we have to be able
to source the script without triggering the main execution flow.
carlfriedrich added a commit to carlfriedrich/forgit that referenced this pull request Sep 6, 2025
…xr#464)

Update CI to install latest bashunit beta version instead of fixed
0.23.0 version because we need the new "data_set" function for
parameterized tests.
carlfriedrich added a commit to carlfriedrich/forgit that referenced this pull request Sep 6, 2025
Copy link
Collaborator

@sandr01d sandr01d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say that this is ready to be merged. Thank you @carlfriedrich!

…xr#464)

Update CI to install latest bashunit beta version instead of fixed
0.23.0 version because we need the new "data_set" function for
parameterized tests.
@carlfriedrich carlfriedrich merged commit 2a4e2e5 into wfxr:main Sep 8, 2025
3 checks passed
carlfriedrich added a commit that referenced this pull request Sep 8, 2025
This is a prequisite for being able to source the script in order to
implement unit tests for individual functions.
carlfriedrich added a commit that referenced this pull request Sep 8, 2025
In order to run unit tests for individual functions we have to be able
to source the script without triggering the main execution flow.
carlfriedrich added a commit that referenced this pull request Sep 8, 2025
Update CI to install latest bashunit beta version instead of fixed
0.23.0 version because we need the new "data_set" function for
parameterized tests.
@carlfriedrich carlfriedrich deleted the add-bashunit-test branch September 8, 2025 04:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add unit testing framework
3 participants