Description
I'd like to find a way to take a variable set of test data and run it trough a test scenario with individual test setup & teardown invocations.
I have a Variables file that lists a number of scenario parameters. I want to have robot run a Test Template with all of the scenarios defined in the file. Optionally skipping some depending on some filters, but I'll leave that out of the example below for complexity's sake.
I found the Templates with FOR loops ability, so I intended to used that.
In "Data driven style", it mentions:
The above example has six separate tests, one for each invalid user/password combination, and the example below illustrates how to have only one test with all the combinations. When using test templates, all the rounds in a test are executed even if there are failures, so there is no real functional difference between these two styles. In the above example separate combinations are named so it is easier to see what they test, but having potentially large number of these tests may mess-up statistics. Which style to use depends on the context and personal preferences.
Turns out there is an important functional difference for me:
In the first example, the Test Setup/Teardown are run for every single line, while for the second example they are only ran once.
This might be implied with the 'six separate tests' vs 'only one test', but it might be worth mentioning explicitly.
I tried to solve this by adding a [Setup]
and [Teardown]
to my Test Scenario keyword, you can only have [Teardown]
, so this fails.
My current workaround is to just call the setup keyword as the first thing in my scenario. This is not an ideal solution for me because we tend to assign different meanings to fails during a setup vs fails during a test body. (Infrastructure failure vs DUT failure, leading to a different team being assigned to the initial investigation.)
I see a few ways to make this work, but there might be others:
- Adding
[Setup]
to user keywords. This seems like it should be straightforward, considering[Teardown]
already exists. This will probably also be useful in other scenarios. - Adding something like a
[Template Setup]
/[Template Teardown]
to templated test cases. - Changing the semantics of a test case with
[Template]
so that it runs the Setup/Teardown for every invocation. This will likely break test cases that rely on the current behaviour, so is probably a bad idea.
EDIT: The difference between 1 test vs many tests is a particular pitfall with libraries that have different scopes. If you change from many tests to a FOR loop within 1 test, it messes up the scope resetting of libraries, making my workaround useless without extra logic.