You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'd like to find a way to take a variable set of test data and run it trough a test scenario with individual test setup & teardown invocations.
I have a Variables file that lists a number of scenario parameters. I want to have robot run a Test Template with all of the scenarios defined in the file. Optionally skipping some depending on some filters, but I'll leave that out of the example below for complexity's sake.
The above example has six separate tests, one for each invalid user/password combination, and the example below illustrates how to have only one test with all the combinations. When using test templates, all the rounds in a test are executed even if there are failures, so there is no real functional difference between these two styles. In the above example separate combinations are named so it is easier to see what they test, but having potentially large number of these tests may mess-up statistics. Which style to use depends on the context and personal preferences.
Turns out there is an important functional difference for me:
In the first example, the Test Setup/Teardown are run for every single line, while for the second example they are only ran once.
This might be implied with the 'six separate tests' vs 'only one test', but it might be worth mentioning explicitly.
I tried to solve this by adding a [Setup] and [Teardown] to my Test Scenario keyword, you can only have [Teardown], so this fails.
My current workaround is to just call the setup keyword as the first thing in my scenario. This is not an ideal solution for me because we tend to assign different meanings to fails during a setup vs fails during a test body. (Infrastructure failure vs DUT failure, leading to a different team being assigned to the initial investigation.)
I see a few ways to make this work, but there might be others:
Adding [Setup] to user keywords. This seems like it should be straightforward, considering [Teardown] already exists. This will probably also be useful in other scenarios.
Adding something like a [Template Setup]/[Template Teardown] to templated test cases.
Changing the semantics of a test case with [Template] so that it runs the Setup/Teardown for every invocation. This will likely break test cases that rely on the current behaviour, so is probably a bad idea.
EDIT: The difference between 1 test vs many tests is a particular pitfall with libraries that have different scopes. If you change from many tests to a FOR loop within 1 test, it messes up the scope resetting of libraries, making my workaround useless without extra logic.
The text was updated successfully, but these errors were encountered:
Adding [Setup] to user keywords would be straightforward. The reason it wasn't added when we added keyword [Teardown] was that there's only a semantic difference with the first keyword and a setup. Apparently even that difference is meaningful in some contexts, and keywords having [Setup] would be good also from consistency point of view.
I believe it's too late to add keyword setup anymore in RF 6.1, but it could be done in 6.2.
Could you @dries007 submit a new issue specifically about adding keyword setup? Then we could close this issue that talks about other alternatives how to enable templated loops with individual setups.
I'd like to find a way to take a variable set of test data and run it trough a test scenario with individual test setup & teardown invocations.
I have a Variables file that lists a number of scenario parameters. I want to have robot run a Test Template with all of the scenarios defined in the file. Optionally skipping some depending on some filters, but I'll leave that out of the example below for complexity's sake.
I found the Templates with FOR loops ability, so I intended to used that.
In "Data driven style", it mentions:
Turns out there is an important functional difference for me:
In the first example, the Test Setup/Teardown are run for every single line, while for the second example they are only ran once.
This might be implied with the 'six separate tests' vs 'only one test', but it might be worth mentioning explicitly.
I tried to solve this by adding a
[Setup]
and[Teardown]
to my Test Scenario keyword, you can only have[Teardown]
, so this fails.My current workaround is to just call the setup keyword as the first thing in my scenario. This is not an ideal solution for me because we tend to assign different meanings to fails during a setup vs fails during a test body. (Infrastructure failure vs DUT failure, leading to a different team being assigned to the initial investigation.)
I see a few ways to make this work, but there might be others:
[Setup]
to user keywords. This seems like it should be straightforward, considering[Teardown]
already exists. This will probably also be useful in other scenarios.[Template Setup]
/[Template Teardown]
to templated test cases.[Template]
so that it runs the Setup/Teardown for every invocation. This will likely break test cases that rely on the current behaviour, so is probably a bad idea.EDIT: The difference between 1 test vs many tests is a particular pitfall with libraries that have different scopes. If you change from many tests to a FOR loop within 1 test, it messes up the scope resetting of libraries, making my workaround useless without extra logic.
The text was updated successfully, but these errors were encountered: