Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Impossible to combine 'Templates with FOR loops' and individual Setup invocations #4745

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dries007 opened this issue Apr 21, 2023 · 2 comments

Comments

@dries007
Copy link

dries007 commented Apr 21, 2023

I'd like to find a way to take a variable set of test data and run it trough a test scenario with individual test setup & teardown invocations.

I have a Variables file that lists a number of scenario parameters. I want to have robot run a Test Template with all of the scenarios defined in the file. Optionally skipping some depending on some filters, but I'll leave that out of the example below for complexity's sake.

I found the Templates with FOR loops ability, so I intended to used that.

In "Data driven style", it mentions:

The above example has six separate tests, one for each invalid user/password combination, and the example below illustrates how to have only one test with all the combinations. When using test templates, all the rounds in a test are executed even if there are failures, so there is no real functional difference between these two styles. In the above example separate combinations are named so it is easier to see what they test, but having potentially large number of these tests may mess-up statistics. Which style to use depends on the context and personal preferences.

Turns out there is an important functional difference for me:
In the first example, the Test Setup/Teardown are run for every single line, while for the second example they are only ran once.
This might be implied with the 'six separate tests' vs 'only one test', but it might be worth mentioning explicitly.

I tried to solve this by adding a [Setup] and [Teardown] to my Test Scenario keyword, you can only have [Teardown], so this fails.

My current workaround is to just call the setup keyword as the first thing in my scenario. This is not an ideal solution for me because we tend to assign different meanings to fails during a setup vs fails during a test body. (Infrastructure failure vs DUT failure, leading to a different team being assigned to the initial investigation.)

I see a few ways to make this work, but there might be others:

  1. Adding [Setup] to user keywords. This seems like it should be straightforward, considering [Teardown] already exists. This will probably also be useful in other scenarios.
  2. Adding something like a [Template Setup]/[Template Teardown] to templated test cases.
  3. Changing the semantics of a test case with [Template] so that it runs the Setup/Teardown for every invocation. This will likely break test cases that rely on the current behaviour, so is probably a bad idea.

EDIT: The difference between 1 test vs many tests is a particular pitfall with libraries that have different scopes. If you change from many tests to a FOR loop within 1 test, it messes up the scope resetting of libraries, making my workaround useless without extra logic.

@pekkaklarck
Copy link
Member

Adding [Setup] to user keywords would be straightforward. The reason it wasn't added when we added keyword [Teardown] was that there's only a semantic difference with the first keyword and a setup. Apparently even that difference is meaningful in some contexts, and keywords having [Setup] would be good also from consistency point of view.

I believe it's too late to add keyword setup anymore in RF 6.1, but it could be done in 6.2.

@pekkaklarck
Copy link
Member

Could you @dries007 submit a new issue specifically about adding keyword setup? Then we could close this issue that talks about other alternatives how to enable templated loops with individual setups.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants