feature request: add a fail-fast mechanic to assertions #458
Description
Assertions may not have the same "importance level" and it would be nice to have a fail-fast mechanic, meaning that if a step has retries, some assertions could result in an early break.
An example use case would be the following:
- name: Wait for virtual machine deployment
steps:
- type: http
url: https://api.example.org/vm/1
retry: 100
assertions:
# API returns will 200 once we have our virtual machine deployed, but it can take several minutes
- result.statuscode ShouldEqual 200
# But in case of 403 for example (like wrong credentials), no need to retry, we're just wasting time
- result.statuscode ShouldNotEqual 403 In the above code, if wrong credentials are passed, it's useless to wait status code to become a 200, because it'll never suceed
Proposed syntaxes:
#1 at step level:
- type: http
url: https://api.example.org/vm/1
retry: 100
assertions:
- result.statuscode ShouldEqual 200
fail-fast:
- result.statuscode ShouldNotEqual 403 #2 at assertion level with a "fail-fast" operator:
- type: http
url: https://api.example.org/vm/1
retry: 100
assertions:
- result.statuscode ShouldEqual 200
- fail-fast:
- result.statuscode ShouldNotEqual 403