Fix SFN.client.start_execution() idempotency #9397
Conversation
| input='{"a": "b", "c": "d"}', | ||
| ) | ||
| # | ||
| execution_two = client.start_execution( | ||
| stateMachineArn=sm["stateMachineArn"], | ||
| name="execution_name", | ||
| input='{"c": "d", "a": "b"}', |
There was a problem hiding this comment.
Here's a more concrete example of what I meant in the description - are these inputs equal or not?
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #9397 +/- ##
==========================================
- Coverage 93.00% 93.00% -0.01%
==========================================
Files 1294 1294
Lines 115774 115780 +6
==========================================
+ Hits 107672 107677 +5
- Misses 8102 8103 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Hi @chriselion , thanks for raising the PR!
No, I'm afraid not - I would (also) have to do a bunch of manual testing to determine the exact behavior. This does look like a tricky scenario, so it does require some more research IMO. I'd be happy to do so, I just can't guarantee a timeline when that would happen. Let me know what you prefer. (Edit: also marking this PR as a draft, to make it clear it's not quite ready yet.) |
|
I ran this script import boto3
import json
import os
definition = {
"StartAt": "Wait",
"States": {
"Wait": {
"Type": "Wait",
"Seconds": 60,
"End": True
}
},
"QueryLanguage": "JSONata"
}
input1 = '{"a": "b", "c": "d"}'
input2 = '{"c": "d", "a": "b"}'
execution_name = "testExecution"
assert json.loads(input1) == json.loads(input2)
client = boto3.client("stepfunctions")
print("creating state machine")
resp = client.create_state_machine(
name="motoRepro1",
definition=json.dumps(definition),
roleArn=os.getenv("ROLE_ARN")
)
state_machine_arn = resp["stateMachineArn"]
try:
execution_response = client.start_execution(
stateMachineArn=state_machine_arn,
name=execution_name,
input=input1,
)
execution_arn = execution_response["executionArn"]
print("Initial execution successful")
except:
print("Initial execution failed")
try:
identical_response = client.start_execution(
stateMachineArn=state_machine_arn,
name=execution_name,
input=input1,
)
print("Identical execution successful")
print(f"{(execution_arn==identical_response["executionArn"])=}")
except:
print("Identical execution failed")
try:
equivalent_response = client.start_execution(
stateMachineArn=state_machine_arn,
name=execution_name,
input=input2,
)
print("Equivalent execution successful")
print(f"{(execution_arn==equivalent_response["executionArn"])}")
except:
print("Equivalent execution failed")
print("deleting state machine")
client.delete_state_machine(stateMachineArn=state_machine_arn)which produces this output: So I'll update |
|
I tried this tonight, but it's more complicated than I expected. I removed the moto/moto/stepfunctions/responses.py Line 210 in e9a44a2 but that ended up breaking a lot of tests. |
Resolves #9394
When an execution with the same name is found, it now compares the inputs:
ExecutionAlreadyExistsexception is raised (same as before)I updated the existing test case to pass an input, and added another test to check that duplicating the name and input is idempotent.
I'm not sure how we should define "equal" inputs here. StateMachine.start_execution() saves the json-decoded input on the Execution
moto/moto/stepfunctions/models.py
Lines 133 to 139 in 86fe368
although the field is supposed to be a
strmoto/moto/stepfunctions/models.py
Lines 310 to 318 in 86fe368
Since the original input is lost, I'm comparing the json-decoded values.
I can do a little more digging to see what AWS actually does (although if you have some intuition, that would save me some time). If we need to check the exact string equality instead, that'll take some more changes.