Restrict EC2CreateInstanceOperator cleanup to waiter failures and add guard flag#61272
Merged
vincbeck merged 1 commit intoFeb 5, 2026
Merged
Conversation
Contributor
Author
|
Sorry for tagging you here but this is effectively the same pattern for the other resource-leaking AWS operators applied retrospectively to |
vincbeck
approved these changes
Feb 5, 2026
jhgoebbert
pushed a commit
to jhgoebbert/airflow_Owen-CH-Leung
that referenced
this pull request
Feb 8, 2026
…61272) Co-authored-by: Sameer Mesiah <[email protected]>
81 tasks
Ratasa143
pushed a commit
to Ratasa143/airflow
that referenced
this pull request
Feb 15, 2026
…61272) Co-authored-by: Sameer Mesiah <[email protected]>
choo121600
pushed a commit
to choo121600/airflow
that referenced
this pull request
Feb 22, 2026
…61272) Co-authored-by: Sameer Mesiah <[email protected]>
This was referenced Feb 26, 2026
Subham-KRLX
pushed a commit
to Subham-KRLX/airflow
that referenced
this pull request
Mar 4, 2026
…61272) Co-authored-by: Sameer Mesiah <[email protected]>
dominikhei
pushed a commit
to dominikhei/airflow
that referenced
this pull request
Mar 11, 2026
…61272) Co-authored-by: Sameer Mesiah <[email protected]>
Ankurdeewan
pushed a commit
to Ankurdeewan/airflow
that referenced
this pull request
Mar 15, 2026
…61272) Co-authored-by: Sameer Mesiah <[email protected]>
radhwene
pushed a commit
to radhwene/airflow
that referenced
this pull request
Mar 21, 2026
…61272) Co-authored-by: Sameer Mesiah <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR follows up on PR #60904 by refining cleanup behavior in
EC2CreateInstanceOperator. Cleanup behavior is guarded by a flag and is opted in by default. Cleanup is only triggered for post-start EC2 instance failures (includingWaiterError), ensuring termination is attempted only when an instance was successfully created and avoiding interception of non-AWS exceptions.Rationale
Restricting cleanup to post-creation EC2 instance failures prevents unintended termination in unrelated failure paths while still addressing orphaned instances created during execution, and aligns EC2 cleanup semantics with recent changes to other AWS resource creation operators proposed in PRs #61145, #61051, and #61010 .
Tests
Existing tests continue to validate cleanup behavior. The new
terminate_instance_on_failureflag is now explicitly set toTruein applicable tests for explicitness.Documentation
The docstring for
EC2CreateInstanceOperatorhas been updated with a brief description of the new flag term.Backwards Compatibility
A new flag,
terminate_instance_on_failure, has been added toEC2CreateInstanceOperatorwith a default value ofTrue. Best-effort cleanup will now be attempted if a post-creation failure (includingWaiterError) occurs after the EC2 instance has been successfully created.