Added explicit check for a bucket when deploying a stack #6171

giograno · 2022-05-30T22:29:55Z

Issue coming from support.
The problem came from the fact that we consider a resource as not created if it does not have notification configs attached.
This causes a BucketAlreadyOwnedByYou error outside us-east-1.
I changed the deploy template with a function that explicitly checks if the bucket exists and, if not, proceeds to create it.
Feedback are welcome, quite a newbie here on CloudFormation.

github-actions · 2022-05-30T23:38:42Z

LocalStack integration with Pro

      3 files ±0       3 suites ±0 1h 9m 30s ⏱️ + 6m 2s
1 071 tests +1 1 038 ✔️ +1 33 💤 ±0 0 ❌ ±0
1 369 runs +1 1 309 ✔️ +1 60 💤 ±0 0 ❌ ±0

Results for commit bf002bd. ± Comparison against base commit 932733d.

♻️ This comment has been updated with latest results.

whummer

LGTM, great set of changes @giograno ! 👍

We could try to cover this with a small test. For example, we already have a small test template with S3 NotificationConfiguration here:

localstack/tests/integration/test_cloudformation.py

Lines 262 to 281 in 0a4359f

    
           TEST_TEMPLATE_17 = """ 
        
           AWSTemplateFormatVersion: 2010-09-09 
        
           Resources: 
        
             TestQueue: 
        
               Type: AWS::SQS::Queue 
        
               Properties: 
        
                 QueueName: %s 
        
                 ReceiveMessageWaitTimeSeconds: 0 
        
                 VisibilityTimeout: 30 
        
                 MessageRetentionPeriod: 1209600 
        
             TestBucket: 
        
               Type: AWS::S3::Bucket 
        
               Properties: 
        
                 BucketName: %s 
        
                 NotificationConfiguration: 
        
                   QueueConfigurations: 
        
                     - Event: s3:ObjectCreated:* 
        
                       Queue: %s 
        
           """

So we could parameterize this test with a flag whether to create the bucket before deploying the stack:

localstack/tests/integration/test_cloudformation.py

Lines 1004 to 1006 in 0a4359f

    
           stack = deploy_cfn_template( 
        
               template=TEST_TEMPLATE_17 % (queue_name, bucket_name, queue_arn), 
        
           )

localstack/services/cloudformation/models/s3.py

dominikschubert

LGTM 🪣 🚀 , minor exception handling nit

dominikschubert · 2022-05-31T11:23:11Z

localstack/services/cloudformation/models/s3.py

+            bucket_name = props.get("BucketName")
+            try:
+                s3_client.head_bucket(Bucket=bucket_name)
+            except Exception:


nit: too broad. we should probably explicitly check for it being a not found exception here and avoid swallowing any unrelated issues here

Added a stronger check 👍

giograno · 2022-05-31T14:00:30Z

@whummer I parametrized the test as you suggested. However, since the issue does not appear for the default region us-east-1, I added also a small fixture to patch the default region_name when we create a client. Does this sound like a viable approach? cc @dominikschubert

thrau

LGTM! thanks for jumping on this. i don't know much about CFN but implementation looks good :-)

giograno · 2022-06-01T14:14:36Z

Simplified the test a bit, as discussed with @whummer. Actually, we don't even need to have a specific region backend with the CloudFormation client to replicate the bug, given the global name space for S3 buckets.

whummer

🚀

Introduced a fixture to get clients for a specific region

giograno temporarily deployed to localstack-ext-tests May 30, 2022 22:29 Inactive

github-actions bot added the size/s label May 30, 2022

giograno assigned whummer and unassigned whummer May 31, 2022

giograno requested review from dominikschubert and whummer May 31, 2022 00:50

giograno marked this pull request as ready for review May 31, 2022 00:50

giograno requested a review from thrau as a code owner May 31, 2022 00:50

whummer approved these changes May 31, 2022

View reviewed changes

localstack/services/cloudformation/models/s3.py Outdated Show resolved Hide resolved

dominikschubert approved these changes May 31, 2022

View reviewed changes

giograno temporarily deployed to localstack-ext-tests May 31, 2022 13:55 Inactive

github-actions bot added the tests label May 31, 2022

giograno force-pushed the explicit-bucket-name-check-cloudformation-create branch from f64916e to 07e43a4 Compare May 31, 2022 16:30

giograno temporarily deployed to localstack-ext-tests May 31, 2022 16:30 Inactive

giograno temporarily deployed to localstack-ext-tests June 1, 2022 14:12 Inactive

github-actions bot removed the tests label Jun 1, 2022

thrau approved these changes Jun 1, 2022

View reviewed changes

whummer approved these changes Jun 1, 2022

View reviewed changes

giograno added 4 commits June 1, 2022 10:51

Added explicit check for a bucket when deploying a stack

64d47cc

Some PR comments

27a6583

Added a test for the support case

1b6e053

Introduced a fixture to get clients for a specific region

Semplifying the test

bf002bd

giograno force-pushed the explicit-bucket-name-check-cloudformation-create branch from 9be5379 to bf002bd Compare June 1, 2022 14:51

giograno temporarily deployed to localstack-ext-tests June 1, 2022 14:52 Inactive

giograno merged commit f47969a into master Jun 1, 2022

giograno deleted the explicit-bucket-name-check-cloudformation-create branch June 1, 2022 16:24

github-actions bot locked and limited conversation to collaborators Jun 1, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Added explicit check for a bucket when deploying a stack #6171

Added explicit check for a bucket when deploying a stack #6171

Uh oh!

giograno commented May 30, 2022

Uh oh!

github-actions bot commented May 30, 2022 •

edited

Loading

Uh oh!

whummer left a comment

Uh oh!

Uh oh!

dominikschubert left a comment

Uh oh!

dominikschubert May 31, 2022

Uh oh!

giograno May 31, 2022

Uh oh!

giograno commented May 31, 2022 •

edited

Loading

Uh oh!

thrau left a comment

Uh oh!

giograno commented Jun 1, 2022

Uh oh!

whummer left a comment

Uh oh!

Uh oh!

	TEST_TEMPLATE_17 = """
	AWSTemplateFormatVersion: 2010-09-09
	Resources:
	TestQueue:
	Type: AWS::SQS::Queue
	Properties:
	QueueName: %s
	ReceiveMessageWaitTimeSeconds: 0
	VisibilityTimeout: 30
	MessageRetentionPeriod: 1209600

	TestBucket:
	Type: AWS::S3::Bucket
	Properties:
	BucketName: %s
	NotificationConfiguration:
	QueueConfigurations:
	- Event: s3:ObjectCreated:*
	Queue: %s
	"""

	stack = deploy_cfn_template(
	template=TEST_TEMPLATE_17 % (queue_name, bucket_name, queue_arn),
	)

Uh oh!

Added explicit check for a bucket when deploying a stack #6171

Added explicit check for a bucket when deploying a stack #6171

Uh oh!

Conversation

giograno commented May 30, 2022

Uh oh!

github-actions bot commented May 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LocalStack integration with Pro

Uh oh!

whummer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dominikschubert left a comment

Choose a reason for hiding this comment

Uh oh!

dominikschubert May 31, 2022

Choose a reason for hiding this comment

Uh oh!

giograno May 31, 2022

Choose a reason for hiding this comment

Uh oh!

giograno commented May 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thrau left a comment

Choose a reason for hiding this comment

Uh oh!

giograno commented Jun 1, 2022

Uh oh!

whummer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented May 30, 2022 •

edited

Loading

giograno commented May 31, 2022 •

edited

Loading