Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix: Unable to forward SNS message to SQS: An error occurred (AWS.SimpleQueueService.EmptyBatchRequest) when calling the SendMessageBatch #7663

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

onhate
Copy link

@onhate onhate commented Feb 11, 2023

fix for #7662

…pleQueueService.EmptyBatchRequest) when calling the SendMessageBatch
@onhate onhate requested a review from bentsku as a code owner February 11, 2023 13:45
Copy link
Contributor

@localstack-bot localstack-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome to LocalStack! Thanks for raising your first Pull Request and landing in your contributions. Our team will reach out with any reviews or feedbacks that we have shortly. We recommend joining our Slack Community and share your PR on the #community channel to share your contributions with us. Please make sure you are following our contributing guidelines and our Code of Conduct.

@localstack-bot
Copy link
Contributor

localstack-bot commented Feb 11, 2023

CLA Assistant Lite bot All contributors have signed the CLA ✍️ βœ…

@onhate
Copy link
Author

onhate commented Feb 11, 2023

I have read the CLA Document and I hereby sign the CLA

@onhate onhate requested review from localstack-bot and removed request for bentsku February 11, 2023 13:50
@bentsku
Copy link
Contributor

bentsku commented Feb 13, 2023

Hi @onhate and thanks for your contribution!

I was pretty puzzled by this bug, because as we can see in the logs of the bug report, it should not actually send work to the worker thread doing the publishing to SQS:
No messages match filter policy, not publishing batch from topic 'arn:aws:sns:us-east-1:000000000000:<redacted>-events.fifo' to subscription 'arn:aws:sns:us-east-1:000000000000:<redacted>-events.fifo:71de2566-97e6-48b8-a222-54c9adadcf96'

However, after looking, it seems we have a deeper bug in there: while filtering the messages for the subscriptions, we modify the context we send to the workers while it is being processed, which leads to the issue. This PR would fix the error messages but still subscribers would not get the messages because it would be filtered when it should not, so we need to fix the underlying bug first, and then no worker would receive an empty message entries anymore.
I also wonder about the fact you had to restart LocalStack as well, as it seems the exception was handled and logged, so maybe something else is amiss here.

Would you mind if I open this bugfix PR pretty quickly, as it's quite a serious bug?

@onhate
Copy link
Author

onhate commented Feb 13, 2023

thanks for the quick response @bentsku
I would love to see a fix to this and learn more about how it works :)

@bentsku
Copy link
Contributor

bentsku commented Feb 13, 2023

Hey @onhate !
I've opened #7674 now and I tried to explain in detail what lead to your issue. I'm really grateful for your report!
For how we structured the publishing code for SNS, here's the PR refactoring it to the current implementation: #7267 (and here's the S3 Notification PR mentioned #6903, introducing this pattern across our codebase).
If you have any question about something in particular, do not hesitate to ask!

@onhate onhate closed this Feb 14, 2023
@onhate onhate deleted the fix-sns-batch-publish-sqs-empty-entries branch February 14, 2023 17:54
@onhate
Copy link
Author

onhate commented Feb 14, 2023

Thank you very much @bentsku for the quick actions, I went through the code changes and I am glad I could understand it all :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants