Thanks to visit codestin.com
Credit goes to github.com

Skip to content

mm: upgrade botocore and boto3 #8210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 2, 2023
Merged

mm: upgrade botocore and boto3 #8210

merged 1 commit into from
May 2, 2023

Conversation

matt-mercer
Copy link
Contributor

@matt-mercer matt-mercer commented Apr 27, 2023 •
edited by thrau
Loading

This PR upgrades botocore to 1.29.121 and boto3 to 1.26.121. the services ivs and ivs-realtime were added to the list of excluded services for now.

@matt-mercer
Copy link
Contributor Author

@thrau PR for just to boto 3/core update from #8184

@matt-mercer matt-mercer force-pushed the mm-upgrade-boto3-botocore branch 2 times, most recently from 22c2f6f to 807c5d9 Compare April 27, 2023 07:26
@matt-mercer matt-mercer mentioned this pull request Apr 27, 2023
Merged
@matt-mercer matt-mercer force-pushed the mm-upgrade-boto3-botocore branch 3 times, most recently from a59a106 to dbe5d94 Compare April 27, 2023 10:16
@matt-mercer
Copy link
Contributor Author

anyone? no idea why ci/circleci: docker-test-arm64 would fail ... no way to reproduce this locally

alexrashed
alexrashed requested changes Apr 27, 2023
View reviewed changes
Copy link
Member

@alexrashed alexrashed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @matt-mercer!
Thanks a lot for your contribution! 🚀
I am sorry about the build pipeline failure, we are currently seeing some flaky tests. We will try to harden them in the future, and I restarted your build to get it green.
Again thanks a lot for grinding over these changes, especially over the - quite complex - service router. But actually, I think most of the changes in this PR are not necessary at this point in time. We might just get away with adding these new services to the exclusions and bump the versions.
Let me know if you need any help, or if any comments are unclear.

@matt-mercer matt-mercer requested a review from alexrashed April 27, 2023 16:03
@matt-mercer matt-mercer force-pushed the mm-upgrade-boto3-botocore branch from 2b2428e to 4d5272f Compare April 27, 2023 20:08
alexrashed
alexrashed requested changes Apr 28, 2023
View reviewed changes
Copy link
Member

@alexrashed alexrashed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing all the comments! I think when we move the remaining changes related to the new runtime support (addressed in #8184), and the tests are green, we should be good to go!

@matt-mercer matt-mercer requested a review from alexrashed April 28, 2023 08:03
@matt-mercer matt-mercer force-pushed the mm-upgrade-boto3-botocore branch 3 times, most recently from e9cf56b to 4ff1616 Compare April 28, 2023 08:56
@matt-mercer
Copy link
Contributor Author

image

now failing on some kafka test that doesn't seem to exist locally ..
image

presumably some inconsistency between package versions ... would a lock file be a good idea?

@matt-mercer
Copy link
Contributor Author

matt-mercer commented Apr 28, 2023 •
edited
Loading

@alexrashed I really think we should pin (for the purposes of unblocking the python3.10 runtime change), the botocore version in this PR ... since yesterday new operations have been added to kafka that do not appear to be properly setup in terms of the 'input_shape.required_members' .. this is making this change a moving target dependant without any kind of version pinning

the package depenency upper bound was pinned previously, to avoid just this kind of issue ...

@alexrashed
Copy link
Member

I agree, this would be a case for changing the upper version boundary to the release excluding yesterdays (which essentially is a pin).
For some context: This failing test seems to be caused by the latest release of botocore (boto/botocore@184a84c#diff-df958faae3065d40a26bb9c5aa7c07994193ccb26dccf000cb3f2fd42384bb69).
This breaks the test (not the service router). The test creation fails due to the missing declaration of path parameters as required. I would actually consider this as a bug in the specs (since this would also fail when using the command in boto or the AWS CLI without the param?).

Concerning the locking: Unfortunately, locking dependencies has quite an impact, that's not something we can do within this PR.

@alexrashed
Copy link
Member

I created an issue in botocore to track this issue in the specs: boto/botocore#2925

@matt-mercer matt-mercer force-pushed the mm-upgrade-boto3-botocore branch from 4ff1616 to 39030a4 Compare April 28, 2023 11:15
@matt-mercer
Copy link
Contributor Author

Concerning the locking: Unfortunately, locking dependencies has quite an impact, that's not something we can do within this PR.

Agree .. definitely not something I want to tackle in this! :)

@matt-mercer matt-mercer force-pushed the mm-upgrade-boto3-botocore branch 3 times, most recently from 7a8c7a8 to 15989e9 Compare April 28, 2023 12:45
@matt-mercer matt-mercer force-pushed the mm-upgrade-boto3-botocore branch from 15989e9 to a64ca5b Compare April 28, 2023 14:02
@matt-mercer matt-mercer force-pushed the mm-upgrade-boto3-botocore branch from a64ca5b to e4f302a Compare April 29, 2023 12:08
alexrashed
alexrashed approved these changes May 2, 2023
View reviewed changes
Copy link
Member

@alexrashed alexrashed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for jumping on this! The two failing checks are currently not working for fork PRs, I executed these tests manually and can confirm that the these tests are green as well. 🚀

@alexrashed alexrashed merged commit 95e5e40 into localstack:master May 2, 2023
@matt-mercer
Copy link
Contributor Author

Thanks a lot for jumping on this! The two failing checks are currently not working for fork PRs, I executed these tests manually and can confirm that the these tests are green as well. rocket

great, thanks @alexrashed ( and yup .. I presumed the pro tests /secrets access wasn't permited for forks )

@thrau
Copy link
Member

thrau commented May 2, 2023

thanks @matt-mercer for going through the process, and welcome to the list of contributors :-)

@matt-mercer matt-mercer deleted the mm-upgrade-boto3-botocore branch May 3, 2023 10:19
@matt-mercer
Copy link
Contributor Author

thanks @matt-mercer for going through the process, and welcome to the list of contributors :-)

thanks for the support!

@alexrashed alexrashed mentioned this pull request May 16, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexrashed alexrashed alexrashed approved these changes

@joe4dev joe4dev Awaiting requested review from joe4dev

@dfangl dfangl Awaiting requested review from dfangl

@dominikschubert dominikschubert Awaiting requested review from dominikschubert

@thrau thrau Awaiting requested review from thrau

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@matt-mercer @alexrashed @thrau