Thanks to visit codestin.com
Credit goes to github.com

Skip to content

remove pin on botocore #8321

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 17, 2023
Merged

remove pin on botocore #8321

merged 2 commits into from
May 17, 2023

Conversation

alexrashed
Copy link
Member

A recent community contribution was aiming at removing the upper version pin on botocore (#8210, thanks @matt-mercer!). Unfortunately, the latest release at the time contained an invalid spec which was reported in boto/botocore#2925.

The latest release of botocore now contains the requested fixes to the kafka service specs: boto/botocore@8bd1a4d
This is why we can finally remove the pin again.

In the future we might introduce an automate approach (using dependabot or similar bots) to explicitly upgrade these core libraries on a fixed schedule (to move more towards a proactive approach instead of the currently rather reactive approach when new releases of these libraries - like werkzeug or botocore - break our builds).
With #8307 we also moved botocore and boto3 from the install dependencies to the runtime extra (aka from a CLI dependency to a runtime dependency, which allows us to be more restrictive with the pins since the CLI dependencies have a wider impact compared to the runtime dependencies which are usually only used within our Docker images).

@alexrashed alexrashed added the semver: patch Non-breaking changes which can be included in patch releases label May 16, 2023
@alexrashed alexrashed requested a review from thrau May 16, 2023 06:50
@github-actions
Copy link

github-actions bot commented May 16, 2023

LocalStack Community integration with Pro

2 044 tests   1 760 ✔️  1h 18m 44s ⏱️
       2 suites     284 💤
       2 files           0

Results for commit d13a19e.

♻️ This comment has been updated with latest results.

@alexrashed
Copy link
Member Author

alexrashed commented May 16, 2023

Unfortunately, the upgrade introduces a new issue.
Only a single test fails:

test_request_with_response_header_location_fields

It seems that the issue is caused by botocore==1.29.133 (the test is successful with 1.29.132 and reliably fails with 1.29.133).
This release contains the following changes (unfortunately quite a big changeset due to formatting changes): boto/botocore@1.29.132...1.29.133

The issue is a bit weird though. The URL path parameter is now weirdly encoded.
Previously, the URL ID was properly

The request generated in the following lines changed:

moto.create_aws_request_context(
"route53", "DeleteHostedZone", {"Id": response["HostedZone"]["Id"]}
)

It now contains the url encoded prefix of the ID

http://localhost.localstack.cloud/2013-04-01/hostedzone/%252Fhostedzone%252FZH4ND9S9VU7TGHE

instead of:

http://localhost.localstack.cloud/2013-04-01/hostedzone/H6GFBU6YIZOKDT8

Edit:
The issue was caused by the following commit: boto/botocore@4f5bc57
This commit removed a specific parameter pre-processing from a botocore internal function we use when creating requests within the request-forwarder.
It was fixed the last commit on this PR (fix param pre-processing for forwarder after botocore upgrade).

@coveralls
Copy link

Coverage Status

Coverage: 82.231% (-0.02%) from 82.247% when pulling d13a19e on remove-botocore-pin into e509193 on master.

Copy link
Member

@thrau thrau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a nice catch! LGTM!

@alexrashed alexrashed merged commit a3145fd into master May 17, 2023
@alexrashed alexrashed deleted the remove-botocore-pin branch May 17, 2023 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver: patch Non-breaking changes which can be included in patch releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants