Thanks to visit codestin.com
Credit goes to github.com

Skip to content

S3 migration - set s3 v2 provider as default #6827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Mar 23, 2023
Merged

S3 migration - set s3 v2 provider as default #6827

merged 10 commits into from
Mar 23, 2023

Conversation

steffyP
Copy link
Member

@steffyP steffyP commented Sep 6, 2022 •
edited
Loading

Summary

This PR should remain minimal in terms of code changes, and is mostly used for keeping track of the PRs regarding the new s3 provider going directly into master and for tracking test failure rates against the new provider.

PRs

The following PRs have been created to fully implement the new s3 provider:

Tasks

  • Migrate all s3 tests to pytest and validates them against AWS

  • Create the new ASF provider

  • Allow patching of botocore specs to match actual AWS behaviour

  • Reroute virtual host addressed requests as path addressed ones to take advantage of the builtin ASF routing

  • Implement major S3 features:

    • S3 notifications
    • All major basic OPs (GetObject, GetBucket, etc)
    • Pre-signed URLs
    • Pre-signed POST
    • S3 Static Website Hosting
    • CORS

  • Fix ongoing issues with current provider not tackled in above major features

    • VersionId and Expires always present
    • GetObjectAttributes not implemented
    • Empty tagging lists
    • Delete versioned objects
    • Fix Terraform tests (fixed in S3 asf fixes detected by Terraform test suite #7027 + full test run green here)
    • remove legacy VirtualHostRewriter
    • Fix issues with presigned urls

  • Implement the Pro provider

  • Implement S3 ASF test job for CircleCI

@steffyP steffyP temporarily deployed to localstack-ext-tests September 6, 2022 12:11 Inactive
@steffyP steffyP temporarily deployed to localstack-ext-tests September 6, 2022 12:37 Inactive
@github-actions
Copy link

github-actions bot commented Sep 6, 2022 •
edited
Loading

LocalStack integration with Pro

       1 files   -     2         1 suites   - 2   1h 7m 29s ⏱️ - 25m 55s
1 821 tests  -   16  1 478 ✔️ +  29  342 💤  -   46  1 ❌ +1 
1 821 runs   - 752  1 478 ✔️  - 338  342 💤  - 415  1 ❌ +1 

For more details on these failures, see this check.

Results for commit b0af983. ± Comparison against base commit 6e81652.

♻️ This comment has been updated with latest results.

@bentsku bentsku mentioned this pull request Sep 6, 2022
Merged
@steffyP steffyP closed this Sep 6, 2022
@bentsku bentsku reopened this Sep 6, 2022
@bentsku bentsku temporarily deployed to localstack-ext-tests September 6, 2022 17:24 Inactive
@bentsku
Copy link
Contributor

bentsku commented Sep 7, 2022

Note: we have 53 failed tests for now, to keep track of progress.

@steffyP steffyP temporarily deployed to localstack-ext-tests September 8, 2022 16:13 Inactive
@steffyP steffyP temporarily deployed to localstack-ext-tests September 9, 2022 15:06 Inactive
@bentsku bentsku mentioned this pull request Sep 12, 2022
Merged
@bentsku bentsku temporarily deployed to localstack-ext-tests September 12, 2022 12:32 Inactive
@bentsku
Copy link
Contributor

bentsku commented Sep 13, 2022

We are at 92 tests failing for this branch before merging any functionality.

@bentsku bentsku temporarily deployed to localstack-ext-tests September 13, 2022 23:06 Inactive
@steffyP steffyP temporarily deployed to localstack-ext-tests September 14, 2022 14:55 Inactive
@bentsku bentsku temporarily deployed to localstack-ext-tests September 14, 2022 20:38 Inactive
@steffyP steffyP temporarily deployed to localstack-ext-tests September 19, 2022 16:13 Inactive
@steffyP steffyP temporarily deployed to localstack-ext-tests September 19, 2022 17:38 Inactive
@bentsku bentsku temporarily deployed to localstack-ext-tests September 20, 2022 20:16 Inactive
@bentsku bentsku requested a review from thrau as a code owner March 20, 2023 16:46
@bentsku bentsku self-requested a review March 20, 2023 16:46
@bentsku bentsku temporarily deployed to localstack-ext-tests March 20, 2023 18:16 — with GitHub Actions Inactive
@bentsku bentsku temporarily deployed to localstack-ext-tests March 22, 2023 15:51 — with GitHub Actions Inactive
@dfangl dfangl requested a review from alexrashed as a code owner March 22, 2023 22:14
@bentsku bentsku merged commit 86d706f into v2 Mar 23, 2023
thrau pushed a commit that referenced this pull request Mar 23, 2023
@steffyP @bentsku @thrau
S3 migration - set s3 v2 provider as default (#6827)
fad6714 
Co-authored-by: Ben Simon Hartung <[email protected]>
alexrashed pushed a commit that referenced this pull request Mar 23, 2023
@steffyP @bentsku @alexrashed
S3 migration - set s3 v2 provider as default (#6827)
a0b59aa 
Co-authored-by: Ben Simon Hartung <[email protected]>
@alexrashed alexrashed deleted the s3-migration branch March 27, 2023 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thrau thrau Awaiting requested review from thrau

@bentsku bentsku Awaiting requested review from bentsku

@alexrashed alexrashed Awaiting requested review from alexrashed

Assignees

@steffyP steffyP

@bentsku bentsku

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@steffyP @bentsku @thrau