Thanks to visit codestin.com
Credit goes to github.com

Skip to content

implement S3 native multiparts operations #8787

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 8, 2023
Merged

implement S3 native multiparts operations #8787

merged 1 commit into from
Aug 8, 2023

Conversation

bentsku
Copy link
Contributor

@bentsku bentsku commented Jul 31, 2023 •
edited
Loading

This PR builds on #8786, and finishes to introduce basic operations.

Implemented operations:

  • CreateMultipartUpload
  • UploadPart
  • UploadPartCopy
  • CompleteMultipartUpload
  • AbortMultipartUpload
  • ListParts
  • ListMultipartUploads

@bentsku bentsku added aws:s3 Amazon Simple Storage Service semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases labels Jul 31, 2023
@bentsku bentsku self-assigned this Jul 31, 2023
This was referenced Jul 31, 2023
Merged
Merged
@bentsku bentsku force-pushed the s3-native-multiparts branch 2 times, most recently from 9830036 to 35742aa Compare August 1, 2023 00:28
@bentsku bentsku changed the title wip: implement S3 native multiparts operations implement S3 native multiparts operations Aug 1, 2023
@bentsku bentsku marked this pull request as ready for review August 1, 2023 14:20
thrau
thrau reviewed Aug 2, 2023
View reviewed changes
Copy link
Member

@thrau thrau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this multipart handling is just 🤯 really cool stuff though!

my comments are mostly the same as in #8786

great job! let's discuss about the storagebackend abstraction and then move forward

@bentsku bentsku force-pushed the s3-native-model-basic-op branch from 6efd1fa to ac855af Compare August 4, 2023 17:14
@bentsku bentsku force-pushed the s3-native-multiparts branch 2 times, most recently from 26a399d to 733f722 Compare August 4, 2023 20:47
@coveralls
Copy link

coveralls commented Aug 4, 2023 •
edited
Loading

Coverage Status

coverage: 81.587% (-0.2%) from 81.816% when pulling 6df3c6d on s3-native-multiparts into 3fae602 on master.

thrau
thrau approved these changes Aug 7, 2023
View reviewed changes
Copy link
Member

@thrau thrau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! the storage abstraction also made this much cleaner!

localstack/services/s3/v3/provider.py
Comment on lines +1385 to +1388
stored_multipart = self._storage_backend.get_multipart(bucket_name, s3_multipart)
stored_s3_part = stored_multipart.open(s3_part)
stored_s3_part.write(body)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌

@bentsku bentsku force-pushed the s3-native-model-basic-op branch from 5e26d7d to 8098b88 Compare August 7, 2023 23:43
@bentsku bentsku force-pushed the s3-native-multiparts branch from 83ac805 to 3fabe02 Compare August 7, 2023 23:56
@bentsku bentsku force-pushed the s3-native-model-basic-op branch from 8098b88 to c9bce20 Compare August 8, 2023 00:08
@bentsku bentsku force-pushed the s3-native-multiparts branch from 3fabe02 to 9686f44 Compare August 8, 2023 00:11
Base automatically changed from s3-native-model-basic-op to master August 8, 2023 02:26
@bentsku bentsku force-pushed the s3-native-multiparts branch from 9686f44 to d8dbb04 Compare August 8, 2023 02:32
@github-actions
Copy link

github-actions bot commented Aug 8, 2023 •
edited
Loading

LocalStack Community integration with Pro

       2 files         2 suites   1h 20m 34s ⏱️
2 041 tests 1 655 ✔️ 386 💤 0 ❌
2 042 runs  1 655 ✔️ 387 💤 0 ❌

Results for commit 6df3c6d.

♻️ This comment has been updated with latest results.

@bentsku bentsku force-pushed the s3-native-multiparts branch from d8dbb04 to 6df3c6d Compare August 8, 2023 13:30
@bentsku bentsku merged commit 87d01c3 into master Aug 8, 2023
@bentsku bentsku deleted the s3-native-multiparts branch August 8, 2023 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thrau thrau thrau approved these changes

@macnev2013 macnev2013 Awaiting requested review from macnev2013

Assignees

@bentsku bentsku

Labels
aws:s3 Amazon Simple Storage Service semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@bentsku @coveralls @thrau