Add s3m.stream to awesome-aws list#258
Open
nbari wants to merge 1 commit into
Open
Conversation
|
Need more detailed descriptions in that context. |
Author
|
Hi @desingh9, maybe this helps: s3m is designed for streaming uploads to S3 from stdin/pipes (e.g. database backups) using resumable multipart uploads. This allows large or hard-to-reproduce streams to continue after network interruptions instead of restarting the entire pipeline. It is optimized for low memory usage and supports optional client-side encryption, zstd compression, throttling, and retries, making it suitable for production backup workflows and constrained hosts. s3m is designed to run in very constrained environments with minimal resource usage: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Review the Contributing Guidelines
Before submitting a pull request, verify it meets all requirements in the Contributing Guidelines.
Describe Why This Is Awesome
Why is this awesome?
The aim of s3m apart from trying to consume fewer resources is to make as much as possible "fault-tolerant" the storage procedure of the incoming stream so that even if the server lost network connectivity, the stream can be resumed and continue the upload process where it was left without the need to start all over again (no when input is from STDIN/pipe).
Like this pull request? Vote for it by adding a 👍