S3 ASF Presigned URL #6908
S3 ASF Presigned URL #6908
Conversation
LocalStack integration with Pro 1 files - 2 1 suites - 2 1h 3m 41s ⏱️ - 9m 40s Results for commit bdeadc8. ± Comparison against base commit dd483f2. ♻️ This comment has been updated with latest results. |
e828540 to
c74ee58
Compare
There was a problem hiding this comment.
awesome 💯 well done on the implementation, some of the code looks really tricky!
i have a fundamental question about pre-signed URL validation: is all the validation state encapsulated in the signature? is there no need to check IAM or the s3 store data? in other words: can the signature validation handler remain stateless? otherwise it could maybe be useful to move s3_presigned_url_request_handler into a a class, and insantiate the class from the provider so we have the possibility to inject dependencies or state into the validator.
maybe a couple of unit tests couldn't hurt to improve the base test coverage :-)
|
Thanks for the review, I will work on that!
To me, signature validation is only doing signature validation. It is a stateless operation validating if the signature provided matches the request received. Then, I agree credentials should be validated in case of I love the idea to be able to inject state from the class, but I am not sure it should be done at this step exactly. I would say the pre-signed URLs are a way to access a resource with credentials provided in another location than the headers. Should we see what should be done with @dfangl? |
f9d8f35 to
6f7abbf
Compare
6f7abbf to
455809b
Compare
|
I added some unit tests, I could also add the actual signing, but this should be tested with integration tests, it would be a bit tedious to unit test, craft some request, get the real signature values, and you need to have the right credentials set to Thanks again for the review, and great points! |
455809b to
f0b32b8
Compare
f0b32b8 to
584c474
Compare
584c474 to
bdeadc8
Compare
Pre-signed URLs are a feature of S3, allowing to use the client or the CLI to pre-sign a request with the credentials used by the client. The docs are quite extensive about it, and you see more about it here:
https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html
https://docs.aws.amazon.com/AmazonS3/latest/userguide/S3_Authentication2.html
You can sign the request with 2 types:
SigV2, which is supposed to be deprecated by still used, andSigV4.You effectively sign the request like you would in the headers normally, but pass it in the query parameters instead.
The LocalStack pre-signed URLs feature is checking if the signature of the request is valid (and does not authenticate the request).
We verify the signature by checking the request we received, reconstructing the Canonical Request as required by AWS, and signing the request with the utilities from
botocoreto check if the signatures are identical. We keep the usage of the feature flagS3_SKIP_SIGNATURE_VALIDATIONto allow requests with wrong signatures to go through.We are making use of the handler chain, to catch the requests before they go through the skeleton to reject them if necessary.
We also modify the responses of operations using
PUTmethod, because AWS "erases" the body content when the request is coming from a pre-signed URL.This PR still miss the pre-signed POST requests, added in a follow-up PR.
The validation should work better than the old provider, and be more precise and in-line with AWS.