S3 ASF fix pre-signed ports permutation #7152
Conversation
2f0a9f4 to
85bb943
Compare
85bb943 to
6120cac
Compare
| for header, value in context.request.headers.items(): | ||
| header_low = header.lower() | ||
| if header_low.startswith("x-amz-") or header_low in ["content-type", "date", "content-md5"]: | ||
| new_headers[header_low] = value |
There was a problem hiding this comment.
not sure what we are doing with the resulting request, but should we maybe preserve the original header casing?
| new_headers[header_low] = value | |
| new_headers[header] = value |
There was a problem hiding this comment.
While creating the signature, it lowers the name of the header as well, so we need to lower the header to be able to match the signature. The corresponding code snippet is here in botocore:
botocore.auth.SigV4Auth.headers_to_sign (l.231)
def headers_to_sign(self, request):
"""
Select the headers from the request that need to be included
in the StringToSign.
"""
header_map = HTTPHeaders()
for name, value in request.headers.items():
lname = name.lower()
if lname not in SIGNED_HEADERS_BLACKLIST:
header_map[lname] = value
if 'host' not in header_map:
# TODO: We should set the host ourselves, instead of relying on our
# HTTP client to set it for us.
header_map['host'] = _host_from_url(request.url)
return header_mapI could add a comment pointing to this. The code is full of traps to reverse. I had to use the lower-cased one or it would fail in CI if I remember well.
| "signing": {"bucket": self._bucket}, | ||
| }, | ||
| } | ||
| self.aws_request: AWSRequest = create_request_object(request_dict) |
There was a problem hiding this comment.
nit: maybe it would be good to add the annotation to the class declaration.
There was a problem hiding this comment.
refactored to return the AWSRequest from the method 👍
1ac6d71 to
a5f314d
Compare
Follow-up PR to #7070
While migrating the tests, it came to light that port permutation while checking for SigV4 signature was not working.
For context, while validating the signature, we check the host value. But with LocalStack, you can access S3 from different ports (edge port, 443, 80 or not port at all for example in the browser). So we permute ports to try for different
hostvalue if the first received one is incorrect.This also comes with a significant refactor, as we would create a lot of resource for every permutation which needed to be created only once.