add rules for ASF S3 virtual host urls #6846
Conversation
39d0c87 to
bcc47b2
Compare
bcc47b2 to
540f802
Compare
540f802 to
69347dd
Compare
localstack/services/s3/provider.py
Outdated
| original_host = copied_headers["Host"] | ||
| copied_headers["Host"] = host | ||
| params = {"data": request.data, "headers": copied_headers} | ||
| response = requests.request(method=request.method, url=rewritten_url, **params) |
There was a problem hiding this comment.
just a question: as i understand we are forwarding the request to moto here, right? is moto doing a lot for us, or could we also implement the serving logic ourselves by accessing the moto backend directly?
There was a problem hiding this comment.
it's doing a lot. this can actually be any api call from s3, as everything in s3 works just with query-parameters and headers.
localstack/services/s3/provider.py
Outdated
| copied_headers = copy.deepcopy(request.headers) | ||
| # TODO is_internal_call_context | ||
| original_host = copied_headers["Host"] | ||
| copied_headers["Host"] = host | ||
| params = {"data": request.data, "headers": copied_headers} | ||
| response = requests.request(method=request.method, url=rewritten_url, **params) | ||
|
|
||
| response.request.headers["Host"] = original_host | ||
| response.request.url = request.url | ||
| ls_response = convert_response(response) |
There was a problem hiding this comment.
have you considered using the localstack.http.proxy module? it has a bunch of utilities that could simplify some of the code here. alternatively you could also use the HttpClient from localstack.http.client. in any case i would avoid importing convert_response from the apigateway provider.
| code, headers, body = fn(self, bucket_name, *args, **kwargs) | ||
| bucket = self.backend.get_bucket(bucket_name) | ||
| headers["x-amz-bucket-region"] = bucket.region_name | ||
| headers["content-type"] = "application/xml" | ||
| return code, headers, body |
There was a problem hiding this comment.
maybe this is out of scope, but would it be better to remove this patch all together, and wrap the head_bucket operation? or is that not an option because the moto response is a problem for our parser?
There was a problem hiding this comment.
Talked to @bentsku about it: we probably change it in a future PR.
Buckets can be addressed either in path-style
s3.<region>.localhost.localstack.cloud:4566/mybucketor virtual host-stylemybucket.<region>.localhost.localstack.cloud:4566.Path style (without region) is covered by moto.
Note: AWS works a bit differently (also compared to the old provider): if bucket is created with a location constraint and the path-style contains another region, it would respond with 301 (for any region other than
us-east-1), or 307 (forus-east-1). see testtest_access_bucket_different_region