Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 412626b

Browse files
bblommersbblommers
committed
Core: Add type hints to aws/core.py
1 parent fa0bbcd commit 412626b

File tree

19 files changed

+109
-95
lines changed

19 files changed

+109
-95
lines changed

‎.circleci/config.yml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -164,15 +164,16 @@ jobs:
164164
steps:
165165
- checkout
166166
- restore_cache:
167-
key: python-requirements-{{ checksum "requirements-dev.txt" }}
167+
key: python-requirements-{{ checksum "requirements-typehint.txt" }}
168168
- run:
169169
name: Setup environment
170170
command: |
171+
make install-dev-types
171172
make install
172173
mkdir -p target/reports
173174
mkdir -p target/coverage
174175
- save_cache:
175-
key: python-requirements-{{ checksum "requirements-dev.txt" }}
176+
key: python-requirements-{{ checksum "requirements-typehint.txt" }}
176177
paths:
177178
- "~/.cache/pip"
178179
- persist_to_workspace:

‎.github/actions/load-localstack-docker-from-artifacts/action.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ runs:
1818
with:
1919
python-version-file: '.python-version'
2020
cache: 'pip'
21-
cache-dependency-path: 'requirements-dev.txt'
21+
cache-dependency-path: 'requirements-typehint.txt'
2222

2323
- name: Install docker helper dependencies
2424
shell: bash

‎.github/actions/setup-tests-env/action.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,11 @@ runs:
88
with:
99
python-version-file: '.python-version'
1010
cache: 'pip'
11-
cache-dependency-path: 'requirements-dev.txt'
11+
cache-dependency-path: 'requirements-typehint.txt'
1212

1313
- name: Install Community Dependencies
1414
shell: bash
15-
run: make install-dev
15+
run: make install-dev-types
1616

1717
- name: Setup environment
1818
shell: bash

‎.github/workflows/aws-main.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -184,7 +184,7 @@ jobs:
184184
fetch-depth: 0
185185

186186
- name: Load Localstack ${{ env.PLATFORM_NAME_AMD64 }} Docker Image
187-
uses: localstack/localstack/.github/actions/load-localstack-docker-from-artifacts@master
187+
uses: ./.github/actions/load-localstack-docker-from-artifacts
188188
with:
189189
platform: ${{ env.PLATFORM_NAME_AMD64 }}
190190

@@ -213,7 +213,7 @@ jobs:
213213
TARGET_IMAGE_NAME="public.ecr.aws/localstack/localstack" ./bin/docker-helper.sh push
214214
215215
- name: Load Localstack ${{ env.PLATFORM_NAME_ARM64 }} Docker Image
216-
uses: localstack/localstack/.github/actions/load-localstack-docker-from-artifacts@master
216+
uses: ./.github/actions/load-localstack-docker-from-artifacts
217217
with:
218218
platform: ${{ env.PLATFORM_NAME_ARM64 }}
219219

‎.github/workflows/aws-tests.yml

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -198,7 +198,7 @@ jobs:
198198
fetch-depth: 0
199199

200200
- name: Prepare Local Test Environment
201-
uses: localstack/localstack/.github/actions/setup-tests-env@master
201+
uses: ./.github/actions/setup-tests-env
202202

203203
- name: Linting
204204
run: make lint
@@ -309,7 +309,7 @@ jobs:
309309
tests/aws/services/lambda_/functions/common
310310
311311
- name: Load Localstack Docker Image
312-
uses: localstack/localstack/.github/actions/load-localstack-docker-from-artifacts@master
312+
uses: ./.github/actions/load-localstack-docker-from-artifacts
313313
with:
314314
platform: "${{ env.PLATFORM }}"
315315

@@ -375,10 +375,10 @@ jobs:
375375
fetch-depth: 0
376376

377377
- name: Prepare Local Test Environment
378-
uses: localstack/localstack/.github/actions/setup-tests-env@master
378+
uses: ./.github/actions/setup-tests-env
379379

380380
- name: Load Localstack Docker Image
381-
uses: localstack/localstack/.github/actions/load-localstack-docker-from-artifacts@master
381+
uses: ./.github/actions/load-localstack-docker-from-artifacts
382382
with:
383383
platform: "${{ env.PLATFORM }}"
384384

@@ -449,7 +449,7 @@ jobs:
449449
fetch-depth: 0
450450

451451
- name: Load Localstack Docker Image
452-
uses: localstack/localstack/.github/actions/load-localstack-docker-from-artifacts@master
452+
uses: ./.github/actions/load-localstack-docker-from-artifacts
453453
with:
454454
platform: "${{ env.PLATFORM }}"
455455

@@ -496,7 +496,7 @@ jobs:
496496
uses: actions/checkout@v4
497497

498498
- name: Prepare Local Test Environment
499-
uses: localstack/localstack/.github/actions/setup-tests-env@master
499+
uses: ./.github/actions/setup-tests-env
500500

501501
- name: Run Cloudwatch v1 Provider Tests
502502
timeout-minutes: 30
@@ -539,7 +539,7 @@ jobs:
539539
uses: actions/checkout@v4
540540

541541
- name: Prepare Local Test Environment
542-
uses: localstack/localstack/.github/actions/setup-tests-env@master
542+
uses: ./.github/actions/setup-tests-env
543543

544544
- name: Download Test Selection
545545
if: ${{ env.TESTSELECTION_PYTEST_ARGS }}
@@ -588,7 +588,7 @@ jobs:
588588
uses: actions/checkout@v4
589589

590590
- name: Prepare Local Test Environment
591-
uses: localstack/localstack/.github/actions/setup-tests-env@master
591+
uses: ./.github/actions/setup-tests-env
592592

593593
- name: Download Test Selection
594594
if: ${{ env.TESTSELECTION_PYTEST_ARGS }}
@@ -639,7 +639,7 @@ jobs:
639639
uses: actions/checkout@v4
640640

641641
- name: Prepare Local Test Environment
642-
uses: localstack/localstack/.github/actions/setup-tests-env@master
642+
uses: ./.github/actions/setup-tests-env
643643

644644
- name: Download Test Selection
645645
if: ${{ env.TESTSELECTION_PYTEST_ARGS }}
@@ -691,7 +691,7 @@ jobs:
691691
fetch-depth: 0
692692

693693
- name: Load Localstack Docker Image
694-
uses: localstack/localstack/.github/actions/load-localstack-docker-from-artifacts@master
694+
uses: ./.github/actions/load-localstack-docker-from-artifacts
695695
with:
696696
platform: "${{ env.PLATFORM }}"
697697

‎.pre-commit-config.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ repos:
1515
hooks:
1616
- id: mypy
1717
entry: bash -c 'cd localstack-core && mypy --install-types --non-interactive'
18+
additional_dependencies: ['botocore-stubs', 'rolo']
1819

1920
- repo: https://github.com/pre-commit/pre-commit-hooks
2021
rev: v5.0.0

‎localstack-core/localstack/aws/api/core.py

Lines changed: 40 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
11
import functools
2-
from typing import Any, NamedTuple, Optional, Protocol, Type, TypedDict, Union
2+
from typing import (
3+
Any,
4+
Callable,
5+
NamedTuple,
6+
ParamSpec,
7+
Protocol,
8+
Type,
9+
TypedDict,
10+
TypeVar,
11+
)
312

413
from botocore.model import OperationModel, ServiceModel
514
from rolo.gateway import RequestContext as RoloRequestContext
@@ -13,6 +22,10 @@ class ServiceRequest(TypedDict):
1322
pass
1423

1524

25+
P = ParamSpec("P")
26+
T = TypeVar("T")
27+
28+
1629
ServiceResponse = Any
1730

1831

@@ -28,7 +41,7 @@ class ServiceException(Exception):
2841
sender_fault: bool
2942
message: str
3043

31-
def __init__(self, *args, **kwargs):
44+
def __init__(self, *args: Any, **kwargs: Any):
3245
super(ServiceException, self).__init__(*args)
3346

3447
if len(args) >= 1:
@@ -72,38 +85,38 @@ class RequestContext(RoloRequestContext):
7285
context, so it can be used for logging or modification before going to the serializer.
7386
"""
7487

75-
request: Optional[Request]
88+
request: Request
7689
"""The underlying incoming HTTP request."""
77-
service: Optional[ServiceModel]
90+
service: ServiceModel | None
7891
"""The botocore ServiceModel of the service the request is made to."""
79-
operation: Optional[OperationModel]
92+
operation: OperationModel | None
8093
"""The botocore OperationModel of the AWS operation being invoked."""
81-
region: Optional[str]
94+
region: str
8295
"""The region the request is made to."""
8396
partition: str
8497
"""The partition the request is made to."""
85-
account_id: Optional[str]
98+
account_id: str
8699
"""The account the request is made from."""
87-
request_id: Optional[str]
100+
request_id: str | None
88101
"""The autogenerated AWS request ID identifying the original request"""
89-
service_request: Optional[ServiceRequest]
102+
service_request: ServiceRequest | None
90103
"""The AWS operation parameters."""
91-
service_response: Optional[ServiceResponse]
104+
service_response: ServiceResponse | None
92105
"""The response from the AWS emulator backend."""
93-
service_exception: Optional[ServiceException]
106+
service_exception: ServiceException | None
94107
"""The exception the AWS emulator backend may have raised."""
95-
internal_request_params: Optional[InternalRequestParameters]
108+
internal_request_params: InternalRequestParameters | None
96109
"""Data sent by client-side LocalStack during internal calls."""
97-
trace_context: dict
110+
trace_context: dict[str, Any]
98111
"""Tracing metadata such as X-Ray trace headers"""
99112

100-
def __init__(self, request=None) -> None:
113+
def __init__(self, request: Request):
101114
super().__init__(request)
102115
self.service = None
103116
self.operation = None
104-
self.region = None
117+
self.region = None # type: ignore[assignment] # type=str, because we know it will always be set downstream
105118
self.partition = "aws" # Sensible default - will be overwritten by region-handler
106-
self.account_id = None
119+
self.account_id = None # type: ignore[assignment] # type=str, because we know it will always be set downstream
107120
self.request_id = long_uid()
108121
self.service_request = None
109122
self.service_response = None
@@ -119,7 +132,7 @@ def is_internal_call(self) -> bool:
119132
return self.internal_request_params is not None
120133

121134
@property
122-
def service_operation(self) -> Optional[ServiceOperation]:
135+
def service_operation(self) -> ServiceOperation | None:
123136
"""
124137
If both the service model and the operation model are set, this returns a tuple of the service name and
125138
operation name.
@@ -130,7 +143,7 @@ def service_operation(self) -> Optional[ServiceOperation]:
130143
return None
131144
return ServiceOperation(self.service.service_name, self.operation.name)
132145

133-
def __repr__(self):
146+
def __repr__(self) -> str:
134147
return f"<RequestContext {self.service=}, {self.operation=}, {self.region=}, {self.account_id=}, {self.request=}>"
135148

136149

@@ -141,7 +154,7 @@ class ServiceRequestHandler(Protocol):
141154

142155
def __call__(
143156
self, context: RequestContext, request: ServiceRequest
144-
) -> Optional[Union[ServiceResponse, Response]]:
157+
) -> ServiceResponse | Response | None:
145158
"""
146159
Handle the given request.
147160
@@ -152,19 +165,21 @@ def __call__(
152165
raise NotImplementedError
153166

154167

155-
def handler(operation: str = None, context: bool = True, expand: bool = True):
168+
def handler(
169+
operation: str | None = None, context: bool = True, expand: bool = True
170+
) -> Callable[[Callable[P, T]], Callable[P, T]]:
156171
"""
157172
Decorator that indicates that the given function is a handler
158173
"""
159174

160-
def wrapper(fn):
175+
def wrapper(fn: Callable[P, T]) -> Callable[P, T]:
161176
@functools.wraps(fn)
162-
def operation_marker(*args, **kwargs):
177+
def operation_marker(*args: P.args, **kwargs: P.kwargs) -> T:
163178
return fn(*args, **kwargs)
164179

165-
operation_marker.operation = operation
166-
operation_marker.expand_parameters = expand
167-
operation_marker.pass_context = context
180+
operation_marker.operation = operation # type: ignore[attr-defined]
181+
operation_marker.expand_parameters = expand # type: ignore[attr-defined]
182+
operation_marker.pass_context = context # type: ignore[attr-defined]
168183

169184
return operation_marker
170185

‎localstack-core/localstack/aws/client.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -395,8 +395,7 @@ def __call__(
395395
operation: OperationModel = request.operation_model
396396

397397
# create request
398-
context = RequestContext()
399-
context.request = create_http_request(request)
398+
context = RequestContext(request=create_http_request(request))
400399

401400
# TODO: just a hacky thing to unblock the service model being set to `sqs-query` blocking for now
402401
# this is using the same services as `localstack.aws.protocol.service_router.resolve_conflicts`, maybe

‎localstack-core/localstack/aws/forwarder.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -262,11 +262,10 @@ def create_aws_request_context(
262262
)
263263

264264
aws_request: AWSPreparedRequest = client._endpoint.create_request(request_dict, operation)
265-
context = RequestContext()
265+
context = RequestContext(request=create_http_request(aws_request))
266266
context.service = service
267267
context.operation = operation
268268
context.region = region
269-
context.request = create_http_request(aws_request)
270269
context.service_request = parameters
271270

272271
return context

‎localstack-core/localstack/testing/aws/util.py

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -108,13 +108,14 @@ def create_client_with_keys(
108108
def create_request_context(
109109
service_name: str, operation_name: str, region: str, aws_request: AWSPreparedRequest
110110
) -> RequestContext:
111-
context = RequestContext()
111+
if hasattr(aws_request.body, "read"):
112+
aws_request.body = aws_request.body.read()
113+
request = create_http_request(aws_request)
114+
115+
context = RequestContext(request=request)
112116
context.service = load_service(service_name)
113117
context.operation = context.service.operation_model(operation_name=operation_name)
114118
context.region = region
115-
if hasattr(aws_request.body, "read"):
116-
aws_request.body = aws_request.body.read()
117-
context.request = create_http_request(aws_request)
118119
parser = create_parser(context.service)
119120
_, instance = parser.parse(context.request)
120121
context.service_request = instance

‎localstack-core/mypy.ini

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[mypy]
22
explicit_package_bases = true
33
mypy_path=localstack-core
4-
files=localstack/packages,localstack/services/kinesis/packages.py
4+
files=localstack/aws/api/core.py,localstack/packages,localstack/services/kinesis/packages.py
55
ignore_missing_imports = False
66
follow_imports = silent
77
ignore_errors = False

‎tests/aws/test_moto.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -246,11 +246,10 @@ def test_call_with_sns_with_full_uri():
246246
headers={"Content-Type": "application/x-www-form-urlencoded; charset=utf-8"},
247247
)
248248
sns_service = load_service("sns")
249-
context = RequestContext()
249+
context = RequestContext(sns_request)
250250
context.account = "test"
251251
context.region = "us-west-1"
252252
context.service = sns_service
253-
context.request = sns_request
254253
context.operation = sns_service.operation_model("CreateTopic")
255254

256255
create_topic_response = moto.call_moto(context)

‎tests/integration/test_forwarder.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ def test_request_forwarder(_, __) -> ServiceResponse:
2525

2626
# invoke the function and expect the result from the fallback function
2727
dispatcher = ForwardingFallbackDispatcher(test_provider, test_request_forwarder)
28-
assert dispatcher["TestOperation"](RequestContext(), ServiceRequest()) == "fallback-result"
28+
assert dispatcher["TestOperation"](RequestContext(None), ServiceRequest()) == "fallback-result"
2929

3030

3131
def test_forwarding_fallback_dispatcher_avoid_fallback():
@@ -44,4 +44,4 @@ def test_request_forwarder(_, __) -> ServiceResponse:
4444
# expect a NotImplementedError exception (and not the ServiceException from the fallthrough)
4545
dispatcher = ForwardingFallbackDispatcher(test_provider, test_request_forwarder)
4646
with pytest.raises(NotImplementedError):
47-
dispatcher["TestOperation"](RequestContext(), ServiceRequest())
47+
dispatcher["TestOperation"](RequestContext(None), ServiceRequest())

‎tests/unit/aws/handlers/analytics.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ def test_ignores_requests_without_service(self):
4040
counter = ServiceRequestCounter(service_request_aggregator=aggregator)
4141

4242
chain = HandlerChain([counter])
43-
chain.handle(RequestContext(), Response())
43+
chain.handle(RequestContext(None), Response())
4444

4545
aggregator.start.assert_not_called()
4646
aggregator.add_request.assert_not_called()

‎tests/unit/aws/handlers/service.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -111,8 +111,7 @@ def test_sets_exception_from_error_response(self, service_response_handler_chain
111111
assert context.service_response is None
112112

113113
def test_nothing_set_does_nothing(self, service_response_handler_chain):
114-
context = RequestContext()
115-
context.request = Request("GET", "/_localstack/health")
114+
context = RequestContext(request=Request("GET", "/_localstack/health"))
116115

117116
service_response_handler_chain.handle(context, Response("ok", 200))
118117

0 commit comments

Comments
 (0)