Remove legacy lambda provider #9543

joe4dev · 2023-11-02T19:13:52Z

Companion PR: https://github.com/localstack/localstack-ext/pull/2296

Motivation

Following the deprecation of the old Lambda provider in v2.0, we can finally get rid of the old Lambda provider in v3.0.

Migration to Lambda v2 in our documentation: https://docs.localstack.cloud/user-guide/aws/lambda/#migrating-to-lambda-v2
Discuss post explaining migration details: New Lambda implementation in LocalStack 2.0

Changes

Remove the code of the legacy Lambda provider previously moved into legacy packages in Move lambda v1 to legacy package #9473
Remove deprecated Lambda configuration options
Remove legacy tests
Remove conditions in Lambda tests depending on old vs. new provider
Remove CI testing for the old Lambda provider

Notes

The current implementation for event source listeners relies on the LegacyInvocationResult, which has been re-named and preserved until we re-work event source mappings (e.g., the sqs_event_source_listener still depends on the LegacyInvocationResult and the adapers can be removed post v3).
The current implementation of the DLQ helper relies on the LegacyInvocationException, which has been re-named and preserved until we re-work the DQL util _send_to_dead_letter_queue.

Discussion

localstack/runtime/analytics.py Do we keep the analytics of deprecated variables post 3.0?
- It could help to understand how many users set the option
- It might distort the actual use of the variable because the option is ignored or require additional filtering

coveralls · 2023-11-02T21:28:32Z

coverage: 84.23% (+0.4%) from 83.855%
when pulling 88193c6 on remove-lambda-legacy
into 541593a on release/v3.0.

github-actions · 2023-11-02T21:45:16Z

LocalStack Community integration with Pro

      2 files ±  0       2 suites ±0 1h 0m 35s ⏱️ - 3m 15s
2 302 tests - 23 1 785 ✔️ - 5 517 💤 - 18 0 ❌ ±0
2 303 runs - 23 1 785 ✔️ - 5 518 💤 - 18 0 ❌ ±0

Results for commit 88193c6. ± Comparison against base commit 541593a.

♻️ This comment has been updated with latest results.

localstack/config.py

joe4dev · 2023-11-03T11:08:17Z

localstack/deprecations.py

@@ -242,6 +242,13 @@ def is_affected(self) -> bool:
        "1.4.0",
        "This feature is marked for removal. Please use AWS client API to seed Kinesis streams.",
    ),
+    EnvVarDeprecation(
+        "PROVIDER_OVERRIDE_LAMBDA",


Adding the provider override as a deprecation option helps users cleaning up their configuration

joe4dev · 2023-11-03T11:09:03Z

localstack/runtime/analytics.py

@@ -20,8 +20,10 @@
    "KINESIS_PROVIDER",
    "KMS_PROVIDER",
    "LAMBDA_DOWNLOAD_AWS_LAYERS",
+    # Irrelevant post v3 but intentionally tracked for some time
    "LAMBDA_EXECUTOR",


It might be interesting to know how many users still have legacy variables configured post v3 🤷

can't hurt and we can still turn it off later 👍

joe4dev · 2023-11-03T11:09:54Z

localstack/services/lambda_/event_source_listeners/lambda_legacy.py

@@ -0,0 +1,11 @@
+# TODO: remove this legacy construct when re-working event source mapping.


The event source mapping should be reworked post v3 but I tried to keep this removal PR minimal and avoid functional changes.

joe4dev · 2023-11-03T11:11:36Z

localstack/services/lambda_/invocation/event_manager.py

@@ -37,6 +36,14 @@ def get_sqs_client(function_version, client_config=None):
    ).sqs


+# TODO: remove once DLQ handling is refactored following the removal of the legacy lambda provider


The DLQ utilility should be reworked post v3 but I tried to keep this removal PR minimal and avoid functional changes.

good choice 😁

localstack/services/lambda_/packages.py

joe4dev · 2023-11-03T11:14:42Z

tests/aws/services/lambda_/test_lambda_integration_kinesis.py

@@ -123,12 +119,10 @@ def _send_and_receive_messages():
        # this will fail in november 2286, if this code is still around by then, read this comment and update to 10
        assert int(math.log10(timestamp)) == 9

-    # FIXME remove usage of this config value with 2.0
-    @patch.object(config, "SYNCHRONOUS_KINESIS_EVENTS", False)
+    # TODO: is this test relevant for the new provider without patching SYNCHRONOUS_KINESIS_EVENTS?


❓ Is this test still relevant?

It appears AWS-validated but breaking

The patching of SYNCHRONOUS_KINESIS_EVENTS was only relevant for the old provider

prob not, but let's defer that until later

dominikschubert

LGTM!

Very thorough removal of nearly everything from the legacy provider! Awesome 🚀

dominikschubert · 2023-11-06T18:35:47Z

tests/aws/services/lambda_/test_lambda_integration_kinesis.py

@@ -123,12 +119,10 @@ def _send_and_receive_messages():
        # this will fail in november 2286, if this code is still around by then, read this comment and update to 10
        assert int(math.log10(timestamp)) == 9

-    # FIXME remove usage of this config value with 2.0
-    @patch.object(config, "SYNCHRONOUS_KINESIS_EVENTS", False)
+    # TODO: is this test relevant for the new provider without patching SYNCHRONOUS_KINESIS_EVENTS?


prob not, but let's defer that until later

dominikschubert · 2023-11-06T18:38:16Z

localstack/services/lambda_/invocation/event_manager.py

@@ -37,6 +36,14 @@ def get_sqs_client(function_version, client_config=None):
    ).sqs


+# TODO: remove once DLQ handling is refactored following the removal of the legacy lambda provider


good choice 😁

dominikschubert · 2023-11-06T18:40:02Z

localstack/runtime/analytics.py

@@ -20,8 +20,10 @@
    "KINESIS_PROVIDER",
    "KMS_PROVIDER",
    "LAMBDA_DOWNLOAD_AWS_LAYERS",
+    # Irrelevant post v3 but intentionally tracked for some time
    "LAMBDA_EXECUTOR",


can't hurt and we can still turn it off later 👍

dfangl

LGTM! All those removed lines 🤩

We still need to keep them in the `CONFIG_ENV_VARS` if we still want deprecation messages to be shown in the LocalStack container.

joe4dev added the semver: major Breaking changes which can be included in a major release only label Nov 2, 2023

joe4dev added this to the 3.0 milestone Nov 2, 2023

joe4dev self-assigned this Nov 2, 2023

joe4dev mentioned this pull request Nov 2, 2023

v3 release PR #9467

Merged

20 tasks

localstack-bot force-pushed the release/v3.0 branch from 6dd6b1b to 7b9e2b4 Compare November 3, 2023 05:03

joe4dev force-pushed the remove-lambda-legacy branch from 4f48606 to 912103d Compare November 3, 2023 11:04

joe4dev commented Nov 3, 2023

View reviewed changes

localstack/config.py Show resolved Hide resolved

joe4dev commented Nov 3, 2023

View reviewed changes

localstack/services/lambda_/packages.py Show resolved Hide resolved

joe4dev commented Nov 3, 2023

View reviewed changes

alexrashed force-pushed the release/v3.0 branch 2 times, most recently from fb5da7d to 99278b3 Compare November 4, 2023 13:46

joe4dev force-pushed the remove-lambda-legacy branch 2 times, most recently from 7850f27 to 7494c20 Compare November 6, 2023 12:38

joe4dev marked this pull request as ready for review November 6, 2023 12:43

joe4dev requested review from dominikschubert, MEPalma, dfangl, sannya-singal, silv-io, pinzon, simonrw, calvernaz, bentsku and thrau as code owners November 6, 2023 12:43

joe4dev requested a review from HarshCasper as a code owner November 6, 2023 12:43

joe4dev mentioned this pull request Nov 6, 2023

Migrate deprecated edge variables #9405

Merged

5 tasks

dominikschubert approved these changes Nov 6, 2023

View reviewed changes

dfangl approved these changes Nov 6, 2023

View reviewed changes

dominikschubert requested review from dfangl and removed request for simonrw, calvernaz, alexrashed, thrau, silv-io, pinzon, sannya-singal, bentsku, HarshCasper and MEPalma November 6, 2023 18:53

dfangl approved these changes Nov 6, 2023

View reviewed changes

joe4dev added 4 commits November 6, 2023 20:53

Remove legacy lambda provider

8a1a04e

Fix skip for not yet implemented concurrency alias test

304d550

Add removed section to LocalStack config

2a394bc

We still need to keep them in the `CONFIG_ENV_VARS` if we still want deprecation messages to be shown in the LocalStack container.

Re-introduce HOSTNAME_FOR_LAMBDA

88193c6

joe4dev force-pushed the remove-lambda-legacy branch from 7494c20 to 88193c6 Compare November 6, 2023 19:55

joe4dev merged commit e2b109b into release/v3.0 Nov 6, 2023

joe4dev deleted the remove-lambda-legacy branch November 6, 2023 22:25

localstack-bot pushed a commit that referenced this pull request Nov 7, 2023

Remove legacy lambda provider (#9543)

df5ef05

alexrashed pushed a commit that referenced this pull request Nov 7, 2023

Remove legacy lambda provider (#9543)

a1aff5d

alexrashed pushed a commit that referenced this pull request Nov 7, 2023

Remove legacy lambda provider (#9543)

3c279fe

localstack-bot pushed a commit that referenced this pull request Nov 9, 2023

Remove legacy lambda provider (#9543)

537c5b5

giograno mentioned this pull request Nov 9, 2023

remove obsolete aws components #9583

Merged

alexrashed pushed a commit that referenced this pull request Nov 9, 2023

Remove legacy lambda provider (#9543)

20a609a

alexrashed pushed a commit that referenced this pull request Nov 9, 2023

Remove legacy lambda provider (#9543)

82fe8be

		@@ -0,0 +1,11 @@
		# TODO: remove this legacy construct when re-working event source mapping.

		@@ -37,6 +36,14 @@ def get_sqs_client(function_version, client_config=None):
		).sqs


		# TODO: remove once DLQ handling is refactored following the removal of the legacy lambda provider

Uh oh!

Remove legacy lambda provider #9543

Remove legacy lambda provider #9543

Uh oh!

Conversation

joe4dev commented Nov 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Notes

Discussion

Uh oh!

coveralls commented Nov 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

LocalStack Community integration with Pro

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joe4dev Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dominikschubert left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dfangl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

joe4dev commented Nov 2, 2023 •

edited

Loading

coveralls commented Nov 2, 2023 •

edited

Loading

github-actions bot commented Nov 2, 2023 •

edited

Loading

joe4dev Nov 3, 2023 •

edited

Loading