Update lambda runtime deprecations and fix Lambda ARM builds #10441
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,8 +7,10 @@ | |
| from pathlib import Path | ||
| from typing import TYPE_CHECKING, Literal, Mapping, Optional, Sequence, overload | ||
|
|
||
| from localstack import config | ||
| from localstack.services.lambda_.runtimes import RUNTIMES_AGGREGATED | ||
| from localstack.utils.files import load_file | ||
| from localstack.utils.platform import Arch, get_arch | ||
| from localstack.utils.strings import short_uid | ||
| from localstack.utils.sync import ShortCircuitWaitException, retry | ||
| from localstack.utils.testutil import get_lambda_log_events | ||
|
|
@@ -35,21 +37,18 @@ | |
| **dict.fromkeys(RUNTIMES_AGGREGATED.get("nodejs"), "index.handler"), | ||
| **dict.fromkeys(RUNTIMES_AGGREGATED.get("ruby"), "function.handler"), | ||
| **dict.fromkeys(RUNTIMES_AGGREGATED.get("java"), "echo.Handler"), | ||
| # The handler value does not matter unless the custom runtime reads it in some way but it is a required field. | ||
| **dict.fromkeys(RUNTIMES_AGGREGATED.get("provided"), "function.handler"), | ||
| "dotnet6": "dotnet6::dotnet6.Function::FunctionHandler", # TODO lets see if we can accumulate those | ||
| } | ||
|
|
||
| PACKAGE_FOR_RUNTIME = { | ||
| **dict.fromkeys(RUNTIMES_AGGREGATED.get("python"), "python"), | ||
| **dict.fromkeys(RUNTIMES_AGGREGATED.get("nodejs"), "nodejs"), | ||
| **dict.fromkeys(RUNTIMES_AGGREGATED.get("ruby"), "ruby"), | ||
| **dict.fromkeys(RUNTIMES_AGGREGATED.get("java"), "java"), | ||
| **dict.fromkeys(RUNTIMES_AGGREGATED.get("provided"), "provided"), | ||
| "dotnet6": "dotnet6", | ||
| } | ||
|
|
||
|
|
||
|
|
@@ -94,7 +93,7 @@ def package_for_lang(scenario: str, runtime: str, root_folder: Path) -> str: | |
| generic_runtime_dir_candidate = scenario_dir / runtime_folder | ||
|
|
||
| # if a more specific folder exists, use that one | ||
| # otherwise: try to fall back to generic runtime (e.g. python for python3.12) | ||
| if runtime_dir_candidate.exists() and runtime_dir_candidate.is_dir(): | ||
| runtime_dir = runtime_dir_candidate | ||
| else: | ||
|
|
@@ -109,7 +108,19 @@ def package_for_lang(scenario: str, runtime: str, root_folder: Path) -> str: | |
| return package_path | ||
|
|
||
| # packaging | ||
|
There was a problem hiding this comment. L90 above: We could consider renaming |
||
| # Use the default Lambda architecture x86_64 unless the ignore architecture flag is configured. | ||
| # This enables local testing of both architectures on multi-architecture platforms such as Apple Silicon machines. | ||
| architecture = "x86_64" | ||
| if config.LAMBDA_IGNORE_ARCHITECTURE: | ||
| architecture = "arm64" if get_arch() == Arch.arm64 else "x86_64" | ||
| build_cmd = ["make", "build", f"ARCHITECTURE={architecture}"] | ||
| LOG.debug( | ||
| "Building Lambda function for scenario %s and runtime %s using %s.", | ||
| scenario, | ||
| runtime, | ||
| " ".join(build_cmd), | ||
| ) | ||
| result = subprocess.run(build_cmd, cwd=runtime_dir) | ||
| if result.returncode != 0: | ||
| raise Exception( | ||
| f"Failed to build multiruntime {scenario=} for {runtime=} with error code: {result.returncode}" | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # Top-level Makefile to invoke all make targets in sub-directories | ||
|
|
||
| # Based on https://stackoverflow.com/a/72209214/6875981 | ||
| SUBDIRS := $(patsubst %/,%,$(wildcard */)) | ||
|
|
||
|
|
||
| .PHONY: all $(MAKECMDGOALS) $(SUBDIRS) | ||
| $(MAKECMDGOALS) all: $(SUBDIRS) | ||
|
|
||
| $(SUBDIRS): | ||
| $(MAKE) -C $@ $(MAKECMDGOALS) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| # Lambda Multiruntime Builds | ||
|
|
||
| This directory contains the source code and build instructions for Lambda multiruntime tests. | ||
| Example tests are available under `tests.aws.lambda_.functions.common` | ||
|
|
||
| Each scenario (e.g., "echo") has the following folder structure: `./<scenario>/runtime/` | ||
| A runtime can be an aggregated runtime defined in `runtimes.py` (e.g., "python") or | ||
| a specific runtime (e.g., "python3.12") if customizations are required. | ||
|
|
||
| Each runtime directory defines a `Makefile` that | ||
| * MUST define a `build` target that: | ||
| * a) creates a `build` directory containing all Lambda sources ready for packaging | ||
| * b) creates a `handler.zip` file with a Lambda deployment package | ||
| * SHOULD define an `ARCHITECTURE` parameter to overwrite the target architecture (i.e., `x86_64` or `arm64`) | ||
| if architecture-specific builds are required (e.g., Dotnet, Golang, Rust). | ||
| * By default, the Makefile should build a deployment package with the same architecture as the host. | ||
|
There was a problem hiding this comment. This default behavior is debatable:
Hence, on multi-architecture platforms (e.g., Apple Silicon), x86_64 builds require a parameter |
||
| * However, for testing on multi-architecture platforms, we should be able to overwrite the `ARCHITECTURE` parameter. | ||
| * We need to standardize `uname -m` into `ARCHITECTURE` because the output differs depending on the platform. | ||
| Ubuntu yields `aarch64` and macOS yields `arm64`. | ||
| * If we want to support dev systems without the `uname` utility, we could add `|| echo x86_64` to the uname detection. | ||
| * SHOULD define a `clean` target that deletes any build artefacts, including the `handler.zip`. | ||
| This helps a lot during development to tidy up and invalidate caching. | ||
|
|
||
| Checkout the [AWS guides](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-package.html) on | ||
| "Building with <language>" (e.g., "Building with Java") for instructions how to | ||
| build Lambda deployment packages correctly. | ||
|
|
||
| The top-level and intermediary directories provided a meta-Makefile that automatically invokes sub-Makefiles such that | ||
| we can run `make clean` at the top-level recursively. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # Top-level Makefile to invoke all make targets in sub-directories | ||
|
|
||
| # Based on https://stackoverflow.com/a/72209214/6875981 | ||
| SUBDIRS := $(patsubst %/,%,$(wildcard */)) | ||
|
|
||
| .PHONY: all $(MAKECMDGOALS) $(SUBDIRS) | ||
| $(MAKECMDGOALS) all: $(SUBDIRS) | ||
|
|
||
| $(SUBDIRS): | ||
| $(MAKE) -C $@ $(MAKECMDGOALS) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,3 @@ | ||
| build: | ||
| mkdir -p build && \ | ||
| cp -r ./src/* build/ | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,3 @@ | ||
| build: | ||
| mkdir build && \ | ||
| cp -r ./src/* build/ | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # Top-level Makefile to invoke all make targets in sub-directories | ||
|
|
||
| # Based on https://stackoverflow.com/a/72209214/6875981 | ||
| SUBDIRS := $(patsubst %/,%,$(wildcard */)) | ||
|
|
||
| .PHONY: all $(MAKECMDGOALS) $(SUBDIRS) | ||
| $(MAKECMDGOALS) all: $(SUBDIRS) | ||
|
|
||
| $(SUBDIRS): | ||
| $(MAKE) -C $@ $(MAKECMDGOALS) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be removed with the follow-up addition in the Dotnet8 PR