Thanks to visit codestin.com
Credit goes to github.com

Skip to content

bug: invoking lambda function created with python3.9 runtime fails while python3.8 works #4725

New issue
New issue
Closed
Closed
bug: invoking lambda function created with python3.9 runtime fails while python3.8 works#4725
Assignees
dfangl
Labels
aws:lambdaAWS Lambdastatus: triage neededRequires evaluation by maintainerstype: bugBug report
@horatio-sans-serif

Description

@horatio-sans-serif
horatio-sans-serif
opened on Oct 12, 2021

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

Invoking a lambda function created with runtime python3.9 fails. Simply changing to runtime python3.8 does not.

Expected Behavior

I expect to be able to use runtime python3.9

How are you starting LocalStack?

With the localstack script

Steps To Reproduce

How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)

ENFORCE_IAM=1 SERVICES=sqs,lambda,s3,iam LAMBDA_REMOTE_DOCKER=0 localstack start

Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)

awslocal lambda create-function --function-name check1 --code S3Bucket="__local__",S3Key="$PWD" --handler check1.lambda_handler --runtime python3.9 --role all

awslocal lambda invoke --function-name check1 --payload '{"check": "xxx"}' output.txt

I am running Docker for Mac.

docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.3)
  compose: Docker Compose (Docker Inc., v2.0.0)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 2
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.47-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 3
 Total Memory: 1.939GiB
 Name: docker-desktop
 ID: KTAF:6A64:NFBM:S42W:RKGN:QEJO:G7MP:JUJI:IH2H:7X44:6YA3:DBYZ
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

python -V
3.9.2

ENFORCE_IAM=1 SERVICES=sqs,lambda,s3,iam LAMBDA_REMOTE_DOCKER=0 localstack start

awslocal lambda create-function --function-name check1 --code S3Bucket="__local__",S3Key="$PWD" --handler check1.lambda_handler --runtime python3.9 --role all

{
    "FunctionName": "check1",
    "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:check1",
    "Runtime": "python3.9",
    "Role": "all",
    "Handler": "check1.lambda_handler",
    "Description": "",
    "Timeout": 3,
 45
    "LastModified": "2021-10-12T14:44:39.494+0000",
    "Version": "$LATEST",
    "VpcConfig": {},
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "RevisionId": "2f96f387-4f98-4769-8d06-f858be34d82b",
    "State": "Active",
    "LastUpdateStatus": "Successful",
    "PackageType": "Zip"
}

cat check1.py

import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)


def lambda_handler(event, _):
    logger.info('Event: %s', event)
    response = {'result': 'hello, world'}
    return response

awslocal lambda invoke --function-name check1 --payload '{"check": "xxx"}' output.txt

{
    "StatusCode": 200,
    "FunctionError": "Unhandled",
    "LogResult": "",
    "ExecutedVersion": "$LATEST"
}

cat output.txt
{"errorMessage":"Lambda process returned with error. Result: . Output:\n","errorType":"InvocationException","stackTrace":["  File \"/opt/code/localstack/localstack/services/awslambda/lambda_api.py\", line 811, in run_lambda\n    lock_discriminator=lock_discriminator,\n","  File \"/opt/code/localstack/localstack/services/awslambda/lambda_executors.py\", line 428, in execute\n    return do_execute()\n","  File \"/opt/code/localstack/localstack/services/awslambda/lambda_executors.py\", line 418, in do_execute\n    return _run(func_arn=func_arn)\n","  File \"/opt/code/localstack/localstack/utils/cloudwatch/cloudwatch_util.py\", line 157, in wrapped\n    raise e\n","  File \"/opt/code/localstack/localstack/utils/cloudwatch/cloudwatch_util.py\", line 153, in wrapped\n    result = func(*args, **kwargs)\n","  File \"/opt/code/localstack/localstack/services/awslambda/lambda_executors.py\", line 405, in _run\n    raise e\n","  File \"/opt/code/localstack/localstack/services/awslambda/lambda_executors.py\", line 401, in _run\n    result = self._execute(lambda_function, inv_context)\n","  File \"/opt/code/localstack/localstack/services/awslambda/lambda_executors.py\", line 709, in _execute\n    result = self.run_lambda_executor(lambda_function=lambda_function, inv_context=inv_context)\n","  File \"/opt/code/localstack/localstack/services/awslambda/lambda_executors.py\", line 637, in run_lambda_executor\n    ) from error\n"]}

Nothing really compelling in the above!

Stop localstack and try again with python3.8 instead of python3.9

ENFORCE_IAM=1 SERVICES=sqs,lambda,s3,iam LAMBDA_REMOTE_DOCKER=0 localstack start

awslocal lambda create-function --function-name check1 --code S3Bucket="__local__",S3Key="$PWD" --handler check1.lambda_handler --runtime python3.8 --role all

awslocal lambda invoke --function-name check1 --payload '{"action": "square", "number": 3}' output.txt

{
    "StatusCode": 200,
    "LogResult": "",
    "ExecutedVersion": "$LATEST"
}

cat output.txt
{"result":"hello, world"}

Environment

- OS: Mac  OS X Big Sur.
- LocalStack: 0.12.18.4

Anything else?

requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/create?tag=python3.9&fromImage=lambci%2Flambda

[10:44:25] starting LocalStack in Docker mode 🐳                                                                                                              localstack.py:106
──────────────────────────────────────────────────────────────── LocalStack Runtime Log (press CTRL-C to quit) ────────────────────────────────────────────────────────────────
['docker', 'run', '-it', '-e', 'LAMBDA_REMOTE_DOCKER=0', '-e', 'DEFAULT_REGION=us-east-1', '-e', 'SERVICES=sqs,lambda,s3,iam', '-e', 'TEST_AWS_ACCOUNT_ID=000000000000', '-e', 'LOCALSTACK_HOSTNAME=localhost', '-e', 'DOCKER_HOST=unix:///var/run/docker.sock', '-e', 'HOST_TMP_FOLDER=/private/var/folders/6t/nlds8_g97tx34j7j51ntr4100000gn/T/localstack', '--rm', '--privileged', '--name', 'localstack_main', '-p', '127.0.0.1:4566:4566', '-v', '/private/var/folders/6t/nlds8_g97tx34j7j51ntr4100000gn/T/localstack:/tmp/localstack', '-v', '/var/run/docker.sock:/var/run/docker.sock', 'localstack/localstack']
Waiting for all LocalStack services to be ready
2021-10-12 14:44:26,048 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2021-10-12 14:44:26,050 INFO supervisord started with pid 16
2021-10-12 14:44:27,053 INFO spawned: 'infra' with pid 22
(. .venv/bin/activate; exec bin/localstack start --host --no-banner)

LocalStack version: 0.12.18
LocalStack Docker container id: a38a1ded3d4a
LocalStack build date: 2021-10-11
LocalStack build git hash: 85c712e5

2021-10-12 14:44:28,900 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Starting edge router (https port 4566)...
Starting mock CloudWatch service on http port 4566 ...
2021-10-12T14:44:29:INFO:bootstrap.py: Execution of "load_plugin_from_path" took 1772.85ms
2021-10-12T14:44:29:INFO:bootstrap.py: Execution of "load_plugins" took 1773.43ms
2021-10-12T14:44:29:INFO:localstack.multiserver: Starting multi API server process on port 47607
Starting mock IAM service on http port 4566 ...
Starting mock Lambda service on http port 4566 ...
Starting mock CloudWatch Logs service on http port 4566 ...
Starting mock S3 service on http port 4566 ...
Starting mock SQS service on http port 4566 ...
[2021-10-12 14:44:29 +0000] [23] [INFO] Running on http://0.0.0.0:47607 (CTRL + C to quit)
[2021-10-12 14:44:29 +0000] [23] [INFO] Running on https://0.0.0.0:4566 (CTRL + C to quit)
2021-10-12T14:44:29:INFO:hypercorn.error: Running on https://0.0.0.0:4566 (CTRL + C to quit)
2021-10-12T14:44:29:INFO:hypercorn.error: Running on http://0.0.0.0:47607 (CTRL + C to quit)
Waiting for all LocalStack services to be ready
Ready.
2021-10-12T14:44:35:INFO:bootstrap.py: Execution of "start_api_services" took 5379.26ms
2021-10-12T14:44:44:INFO:localstack.services.awslambda.lambda_utils: Determined main container target IP: 172.17.0.2
2021-10-12T14:44:44:INFO:localstack.services.awslambda.lambda_executors: Running lambda: arn:aws:lambda:us-east-1:000000000000:function:chec1
2021-10-12T14:44:46:INFO:localstack.services.awslambda.lambda_api: Error executing Lambda function arn:aws:lambda:us-east-1:000000000000:function:check1: Lambda process returned with error. Result: . Output:
 Traceback (most recent call last):
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/create

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/code/localstack/localstack/utils/docker.py", line 1325, in create_container
    container = create_container()
  File "/opt/code/localstack/localstack/utils/docker.py", line 1321, in create_container
    **kwargs,
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/docker/models/containers.py", line 870, in create
    resp = self.client.api.create_container(**create_kwargs)
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/docker/api/container.py", line 430, in create_container
    return self.create_container_from_config(config, name)
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/docker/api/container.py", line 441, in create_container_from_config
    return self._result(res, True)
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error for http+docker://localhost/v1.41/containers/create: Not Found ("No such image: lambci/lambda:python3.9")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/images/create?tag=python3.9&fromImage=lambci%2Flambda

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/code/localstack/localstack/utils/docker.py", line 1140, in pull_image
    self.client().images.pull(repository, tag)
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/docker/models/images.py", line 447, in pull
    repository, tag=tag, stream=True, all_tags=all_tags, **kwargs
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/docker/api/image.py", line 430, in pull
    self._raise_for_status(response)
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/opt/code/localstack/.venv/lib/python3.7/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.41/images/create?tag=python3.9&fromImage=lambci%2Flambda: Not Found ("manifest for lambci/lambda:python3.9 not found: manifest unknown: manifest unknown")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/code/localstack/localstack/services/awslambda/lambda_executors.py", line 598, in run_lambda_executor
    stdin=event_stdin_bytes,
  File "/opt/code/localstack/localstack/services/awslambda/lambda_executors.py", line 1109, in execute_in_container
    stdin=stdin,
  File "/opt/code/localstack/localstack/utils/docker.py", line 1377, in run_container
    workdir=workdir,
  File "/opt/code/localstack/localstack/utils/docker.py", line 1327, in create_container
    self.pull_image(image_name)
  File "/opt/code/localstack/localstack/utils/docker.py", line 1144, in pull_image
    raise ContainerException()
localstack.utils.docker.ContainerException

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/code/localstack/localstack/services/awslambda/lambda_api.py", line 811, in run_lambda
    lock_discriminator=lock_discriminator,
  File "/opt/code/localstack/localstack/services/awslambda/lambda_executors.py", line 428, in execute
    return do_execute()
  File "/opt/code/localstack/localstack/services/awslambda/lambda_executors.py", line 418, in do_execute
    return _run(func_arn=func_arn)
  File "/opt/code/localstack/localstack/utils/cloudwatch/cloudwatch_util.py", line 157, in wrapped
    raise e
  File "/opt/code/localstack/localstack/utils/cloudwatch/cloudwatch_util.py", line 153, in wrapped
    result = func(*args, **kwargs)
  File "/opt/code/localstack/localstack/services/awslambda/lambda_executors.py", line 405, in _run
    raise e
  File "/opt/code/localstack/localstack/services/awslambda/lambda_executors.py", line 401, in _run
    result = self._execute(lambda_function, inv_context)

Metadata

Metadata

Assignees

Labels

aws:lambdaAWS Lambdastatus: triage neededRequires evaluation by maintainerstype: bugBug report

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions