Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix: Strip leading ./ in S3 key if artifacts_dir is set to something like ${path.root}/mypath/ #168

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

pdecat
Copy link
Contributor

@pdecat pdecat commented Jul 1, 2021

Description

This change strips the leading ./ in S3 key if artifacts_dir is set to something like ${path.root}/mypath/.

Motivation and Context

Without this change, this configuration:

module "lambda-my-layer" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "2.4.0"

  create_layer = true
  layer_name   = "my-layer"

  runtime         = "python3.8"
  build_in_docker = true

  source_path = [{
    path             = "${path.module}/lambda/my-layer/",
    pip_requirements = true,
    prefix_in_zip    = "python",
  }]

  artifacts_dir = "${path.root}/.terraform/lambda-builds/"

  store_on_s3             = true
  s3_bucket               = aws_s3_bucket.mybuckt.bucket
  s3_object_storage_class = "STANDARD"
}

gives the following plan excerpt:

  # module.lambda-my-layer.aws_lambda_layer_version.this[0] will be created
  + resource "aws_lambda_layer_version" "this" {
      + arn                         = (known after apply)
      + compatible_runtimes         = [
          + "python3.8",
        ]
      + created_date                = (known after apply)
      + id                          = (known after apply)
      + layer_arn                   = (known after apply)
      + layer_name                  = "my-layer"
      + s3_bucket                   = "s3-my-lambdas-eu-west-3-0123456789012"
      + s3_key                      = "./.terraform/lambda-builds/41c9284df0f8fadfa3437b7a70066400d35ca912bfb089b7d3e01b59870dd077.zip"
      + s3_object_version           = (known after apply)
      + signing_job_arn             = (known after apply)
      + signing_profile_version_arn = (known after apply)
      + source_code_hash            = (known after apply)
      + source_code_size            = (known after apply)
      + version                     = (known after apply)
    }

Which fails at apply:

│ Error: Error creating lambda layer: InvalidParameterValueException: Error occurred while GetObjectVersion. S3 Error Code: NoSuchVersion. S3 Error Message: The specified version does not exist.
│ {
│   RespMetadata: {
│     StatusCode: 400,
│     RequestID: "18230ee0-2112-45d9-850f-0123456789012"
│   },
│   Message_: "Error occurred while GetObjectVersion. S3 Error Code: NoSuchVersion. S3 Error Message: The specified version does not exist.",
│   Type: "User"
│ }
│
│   with module.lambda-cleanup-layer.aws_lambda_layer_version.this[0],
│   on .terraform/modules/lambda-cleanup-layer/main.tf line 93, in resource "aws_lambda_layer_version" "this":
│   93: resource "aws_lambda_layer_version" "this" {

With this change, this part of the plan:

  # module.lambda-my-layer.aws_lambda_layer_version.this[0] will be created
  + resource "aws_lambda_layer_version" "this" {
      + arn                         = (known after apply)
      + compatible_runtimes         = [
          + "python3.8",
        ]
      + created_date                = (known after apply)
      + id                          = (known after apply)
      + layer_arn                   = (known after apply)
      + layer_name                  = "my-layer"
      + s3_bucket                   = "s3-my-lambdas-eu-west-3-0123456789012"
      + s3_key                      = ".terraform/lambda-builds/41c9284df0f8fadfa3437b7a70066400d35ca912bfb089b7d3e01b59870dd077.zip"
      + s3_object_version           = (known after apply)
      + signing_job_arn             = (known after apply)
      + signing_profile_version_arn = (known after apply)
      + source_code_hash            = (known after apply)
      + source_code_size            = (known after apply)
      + version                     = (known after apply)
    }

applies successfully.

Breaking Changes

This should not be a breaking change as keys with a leading ./ are not accepted by S3.

How Has This Been Tested?

  • I have tested and validated these changes using one or more of the provided examples/* projects

Tested with examples/complete with build_in_docker = true added as I only have python3.9 locally: terraform-aws-lambda-168.log

I have also tested this change with my own terraform configurations.

@pdecat pdecat force-pushed the s3_strip_leading_dot_slash_in_key branch from 3cde7da to 1cca5a2 Compare July 15, 2021 11:02
@pdecat pdecat changed the title Strip leading ./ in S3 key if artifacts_dir is set to something like ${path.root}/mypath/ fix: Strip leading ./ in S3 key if artifacts_dir is set to something like ${path.root}/mypath/ Jul 22, 2021
@pdecat pdecat force-pushed the s3_strip_leading_dot_slash_in_key branch from 1cca5a2 to 9212242 Compare July 22, 2021 12:07
…like `${path.root}/.terraform/lambda-builds/`
@pdecat pdecat force-pushed the s3_strip_leading_dot_slash_in_key branch from 8f7c1a6 to c94df78 Compare August 21, 2021 07:41
@@ -6,7 +6,7 @@ locals {

# s3_* - to get package from S3
s3_bucket = var.s3_existing_package != null ? lookup(var.s3_existing_package, "bucket", null) : (var.store_on_s3 ? var.s3_bucket : null)
s3_key = var.s3_existing_package != null ? lookup(var.s3_existing_package, "key", null) : (var.store_on_s3 ? element(concat(data.external.archive_prepare.*.result.filename, [null]), 0) : null)
s3_key = var.s3_existing_package != null ? lookup(var.s3_existing_package, "key", null) : (var.store_on_s3 ? format("%s/%s", dirname(element(concat(data.external.archive_prepare.*.result.filename, [null]), 0)), basename(element(concat(data.external.archive_prepare.*.result.filename, [null]), 0))) : null)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, using dirname() and basename() is probably going to cause issues on Windows...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it probably will not work there. I am going to close this PR.

Copy link
Contributor Author

@pdecat pdecat Aug 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@antonbabenko I changed the method to strip the leading ./ and opened a new PR: #191

@pdecat pdecat mentioned this pull request Aug 23, 2021
1 task
@github-actions
Copy link

github-actions bot commented Nov 9, 2022

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 9, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants