Why is this needed?

GitHub actions/upload-artifact v4 has a racing condition when uploading artifacts with the same artifact name in parallel.

For example, our Lambda Layer CDK artifacts are uploaded on a per region deploy in parallel using the a static output name.

This caused today's release to fail partially - re-running failed jobs served as a workaround.

Which area does this relate to?

No response

Solution

Since it's a race condition on upload, we need a dynamic artifact name e.g., one per region. Then merge these artifacts on download.

Updating to a dynamic name

      - name: Save Layer ARN artifact
        if: ${{ inputs.stage == 'PROD' }}
        uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3 # v4.3.1
        with:
          name: cdk-layer-stack-{{ matrix.region }}
          path: ./layer/cdk-layer-stack/* # NOTE: upload-artifact does not inherit working-directory setting.
          if-no-files-found: error
          retention-days: 1
          overwrite: true   # still needed to allow failed retries

Downloading all artifacts for all regions

      - name: Download CDK layer artifact
        uses: actions/download-artifact@eaceaf801fd36c7dee90939fad912460b18a1ffe # v4.1.2
        with:
          name: cdk-layer-stack
          path: cdk-layer-stack/
          merge-multiple: true

We'll need a testing environment to create a parallel job to test this out.

Acknowledgment

• This request meets Powertools for AWS Lambda (Python) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET