Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Run e2e tests on each PR. #584

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 51 commits into
base: main
Choose a base branch
from
Open

Run e2e tests on each PR. #584

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
f19f26c
Always publish to us-east-1.
purple4reina Apr 10, 2025
abfb1b6
WIP
purple4reina Apr 24, 2025
2b283fd
Publishes to us-west-2.
purple4reina Jun 13, 2025
919adcf
Fix needs and update things.
purple4reina Jun 16, 2025
937d480
Use underscore.
purple4reina Jun 17, 2025
38c126f
No dot.
purple4reina Jun 17, 2025
9e62357
Correct replace.
purple4reina Jun 17, 2025
c59ba03
Use variables.
purple4reina Jun 17, 2025
88a298c
Properly get arn.
purple4reina Jun 17, 2025
4eaf0fa
Just Trim.
purple4reina Jun 17, 2025
dd9a830
Multiline string.
purple4reina Jun 17, 2025
1c6a7e3
Change to template comment.
purple4reina Jun 17, 2025
cff5b21
No more multiline.
purple4reina Jun 17, 2025
e73e4fe
Remove comment?
purple4reina Jun 17, 2025
547c054
Combine lines again.
purple4reina Jun 17, 2025
03cc647
Move comment.
purple4reina Jun 17, 2025
988f280
Another multiline.
purple4reina Jun 17, 2025
b47528b
Exit if not found.
purple4reina Jun 17, 2025
fb4d6d8
Set -x.
purple4reina Jun 17, 2025
6356961
More debugging.
purple4reina Jun 17, 2025
30e7960
Even more debugging.
purple4reina Jun 17, 2025
ed3cd64
Broaden regex.
purple4reina Jun 17, 2025
5b8d616
Subscript.
purple4reina Jun 17, 2025
ed82d35
No more subscript.
purple4reina Jun 17, 2025
5acf912
Artifacts true.
purple4reina Jun 17, 2025
bdd8a39
Inherit dependencies and variables.
purple4reina Jun 17, 2025
6707673
Add testing downstream branch.
purple4reina Jun 17, 2025
11ebe29
Remove dependencies use variables.
purple4reina Jun 17, 2025
e914d4c
Use loop and main.
purple4reina Jun 17, 2025
353ad17
Uncomment other tests.
purple4reina Jun 17, 2025
7676cc9
Cleanup version env vars.
purple4reina Jun 17, 2025
bd8f4d9
Whitespace cleanups.
purple4reina Jun 18, 2025
0ae5a4f
Only deploy layers on success.
purple4reina Jun 18, 2025
b159dcf
Add e2e-status check.
purple4reina Jun 18, 2025
0c3f604
Improve output of job.
purple4reina Jun 20, 2025
7255a2b
Needing publish loads dotenv artifact.
purple4reina Jun 20, 2025
d158eb0
Have e2e-status job start right away then poll.
purple4reina Jun 20, 2025
afda470
Comment out tests for now.
purple4reina Jun 20, 2025
b2b1ba5
Use apk.
purple4reina Jun 20, 2025
92607f2
Debugging.
purple4reina Jun 20, 2025
a0c1a76
Try JOB-TOKEN.
purple4reina Jun 20, 2025
6924947
Print out URL.
purple4reina Jun 20, 2025
dfdc510
Try bridges api.
purple4reina Jun 20, 2025
20a2b56
Try GITLAB_API_TOKEN.
purple4reina Jun 20, 2025
75357ef
Use CI_JOB_STATUS to determine pass/fail.
purple4reina Jun 20, 2025
57056ba
Remove switch.
purple4reina Jun 21, 2025
6b9af41
Create dotenv file from publish script.
purple4reina Jun 30, 2025
e78f44e
Use docker docker image.
purple4reina Jun 30, 2025
cb0bdee
This needs not needed.
purple4reina Jun 30, 2025
59bee4c
Push check status to file in e2e repo.
purple4reina Jun 30, 2025
648de9a
Start status checking in test stage.
purple4reina Jun 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 44 additions & 6 deletions ci/input_files/build.yaml.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
{{- $e2e_region := "us-west-2" -}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a variable declaration? If so, why not use an environment variable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is just a variable declaration. I hadn't thought of just using an environment variable. I was thinking that this way, the value is just built into the final yaml file, instead of being resolved later, at ci execution time. What are your thoughts on that?

Ah, okay, so one of the places this variable is used is in a needs block. According to gitlab docs, variables are not available in needs blocks.


stages:
- build
- test
- sign
- publish
- e2e
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e2e needs publish which needs test


.python-before-script: &python-before-script
- pip install virtualenv
Expand Down Expand Up @@ -56,11 +59,11 @@ check-layer-size ({{ $runtime.name }}-{{ $runtime.arch }}):
stage: test
tags: ["arch:amd64"]
image: registry.ddbuild.io/images/docker:20.10
needs:
needs:
- build-layer ({{ $runtime.name }}-{{ $runtime.arch }})
dependencies:
- build-layer ({{ $runtime.name }}-{{ $runtime.arch }})
script:
script:
- PYTHON_VERSION={{ $runtime.python_version }} ARCH={{ $runtime.arch }} ./scripts/check_layer_size.sh

lint python:
Expand All @@ -69,7 +72,7 @@ lint python:
image: registry.ddbuild.io/images/mirror/python:{{ $runtime.image }}
cache: &{{ $runtime.name }}-{{ $runtime.arch }}-cache
before_script: *python-before-script
script:
script:
- source venv/bin/activate
- ./scripts/check_format.sh

Expand All @@ -79,15 +82,15 @@ unit-test ({{ $runtime.name }}-{{ $runtime.arch }}):
image: registry.ddbuild.io/images/mirror/python:{{ $runtime.image }}
cache: &{{ $runtime.name }}-{{ $runtime.arch }}-cache
before_script: *python-before-script
script:
script:
- source venv/bin/activate
- pytest -vv

integration-test ({{ $runtime.name }}-{{ $runtime.arch }}):
stage: test
tags: ["arch:amd64"]
image: registry.ddbuild.io/images/docker:20.10-py3
needs:
needs:
- build-layer ({{ $runtime.name }}-{{ $runtime.arch }})
dependencies:
- build-layer ({{ $runtime.name }}-{{ $runtime.arch }})
Expand Down Expand Up @@ -132,16 +135,22 @@ sign-layer ({{ $runtime.name }}-{{ $runtime.arch }}):
- LAYER_FILE=datadog_lambda_py-{{ $runtime.arch}}-{{ $runtime.python_version }}.zip ./scripts/sign_layers.sh prod

{{ range $environment_name, $environment := (ds "environments").environments }}
{{ $dotenv := print $runtime.name "_" $runtime.arch "_" $environment_name ".env" }}

publish-layer-{{ $environment_name }} ({{ $runtime.name }}-{{ $runtime.arch }}):
stage: publish
tags: ["arch:amd64"]
image: registry.ddbuild.io/images/docker:20.10-py3
rules:
- if: '"{{ $environment_name }}" == "sandbox" && $REGION == "{{ $e2e_region }}" && "{{ $runtime.arch }}" == "amd64"'
when: on_success
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always be sure to publish to the sandbox account in us-west-2 for e2e testing.

- if: '"{{ $environment_name }}" == "sandbox"'
when: manual
allow_failure: true
- if: '$CI_COMMIT_TAG =~ /^v.*/'
artifacts:
reports:
dotenv: {{ $dotenv }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When saved as a dotenv report, this file will get sourced for its env vars in every downstream job.

needs:
{{ if or (eq $environment_name "prod") }}
- sign-layer ({{ $runtime.name }}-{{ $runtime.arch}})
Expand All @@ -166,7 +175,7 @@ publish-layer-{{ $environment_name }} ({{ $runtime.name }}-{{ $runtime.arch }}):
before_script:
- EXTERNAL_ID_NAME={{ $environment.external_id }} ROLE_TO_ASSUME={{ $environment.role_to_assume }} AWS_ACCOUNT={{ $environment.account }} source ./ci/get_secrets.sh
script:
- STAGE={{ $environment_name }} PYTHON_VERSION={{ $runtime.python_version }} ARCH={{ $runtime.arch }} ./ci/publish_layers.sh
- STAGE={{ $environment_name }} PYTHON_VERSION={{ $runtime.python_version }} ARCH={{ $runtime.arch }} DOTENV={{ $dotenv }} ./ci/publish_layers.sh

{{- end }}

Expand Down Expand Up @@ -232,3 +241,32 @@ signed layer bundle:
- rm -rf datadog_lambda_py-signed-bundle-${CI_JOB_ID}
- mkdir -p datadog_lambda_py-signed-bundle-${CI_JOB_ID}
- cp .layers/datadog_lambda_py-*.zip datadog_lambda_py-signed-bundle-${CI_JOB_ID}

e2e-test:
stage: e2e
trigger:
project: DataDog/serverless-e2e-tests
strategy: depend
variables:
LANGUAGES_SUBSET: python
# These env vars are inherited from the dotenv reports of the publish-layer jobs
{{- range (ds "runtimes").runtimes }}
{{- if eq .arch "amd64" }}
{{- $version := print (.name | strings.Trim "python") }}
PYTHON_{{ $version }}_VERSION: $PYTHON_{{ $version }}_VERSION
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explicitly pass environment variables from the sourced publish job dotenv file to the downstream job. These env vars are not passed downstream otherwise.

{{- end }}
{{- end }}
needs: {{ range (ds "runtimes").runtimes }}
{{- if eq .arch "amd64" }}
- "publish-layer-sandbox ({{ .name }}-{{ .arch }}): [{{ $e2e_region }}]"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Run after all the publish jobs have completed.

{{- end }}
{{- end }}

e2e-test-status:
stage: test
image: registry.ddbuild.io/images/docker:20.10-py3
tags: ["arch:amd64"]
script:
- git clone -b rey.abolofia/status-check --single-branch https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.ddbuild.io/DataDog/serverless-e2e-tests
- cd ./serverless-e2e-tests
- ./scripts/check_e2e_status.sh
8 changes: 8 additions & 0 deletions ci/publish_layers.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,8 @@ fi
while [ $latest_version -lt $VERSION ]; do
latest_version=$(publish_layer $REGION $layer $aws_cli_python_version_key $layer_path)
printf "[$REGION] Published version $latest_version for layer $layer in region $REGION\n"
latest_arn=$(aws lambda get-layer-version --layer-name $layer --version-number $latest_version --region $REGION --query 'LayerVersionArn' --output text)
printf "[$REGION] Published arn $latest_arn\n"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Be sur to always print the arn to stdout when publishing layers. The publish gitlab jobs will do a regex search on this stdout to determine the arn for a newly published layer.


# This shouldn't happen unless someone manually deleted the latest version, say 28, and
# then tries to republish 28 again. The published version would actually be 29, because
Expand All @@ -209,4 +211,10 @@ while [ $latest_version -lt $VERSION ]; do
fi
done

if [ -n "$DOTENV" ]; then
printf "[$REGION] Exporting layer version to $DOTENV file...\n"
echo "PYTHON_${PYTHON_VERSION/./}_VERSION=$latest_arn" >> "$DOTENV"
cat "$DOTENV"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is where we write the arn to the given dotenv file. This dotenv file will be saved as an artifact on the job.

fi

printf "[$REGION] Finished publishing layers...\n\n"
Loading