Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Upload toolstates.json to rust-lang-ci2 #65274

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 31, 2019
Merged

Upload toolstates.json to rust-lang-ci2 #65274

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4479de4
ci: extract uploading artifacts into a script
pietroalbini Oct 10, 2019
0200050
ci: upload toolstates.json to rust-lang-ci2
pietroalbini Oct 10, 2019
bdfcde4
Apply suggestions from lzutao
pietroalbini Oct 29, 2019
ca34687
ci: move toolstates.json to /tmp/toolstate/ and docker mount it
pietroalbini Oct 30, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
ci: extract uploading artifacts into a script
  • Loading branch information
@pietroalbini
pietroalbini committed Oct 29, 2019
commit 4479de4b4a375e792127ddcbcb5f91e3b58ce237
44 changes: 14 additions & 30 deletions src/ci/azure-pipelines/steps/run.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,37 +201,21 @@ steps:
condition: and(succeeded(), not(variables.SKIP_JOB))
displayName: Run build

# If we're a deploy builder, use the `aws` command to publish everything to our
# bucket.
- bash: |
set -e
source src/ci/shared.sh
if [ "$AGENT_OS" = "Linux" ]; then
rm -rf obj/build/dist/doc
upload_dir=obj/build/dist
else
rm -rf build/dist/doc
upload_dir=build/dist
fi
ls -la $upload_dir
deploy_dir=rustc-builds
if [ "$DEPLOY_ALT" == "1" ]; then
deploy_dir=rustc-builds-alt
fi
retry aws s3 cp --no-progress --recursive --acl public-read ./$upload_dir s3://$DEPLOY_BUCKET/$deploy_dir/$BUILD_SOURCEVERSION
- bash: src/ci/scripts/upload-artifacts.sh
env:
AWS_ACCESS_KEY_ID: $(UPLOAD_AWS_ACCESS_KEY_ID)
AWS_SECRET_ACCESS_KEY: $(UPLOAD_AWS_SECRET_ACCESS_KEY)
condition: and(succeeded(), not(variables.SKIP_JOB), or(eq(variables.DEPLOY, '1'), eq(variables.DEPLOY_ALT, '1')))
displayName: Upload artifacts

# Upload CPU usage statistics that we've been gathering this whole time. Always
# execute this step in case we want to inspect failed builds, but don't let
# errors here ever fail the build since this is just informational.
- bash: aws s3 cp --acl public-read cpu-usage.csv s3://$DEPLOY_BUCKET/rustc-builds/$BUILD_SOURCEVERSION/cpu-$CI_JOB_NAME.csv
env:
AWS_ACCESS_KEY_ID: $(UPLOAD_AWS_ACCESS_KEY_ID)
AWS_SECRET_ACCESS_KEY: $(UPLOAD_AWS_SECRET_ACCESS_KEY)
condition: variables['UPLOAD_AWS_SECRET_ACCESS_KEY']
continueOnError: true
displayName: Upload CPU usage statistics
# Adding a condition on DEPLOY=1 or DEPLOY_ALT=1 is not needed as all deploy
# builders *should* have the AWS credentials available. Still, explicitly
# adding the condition is helpful as this way CI will not silently skip
# deploying artifacts from a dist builder if the variables are misconfigured,
# erroring about invalid credentials instead.
condition: |
and(
succeeded(), not(variables.SKIP_JOB),
or(
variables.UPLOAD_AWS_SECRET_ACCESS_KEY,
eq(variables.DEPLOY, '1'), eq(variables.DEPLOY_ALT, '1')
)
)
36 changes: 36 additions & 0 deletions src/ci/scripts/upload-artifacts.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
#!/bin/bash
# Upload all the artifacts to our S3 bucket. All the files inside ${upload_dir}
# will be uploaded to the deploy bucket and eventually signed and released in
# static.rust-lang.org.

set -euo pipefail
IFS=$'\n\t'

source "$(cd "$(dirname "$0")" && pwd)/../shared.sh"

upload_dir="$(mktemp -d)"

# Release tarballs produced by a dist builder.
if [[ "${DEPLOY-0}" = "1" ]] || [[ "${DEPLOY_ALT-0}" = "1" ]]; then
dist_dir=build/dist
if isLinux; then
dist_dir=obj/build/dist
fi
rm -rf "${dist_dir}/doc"
cp -r "${dist_dir}"/* "${upload_dir}"
fi

# CPU usage statistics.
cp cpu-usage.csv "${upload_dir}/cpu-${CI_JOB_NAME}.csv"

echo "Files that will be uploaded:"
ls -lah "${upload_dir}"
echo

deploy_dir="rustc-builds"
if [[ "${DEPLOY_ALT-0}" = "1" ]]; then
deploy_dir="rustc-builds-alt"
fi
deploy_url="s3://${DEPLOY_BUCKET}/${deploy_dir}/$(ciCommit)"

retry aws s3 cp --no-progress --recursive --acl public-read "${upload_dir}" "${deploy_url}"
4 changes: 4 additions & 0 deletions src/ci/shared.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,10 @@ function getCIBranch {
echo "$BUILD_SOURCEBRANCHNAME"
}

function ciCommit {
echo "${BUILD_SOURCEVERSION}"
}

function ciCommandAddPath {
if [[ $# -ne 1 ]]; then
echo "usage: $0 <path>"
Expand Down