Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ci: Do release tagging in CI and add --draft support #5652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Jan 11, 2023
Merged

ci: Do release tagging in CI and add --draft support #5652

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
4fa5100
ci: Do release tagging in CI and add --draft support
mafredri Jan 10, 2023
b7fc031
Ignore metadata to test workflow
mafredri Jan 10, 2023
368786f
Add -h, --help to release.sh
mafredri Jan 10, 2023
2449a80
Add -h, --help to increment_version_tag.sh
mafredri Jan 10, 2023
c5a6ead
Update .github/workflows/release.yaml
mafredri Jan 10, 2023
7381383
Use release notes file
mafredri Jan 10, 2023
e538ec4
Delay git tag push
mafredri Jan 10, 2023
3b4ffa0
Limit release concurrency
mafredri Jan 10, 2023
e211660
Remove code from bad fixup
mafredri Jan 10, 2023
29a33be
Rename as version_tag.sh (review feedback)
mafredri Jan 10, 2023
e1c802b
Merge branch 'main' into mafredri/ci-release-via-actions
mafredri Jan 10, 2023
7055b1d
Fix release notes check
mafredri Jan 10, 2023
9953d9b
Add automatic release watching
mafredri Jan 10, 2023
cf2c523
Fix watch
mafredri Jan 10, 2023
61b6a90
Fix notes
mafredri Jan 10, 2023
2108056
Prettify output
mafredri Jan 10, 2023
c559c8a
ci: Add git config, tag as "GitHub Actions Bot"
mafredri Jan 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 60 additions & 4 deletions .github/workflows/release.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,21 @@
# GitHub release workflow.
name: release
name: Release
on:
push:
tags:
- "v*"
workflow_dispatch:
inputs:
increment:
description: Preferred version increment (release script may promote e.g. patch to minor depending on changes).
type: choice
required: true
default: patch
options:
- patch
- minor
- major
draft:
description: Create a draft release (for manually editing release notes before publishing).
type: boolean
required: true
snapshot:
description: Force a dev version to be generated, implies dry_run.
type: boolean
Expand All @@ -25,9 +35,13 @@ permissions:

env:
CODER_RELEASE: ${{ github.event.inputs.snapshot && 'false' || 'true' }}
DRY_RUN: ${{ (github.event.inputs.dry_run || github.event.inputs.snapshot) && 'true' || 'false' }}

concurrency: ${{ github.workflow }}-${{ github.ref }}

jobs:
release:
name: Create and publish
runs-on: ${{ github.repository_owner == 'coder' && 'ubuntu-latest-16-cores' || 'ubuntu-latest' }}
env:
# Necessary for Docker manifest
Expand All @@ -45,6 +59,12 @@ jobs:
- name: Fetch git tags
run: git fetch --tags --force

# Configure git user name/email for creating annotated version tag.
- name: Setup git config
run: |
git config user.name "GitHub Actions Bot"
git config user.email ""

- name: Docker Login
uses: docker/login-action@v2
with:
Expand Down Expand Up @@ -100,6 +120,38 @@ jobs:
AC_CERTIFICATE_PASSWORD: ${{ secrets.AC_CERTIFICATE_PASSWORD }}
AC_APIKEY_P8_BASE64: ${{ secrets.AC_APIKEY_P8_BASE64 }}

- name: Create release tag and release notes
run: |
set -euo pipefail
ref=HEAD
old_version="$(git describe --abbrev=0 "$ref^1")"

if [[ $DRY_RUN == true ]]; then
# Allow dry-run of branches to pass.
export CODER_IGNORE_MISSING_COMMIT_METADATA=1
fi

# Cache commit metadata.
. ./scripts/release/check_commit_metadata.sh "$old_version" "$ref"

# Create new release tag.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Is it possible to run the workflow for the same version twice? It happened to me a few times in the past that the release script broke down in the middle of releasing and I had to clean tags and artifacts manually before retrying. I'm curious if we need the safety mechanism here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good nit. I was thinking we can use the GH actions concurrency keyword to ensure only one instance can run at a time: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#concurrency

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To add to this, the tag_version.sh script will error if the tag already exists, meaning we can't re-run this on main if HEAD already has been tagged:

❯ git tag v0.14.2
❯ CI=1 ./scripts/release/tag_version.sh --ref HEAD --patch
Checking commit metadata for changes since v0.14.1...
No breaking changes detected, using "patch" increment.
Old version: v0.14.1
New version: v0.14.2
$ git tag -a v0.14.2 -m Release v0.14.2 f5d623ff
fatal: tag 'v0.14.2' already exists

Does this, combined with concurrency (meaning only one release can run at once) take care of your concern @mtojek?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, although more convenient behavior is to allow for multiple runs to perform missing actions. For example, if the first run didn't publish the release notes due to HTTP 500, the second can fix it.

Copy link
Member Author

@mafredri mafredri Jan 11, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you mean. I think that's fairly well addressed here since right now the critical part happens in "Publish release". If anything fails before the tag upload (GH release immediately follows), we can just restart the workflow.

I just realized it's possible to run the workflow for a specific tag, not just for a branch. If we need to in the future it should be fairly easy to modify it to not error in tag_version.sh if we're running it for a tag since most steps are repeatable.

I'll think about this some more, thanks for bringing it up.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying!

version="$(
./scripts/release/tag_version.sh \
${{ (github.event.inputs.dry_run || github.event.inputs.snapshot) && '--dry-run' }} \
--ref "$ref" \
--${{ github.event.inputs.increment }}
)"

# Generate notes.
release_notes_file="$(mktemp -t release_notes.XXXXXX)"
./scripts/release/generate_release_notes.sh --old-version "$old_version" --new-version "$version" --ref "$ref" >> "$release_notes_file"
echo CODER_RELEASE_NOTES_FILE="$release_notes_file" >> $GITHUB_ENV

- name: Echo release notes
run: |
set -euo pipefail
cat "$CODER_RELEASE_NOTES_FILE"

- name: Build binaries
run: |
set -euo pipefail
Expand Down Expand Up @@ -157,8 +209,11 @@ jobs:

- name: Publish release
run: |
set -euo pipefail
./scripts/release/publish.sh \
${{ github.event.inputs.draft && '--draft' }} \
${{ (github.event.inputs.dry_run || github.event.inputs.snapshot) && '--dry-run' }} \
--release-notes-file "$CODER_RELEASE_NOTES_FILE" \
./build/*_installer.exe \
./build/*.zip \
./build/*.tar.gz \
Expand Down Expand Up @@ -195,6 +250,7 @@ jobs:
with:
name: release-artifacts
path: |
./build/*_installer.exe
./build/*.zip
./build/*.tar.gz
./build/*.tgz
Expand Down
186 changes: 129 additions & 57 deletions scripts/release.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,44 +1,80 @@
#!/usr/bin/env bash

# This script should be called to create a new release.
#
# When run, this script will display the new version number and optionally a
# preview of the release notes. The new version will be selected automatically
# based on if the release contains breaking changes or not. If the release
# contains breaking changes, a new minor version will be created. Otherwise, a
# new patch version will be created.
#
# Set --ref if you need to specify a specific commit that the new version will
# be tagged at, otherwise the latest commit will be used.
#
# Set --minor to force a minor version bump, even when there are no breaking
# changes.
#
# To mark a release as containing breaking changes, the commit title should
# either contain a known prefix with an exclamation mark ("feat!:",
# "feat(api)!:") or the PR that was merged can be tagged with the
# "release/breaking" label.
#
# Usage: ./release.sh [--ref <ref>] [--minor]

set -euo pipefail
# shellcheck source=scripts/lib.sh
source "$(dirname "${BASH_SOURCE[0]}")/lib.sh"
cdroot

usage() {
cat <<EOH
Usage: ./release.sh [--branch <name>] [--draft] [--dry-run] [--ref <ref>] [--major | --minor | --patch]

This script should be called to create a new release.

When run, this script will display the new version number and optionally a
preview of the release notes. The new version will be selected automatically
based on if the release contains breaking changes or not. If the release
contains breaking changes, a new minor version will be created. Otherwise, a
new patch version will be created.

Set --ref if you need to specify a specific commit that the new version will
be tagged at, otherwise the latest commit will be used.

Set --minor to force a minor version bump, even when there are no breaking
changes. Likewise for --major. By default a patch version will be created.

Set --dry-run to run the release workflow in CI as a dry-run (no release will
be created).

To mark a release as containing breaking changes, the commit title should
either contain a known prefix with an exclamation mark ("feat!:",
"feat(api)!:") or the PR that was merged can be tagged with the
"release/breaking" label.

To test changes to this script, you can set --branch <my-branch>, which will
run the release workflow in CI as a dry-run and use the latest commit on the
specified branch as the release commit. This will also set --dry-run.
EOH
}

branch=main
draft=0
dry_run=0
ref=
minor=0
increment=

args="$(getopt -o n -l ref:,minor -- "$@")"
args="$(getopt -o h -l branch:,draft,dry-run,help,ref:,major,minor,patch -- "$@")"
eval set -- "$args"
while true; do
case "$1" in
--branch)
branch="$2"
log "Using branch $branch, implies DRYRUN and CODER_IGNORE_MISSING_COMMIT_METADATA."
dry_run=1
export CODER_IGNORE_MISSING_COMMIT_METADATA=1
shift 2
;;
--draft)
draft=1
shift
;;
--dry-run)
dry_run=1
shift
;;
-h | --help)
usage
exit 0
;;
--ref)
ref="$2"
shift 2
;;
--minor)
minor=1
--major | --minor | --patch)
if [[ -n $increment ]]; then
error "Cannot specify multiple version increments."
fi
increment=${1#--}
shift
;;
--)
Expand All @@ -54,15 +90,20 @@ done
# Check dependencies.
dependencies gh sort

if [[ -z $increment ]]; then
# Default to patch versions.
increment="patch"
fi

# Make sure the repository is up-to-date before generating release notes.
log "Fetching main and tags from origin..."
git fetch --quiet --tags origin main
log "Fetching $branch and tags from origin..."
git fetch --quiet --tags origin "$branch"

# Resolve to the latest ref on origin/main unless otherwise specified.
ref=$(git rev-parse --short "${ref:-origin/main}")
ref=$(git rev-parse --short "${ref:-origin/$branch}")

# Make sure that we're running the latest release script.
if [[ -n $(git diff --name-status origin/main -- ./scripts/release.sh) ]]; then
if [[ -n $(git diff --name-status origin/"$branch" -- ./scripts/release.sh) ]]; then
error "Release script is out-of-date. Please check out the latest version and try again."
fi

Expand All @@ -71,42 +112,73 @@ fi
mapfile -t versions < <(gh api -H "Accept: application/vnd.github+json" /repos/coder/coder/git/refs/tags -q '.[].ref | split("/") | .[2]' | grep '^v' | sort -r -V)
old_version=${versions[0]}

log "Checking commit metadata for changes since $old_version..."
# shellcheck source=scripts/release/check_commit_metadata.sh
source "$SCRIPT_DIR/release/check_commit_metadata.sh" "$old_version" "$ref"

mapfile -d . -t version_parts <<<"$old_version"
if [[ $minor == 1 ]] || [[ $COMMIT_METADATA_BREAKING == 1 ]]; then
if [[ $COMMIT_METADATA_BREAKING == 1 ]]; then
log "Breaking change detected, incrementing minor version..."
else
log "Forcing minor version bump..."
fi
version_parts[1]=$((version_parts[1] + 1))
version_parts[2]=0
else
log "No breaking changes detected, incrementing patch version..."
version_parts[2]=$((version_parts[2] + 1))
fi
new_version="${version_parts[0]}.${version_parts[1]}.${version_parts[2]}"

log "Old version: ${old_version}"
log "New version: ${new_version}"

new_version="$(execrelative ./release/tag_version.sh --dry-run --ref "$ref" --"$increment")"
release_notes="$(execrelative ./release/generate_release_notes.sh --old-version "$old_version" --new-version "$new_version" --ref "$ref")"

echo
log
read -p "Preview release notes? (y/n) " -n 1 -r show_reply
echo
log
if [[ $show_reply =~ ^[Yy]$ ]]; then
echo -e "$release_notes\n"
fi

read -p "Create release? (y/n) " -n 1 -r create
echo
if [[ $create =~ ^[Yy]$ ]]; then
log "Tagging commit $ref as $new_version..."
git tag -a "$new_version" -m "$new_version" "$ref"
log "Pushing tag to origin..."
git push -u origin "$new_version"
create_message="Create release"
if ((draft)); then
create_message="Create draft release"
fi
if ((dry_run)); then
create_message+=" (DRYRUN)"
fi
read -p "$create_message? (y/n) " -n 1 -r create
log
if ! [[ $create =~ ^[Yy]$ ]]; then
exit 0
fi

args=()
if ((draft)); then
args+=(-F draft=true)
fi
if ((dry_run)); then
args+=(-F dry_run=true)
fi

log
gh workflow run release.yaml \
--ref "$branch" \
-F increment="$increment" \
-F snapshot=false \
"${args[@]}"
log

read -p "Watch release? (y/n) " -n 1 -r watch
log
if ! [[ $watch =~ ^[Yy]$ ]]; then
exit 0
fi

log 'Waiting for job to become "in_progress"...'

# Wait at most 3 minutes (3*60)/3 = 60 for the job to start.
for _ in $(seq 1 60); do
mapfile -t run < <(
# Output:
# 3886828508
# in_progress
gh run list -w release.yaml \
--limit 1 \
--json status,databaseId \
--jq '.[] | (.databaseId | tostring), .status'
)
if [[ ${run[1]} != "in_progress" ]]; then
sleep 3
continue
fi
gh run watch --exit-status "${run[0]}"
exit 0
done

error "Waiting for job to start timed out."
19 changes: 14 additions & 5 deletions scripts/release/check_commit_metadata.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ dependencies gh
COMMIT_METADATA_BREAKING=0
declare -A COMMIT_METADATA_TITLE COMMIT_METADATA_CATEGORY

# This environment variable can be set to 1 to ignore missing commit metadata,
# useful for dry-runs.
ignore_missing_metadata=${CODER_IGNORE_MISSING_COMMIT_METADATA:-0}

main() {
# Match a commit prefix pattern, e.g. feat: or feat(site):.
prefix_pattern="^([a-z]+)(\([a-z]*\))?:"
Expand Down Expand Up @@ -87,9 +91,11 @@ main() {
commit_sha_long=${parts[1]}
commit_prefix=${parts[2]}

# Safety-check, guarantee all commits had their metadata fetched.
if [[ ! -v labels[$commit_sha_long] ]]; then
error "Metadata missing for commit $commit_sha_short"
if [[ $ignore_missing_metadata != 1 ]]; then
# Safety-check, guarantee all commits had their metadata fetched.
if [[ ! -v labels[$commit_sha_long] ]]; then
error "Metadata missing for commit $commit_sha_short"
fi
fi

# Store the commit title for later use.
Expand All @@ -99,11 +105,11 @@ main() {

# First, check the title for breaking changes. This avoids doing a
# GH API request if there's a match.
if [[ $commit_prefix =~ $breaking_title ]] || [[ ${labels[$commit_sha_long]} = *"label:$breaking_label"* ]]; then
if [[ $commit_prefix =~ $breaking_title ]] || [[ ${labels[$commit_sha_long]:-} = *"label:$breaking_label"* ]]; then
COMMIT_METADATA_CATEGORY[$commit_sha_short]=$breaking_category
COMMIT_METADATA_BREAKING=1
continue
elif [[ ${labels[$commit_sha_long]} = *"label:$security_label"* ]]; then
elif [[ ${labels[$commit_sha_long]:-} = *"label:$security_label"* ]]; then
COMMIT_METADATA_CATEGORY[$commit_sha_short]=$security_label
continue
fi
Expand Down Expand Up @@ -137,6 +143,9 @@ export_commit_metadata() {
if [[ ${_COMMIT_METADATA_CACHE:-} == "${range}:"* ]]; then
eval "${_COMMIT_METADATA_CACHE#*:}"
else
if [[ $ignore_missing_metadata == 1 ]]; then
log "WARNING: Ignoring missing commit metadata, breaking changes may be missed."
fi
main
fi

Expand Down
Loading