Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ci: improve test parallelism #7943

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jun 9, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/actions/setup-go/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,32 @@ runs:
with:
cache: true
go-version: "1.20.5"

- name: Cache go
uses: buildjet/cache@v3
with:
# ~/go/pkg is the same across operating systems.
path: |
~/go/pkg
~/.cache/go-build
~/AppData/Local/go-build
~/Library/Caches/go-build
# Job name must be included in the key for effective
# test cache reuse.
key: go-${{ runner.os }}-${{ github.job }}-${{ hashFiles('**/*.go', 'go.**') }}
restore-keys: |
go-${{ runner.os }}-${{ github.job }}-
go-${{ runner.os }}-
go-

- name: Install gotestsum
uses: jaxxstorm/[email protected]
with:
repo: gotestyourself/gotestsum
tag: v1.9.0

# It isn't necessary that we ever do this, but it helps
# separate the "setup" from the "run" times.
- name: go mod download
shell: bash
run: go mod download -x
19 changes: 11 additions & 8 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -218,8 +218,18 @@ jobs:
echo "cover=false" >> $GITHUB_OUTPUT
fi

# By default Go will use the number of logical CPUs, which
# is a fine default.
PARALLEL_FLAG=""
if [ "${{ matrix.os }}" == "windows-2019" ]; then
# Windows appears more I/O bound, so we increase parallelism
# to make better use of CPU.
PARALLEL_FLAG="-parallel=16"
fi

export TS_DEBUG_DISCO=true
gotestsum --junitfile="gotests.xml" --jsonfile="gotests.json" --packages="./..." -- -parallel=8 -timeout=7m -short -failfast $COVERAGE_FLAGS
gotestsum --junitfile="gotests.xml" --jsonfile="gotests.json" \
--packages="./..." -- $PARALLEL_FLAG -short -failfast $COVERAGE_FLAGS

- name: Print test stats
if: success() || failure()
Expand All @@ -228,13 +238,6 @@ jobs:
# so we need to print the test stats to the log.
go run ./scripts/ci-report/main.go gotests.json | tee gotests_stats.json

- uses: actions/upload-artifact@v3
if: success() || failure()
with:
name: gotests-${{ matrix.os }}.xml
path: ./gotests.xml
retention-days: 30

- uses: ./.github/actions/upload-datadog
if: always()
with:
Expand Down
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -610,15 +610,15 @@ test: test-clean

# When updating -timeout for this test, keep in sync with
# test-go-postgres (.github/workflows/coder.yaml).
# Do add coverage flags so that test caching works.
test-postgres: test-clean test-postgres-docker
# The postgres test is prone to failure, so we limit parallelism for
# more consistent execution.
DB=ci DB_FROM=$(shell go run scripts/migrate-ci/main.go) gotestsum \
--junitfile="gotests.xml" \
--jsonfile="gotests.json" \
--packages="./..." -- \
-covermode=atomic -coverprofile="gotests.coverage" -timeout=20m \
-coverpkg=./... \
-timeout=20m \
-failfast
.PHONY: test-postgres

Expand Down