CI is currently pretty slow. In particular, if there are several PRs in quick succession, we'll run out of CI capacity. A reasonable medium-term goal might to get CI builds finish in 10 minutes in typical cases. I'm not sure if that is realistic, however.
Here are some ideas about how to make things better:
- Drop appveyor (it seems redundant now) [done]
- Run selected slow tests only after a PR has been merged
- Alternatively, only run them once per PR iteration, and run on all platforms and Python versions after merge
- If a PR only touches mypyc, don't run non-compiled mypy tests
- Remove some slow tests from CI if they don't bring much value
- Run more tests in incremental mode
- If a PR only touches docs, don't run all tests
- Test fewer platform / Python version combinations (possibly only in PR builds)
- Speed up individual tests somehow
- Somehow get more CI capacity
Can anybody suggest more things that could help, or which of the above things would be worth pursuing first?
CI is currently pretty slow. In particular, if there are several PRs in quick succession, we'll run out of CI capacity. A reasonable medium-term goal might to get CI builds finish in 10 minutes in typical cases. I'm not sure if that is realistic, however.
Here are some ideas about how to make things better:
Can anybody suggest more things that could help, or which of the above things would be worth pursuing first?