ci: Reduce CI for draft pull requests - #157360

I'm -1 on this. I fear it may lead to surprises, since we're skipping jobs, and it also complicates our workflows quite a bit.

I also don't see much benefit. The JIT workflows only run in very specific scenarios (that is, when you're working on the JIT) and I presume that in those cases you'd want all of them to run. The mypy jobs only take a few seconds, so I'm not sure we're saving much there either. And finally, cutting most of the tests seems counterproductive as draft PRs will no longer be particularly useful.

Also, do we really have that many draft PRs? I suppose Hugo can probably get some stats from Datadog on their CI usage.

For branches that are still under development, I create PRs against my own fork rather than against python/cpython. That is probably why there are relatively few draft PRs in python/cpython.

The problem is that a large number of Actions are triggered for branches that are still under development. This is a PR within my fork, and many Actions are being triggered by both push and pull_request events.

To suppress them, I manually disabled the workflows. However, the option to disable a workflow is buried in a very unintuitive place, so I suspect many people are running a large number of unnecessary Actions simply because they do not know how to suppress them.

With the rapid development of AI, computing resources are becoming increasingly constrained worldwide. Microsoft currently provides everyone with a generous free allowance, but it would still be a good idea to start conserving resources now.

For example, what about running tests on pull_request only when the repository is python/cpython? For branches that are still under development, running tests on push alone should be sufficient. That would cut the number of Actions runs roughly in half.

Ok, so if I understand correctly, this is targeted at people running CI on their forks, rather than on python/cpython.

I'm still not convinced this would be desirable for users. I know some people run CI on their fork before opening a PR, to ensure there are no failures. That would no longer be reliable.

The problem is that a large number of Actions are triggered for branches that are still under development. This is a PR within my fork, and many Actions are being triggered by both push and pull_request events.

Indeed, in that case you'll get duplicate runs. We could add skips for one when both are true to avoid the duplicates, but that's also a little messy. It would cut the number of runs for a PR opened in a fork down by half. You can, however, also make such changes only in your fork.

To suppress them, I manually disabled the workflows. However, the option to disable a workflow is buried in a very unintuitive place, so I suspect many people are running a large number of unnecessary Actions simply because they do not know how to suppress them.

That seems like a better solution to me, as users who are concerned about their usage can selectively disable workflows they don't need. I think that's far better than making that decision for them. We could perhaps document this in the devguide?

With the rapid development of AI, computing resources are becoming increasingly constrained worldwide. Microsoft currently provides everyone with a generous free allowance, but it would still be a good idea to start conserving resources now.

Currently, we're provided with an unlimited number of Actions minutes (at least for python/cpython). I also presume Python brings Microsoft more financial benefit than it costs them to pay for our CI. But since this isn't currently an issue, I don't think there's much need to get into that discussion.

For example, what about running tests on pull_request only when the repository is python/cpython? For branches that are still under development, running tests on push alone should be sufficient. That would cut the number of Actions runs roughly in half.

In python/cpython, that's already the case. See the CI runs on this PR, for example, they're all triggered only by pull_request. push requires the branch to be in the same repository.

After checking, I found that only the lint workflow was triggered by both push and pull_request; the tests were triggered only by pull_request.

https://github.com/methane/cpython/actions/runs/34736727947/job/103669363596

I do get the impression that GitHub Actions has become less reliable, but let's leave things as they are until Microsoft reduces the free allowance.