rulesets/nixpkgs: switch merge queue to "headgreen" strategy #171
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The `ALLGREEN` setting corresponds to "Require all queue entries to pass required checks". `HEADGREEN` is the opposite, when disabling that checkbox, which then behaves like this: > only the commit at the head of the merge group, i.e. the commit > containing changes from all of the PRs in the group, must pass its > required checks to merge. This should fix an odd scheduling behavior we're seeing for a few days now: With a reasonable number of jobs queued, GitHub will sometimes just forget to schedule jobs for a merge queue item. These will then wait for an hour for the status check to timeout, be kicked out of the queue and all other jobs run again. We don't really need to do that, though: We already require status checks for each PR individually, thus there is a *very low chance* of introducing introducing a commit that: - Passes CI in the PR, and - Fails CI in the Merge Queue, and - Is fixed by a random PR queue up behind it. While at it, also reduce the timeout from 60 minutes to 20 minutes, because we'll certainly not need more time for any merge to master, even if we had the Eval and Build workflows running (which is not even the case, yet).
MattSturgeon
approved these changes
Sep 21, 2025
Comment on lines
22
to
24
| "min_entries_to_merge": 1, | ||
| "max_entries_to_merge": 5, | ||
| "min_entries_to_merge_wait_minutes": 5, |
There was a problem hiding this comment.
While we're talking about finessing the numbers, we should look at tweaking these numbers soon.
In particular, I think we should:
- Increase "max entries" (build & merge), so that we test & merge larger groups
- Increase "min entries," so that we don't start testing until we have a few PRs queued
- Consider decreasing "minutes to wait," so that we don't have to wait too long if the queue is really quiet for some reason (maybe 5m is ok already, though?)
Increasing the "max entries" is based on the assumption that failures will be rare. Therefore, we should optimistically attempt to test & merge larger groups, thereby speeding up the queue. Failures will delay more PRs, but the happy path will be faster.
There was a problem hiding this comment.
Before tweaking these numbers, I think we should move more stuff to the queue first. Eval, Build etc. - right now, with the very basic queue, it might have very different behavior.
Mic92
approved these changes
Sep 21, 2025
infinisil
approved these changes
Sep 24, 2025
Lassulus
approved these changes
Sep 24, 2025
|
Applied! |
|
Thank you! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The
ALLGREENsetting corresponds to "Require all queue entries to pass required checks".HEADGREENis the opposite, when disabling that checkbox, which then behaves like this:This should fix an odd scheduling behavior we're seeing for a few days now: With a reasonable number of jobs queued, GitHub will sometimes just forget to schedule jobs for a merge queue item. These will then wait for an hour for the status check to timeout, be kicked out of the queue and all other jobs run again.
We don't really need to do that, though: We already require status checks for each PR individually, thus there is a very low chance of introducing introducing a commit that:
While at it, also reduce the timeout from 60 minutes to 20 minutes, because we'll certainly not need more time for any merge to master, even if we had the Eval and Build workflows running (which is not even the case, yet).
The "broken" state of the merge queue looks like this:
Note how the first item has been queued for more than 20 minutes and still didn't post a result. Going into the Actions tab, we can also see that GitHub never scheduled a run for this commit...
cc @NixOS/nixpkgs-ci