Early exit repo sync if merge-upstream requires workflow scope #7612

williammartin · 2023-06-21T14:03:37Z

Description

This is a step towards addressing #7574.

As a result of changes around repository APIs, if commits that are to be synced from upstream include workflow file changes, the API requires the token to have workflow scope. When that issue was raised this was failing as opaque errors on the /merge-upstream and /repos/<OWNER>/<REPO>/git/refs/heads/<branch> endpoints. The /merge-upstream endpoint has been updated to return a more transparent error of the form:

{
  "message": "refusing to allow an OAuth App to create or update workflow `.github/workflows/bench.yml` without `workflow` scope",
  "documentation_url": "https://docs.github.com/rest/branches/branches#sync-a-fork-branch-with-the-upstream-repository"
}

This PR attempts to capture that and bubble an error to the user stating that they need to get these new scopes. Sadly, the API doe not report that it needs the workflow scope in the X-Accepted-Oauth-Scopes header and even if it did we skip the scope suggestion on 422. I'm not sure what the reason for that is, so maybe someone can help me understand that.

Demonstration

Before

➜  dune git:(main) gh repo sync williammartin-test-org/dune
HTTP 404: Not Found (https://api.github.com/repos/williammartin-test-org/dune/git/refs/heads/main)

After

➜  dune git:(main) ~/workspace/cli/bin/gh repo sync williammartin-test-org/dune
Upstream commits contain workflow changes, which require the `workflow` scope to merge. To request it, run: gh auth refresh -s workflow

Reproduction

To reproduce this, you can:

Create a repository upstream
Create a fork downstream
Push a workflow file change in upstream
gh repo sync <downstream>

Limitations

Since the /upstream-merge endpoint doesn't support --force, I think the following scenario is possible but I haven't confirmed it.

Given a repo that has diverged from upstream and that diverged history contains a workflow file change.
When I pass --force to the CLI
Then the /merge-upstream endpoint returns a 409 for conflict
And then we fall back to /repos/<OWNER>/<REPO>/git/refs/heads/main which still returns a 404 opaque error.

However, this seems a significant improvement either way and with GH_DEBUG the failure on the /merge-upstream is already much more apparent.

williammartin · 2023-06-21T14:11:57Z

pkg/httpmock/stub.go

+func StatusJSONResponse(status int, body interface{}) Responder {
+	return func(req *http.Request) (*http.Response, error) {
+		b, _ := json.Marshal(body)
+		header := http.Header{


This header was needed because the graphql error message parsing only happens if the content type is JSON

williammartin · 2023-06-21T14:12:14Z

pkg/httpmock/stub.go

 	return func(req *http.Request) (*http.Response, error) {
 		b, _ := json.Marshal(body)
-		return httpResponse(200, req, bytes.NewBuffer(b)), nil
+		header := http.Header{


Not strictly sure this was necessary but seemed sensible since it was required on the unhappy path.

samcoe

LGTM, thanks for this improvement!

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [aquaproj/aqua-registry](https://togithub.com/aquaproj/aqua-registry) | minor | `v4.25.0` -> `v4.26.0` | | [cli/cli](https://togithub.com/cli/cli) | minor | `v2.31.0` -> `v2.32.0` | | [fluxcd/flux2](https://togithub.com/fluxcd/flux2) | patch | `v2.0.0` -> `v2.0.1` | --- ### Release Notes <details> <summary>aquaproj/aqua-registry (aquaproj/aqua-registry)</summary> ### [`v4.26.0`](https://togithub.com/aquaproj/aqua-registry/releases/tag/v4.26.0) [Compare Source](https://togithub.com/aquaproj/aqua-registry/compare/v4.25.0...v4.26.0) [Issues](https://togithub.com/aquaproj/aqua-registry/issues?q=is%3Aissue+milestone%3Av4.26.0) | [Pull Requests](https://togithub.com/aquaproj/aqua-registry/pulls?q=is%3Apr+milestone%3Av4.26.0) | aquaproj/aqua-registry@v4.25.0...v4.26.0 #### 🎉 New Packages [#13656](https://togithub.com/aquaproj/aqua-registry/issues/13656) [cilium/cilium-cli](https://togithub.com/cilium/cilium-cli): CLI to install, manage & troubleshoot Kubernetes clusters running Cilium [@honahuku](https://togithub.com/honahuku) [#13657](https://togithub.com/aquaproj/aqua-registry/issues/13657) [ponkio-o/ec2x](https://togithub.com/ponkio-o/ec2x): A cli tool of connect to ec2 instance [@ponkio-o](https://togithub.com/ponkio-o) #### 🎉 New Contributor [@honahuku](https://togithub.com/honahuku) [#13656](https://togithub.com/aquaproj/aqua-registry/issues/13656) </details> <details> <summary>cli/cli (cli/cli)</summary> ### [`v2.32.0`](https://togithub.com/cli/cli/releases/tag/v2.32.0): GitHub CLI 2.32.0 [Compare Source](https://togithub.com/cli/cli/compare/v2.31.0...v2.32.0) hello terminal fans, it's me [@vilmibm](https://togithub.com/vilmibm) . We are pleased to bring you a new release of the GitHub CLI on this Confusion 46, 3189 YOLD. We've got features. we've got bugfixes. This release has a real zest for life; can you feel it? First though, a sad note. [@mislav](https://togithub.com/mislav) has moved on from GitHub. He was instrumental to this project and without him I don't think it would have ever happened. BIG THANKS MUCH LOVE to this wonderful person! Happily, we have a new core team member though who is, truly, a delight. Please welcome [@williammartin](https://togithub.com/williammartin) , whom you'll see a lot more on issues and pull requests moving forward. Now, let's talk ***BUSINESS***. #### gh ruleset [Repository Rulesets](https://github.blog/changelog/2023-04-17-introducing-repository-rules-public-beta/) are in beta on git hub dot com and they are now also in beta on git hub sea ell eye. You can list, view, and check branches against rulesets set at the repository or organization level. Major shoutouts to [@vaindil](https://togithub.com/vaindil) for this [big contribution](https://togithub.com/cli/cli/pull/7650). My favorite command in here in `gh rs check <branch>` which will tell you what rules would apply to a hypothetical branch name. To learn more, run `gh help ruleset`. #### gh cache `gh cache` is a new top level command in our suite of support for Actions. It lets you list and delete caches saved in Actions. It's [neat](https://togithub.com/cli/cli/pull/7403) and I like it. Thanks, [@joshkraft](https://togithub.com/joshkraft) ! To learn more, run `gh help cache`. #### Other new stuff - Add option to remove file from gist by [@kousikmitra](https://togithub.com/kousikmitra) in [https://github.com/cli/cli/pull/7560](https://togithub.com/cli/cli/pull/7560) - Add remove/reset to auth refresh by [@n1lesh](https://togithub.com/n1lesh) in [https://github.com/cli/cli/pull/7597](https://togithub.com/cli/cli/pull/7597) - Small tweaks to auth refresh remove-scopes and reset-scopes flags by [@samcoe](https://togithub.com/samcoe) in [https://github.com/cli/cli/pull/7631](https://togithub.com/cli/cli/pull/7631) - make `gh pr create` behavior like `gh repo fork` by [@leavesster](https://togithub.com/leavesster) in [https://github.com/cli/cli/pull/7330](https://togithub.com/cli/cli/pull/7330) - chore: add title to iteration and milestone fields by [@kamaz](https://togithub.com/kamaz) in [https://github.com/cli/cli/pull/7621](https://togithub.com/cli/cli/pull/7621) - AutoFetch new branch created with gh issue develop by [@shion1305](https://togithub.com/shion1305) in [https://github.com/cli/cli/pull/7589](https://togithub.com/cli/cli/pull/7589) - feat: add statuscheck description to pr checks output by [@jgrumboe](https://togithub.com/jgrumboe) in [https://github.com/cli/cli/pull/7451](https://togithub.com/cli/cli/pull/7451) - Always show created gist privacy status. by [@seanbright](https://togithub.com/seanbright) in [https://github.com/cli/cli/pull/7645](https://togithub.com/cli/cli/pull/7645) - \[Codespaces] Support random `--server-port=0` and printing connection details by [@josebalius](https://togithub.com/josebalius) in [https://github.com/cli/cli/pull/7655](https://togithub.com/cli/cli/pull/7655) - gh release edit: support --verify-tag like gh release create by [@rajhawaldar](https://togithub.com/rajhawaldar) in [https://github.com/cli/cli/pull/7646](https://togithub.com/cli/cli/pull/7646) - Feature: Add `fill-first` flag to `pr create` command by [@kousikmitra](https://togithub.com/kousikmitra) in [https://github.com/cli/cli/pull/7398](https://togithub.com/cli/cli/pull/7398) - Return error on no-browser option if repo don't exists by [@kousikmitra](https://togithub.com/kousikmitra) in [https://github.com/cli/cli/pull/7314](https://togithub.com/cli/cli/pull/7314) #### Bugfixes - Early exit repo sync if merge-upstream requires workflow scope by [@williammartin](https://togithub.com/williammartin) in [https://github.com/cli/cli/pull/7612](https://togithub.com/cli/cli/pull/7612) - Don't deduplicate checks that stem from different events by [@samcoe](https://togithub.com/samcoe) in [https://github.com/cli/cli/pull/7618](https://togithub.com/cli/cli/pull/7618) - gh run cancel needs input validation by [@rajhawaldar](https://togithub.com/rajhawaldar) in [https://github.com/cli/cli/pull/7647](https://togithub.com/cli/cli/pull/7647) - Ensure gist edit request body matches desired schema by [@williammartin](https://togithub.com/williammartin) in [https://github.com/cli/cli/pull/7635](https://togithub.com/cli/cli/pull/7635) - fix(api): do not interpret "branch" placeholder when `GH_REPO` is set by [@alex-petrov-vt](https://togithub.com/alex-petrov-vt) in [https://github.com/cli/cli/pull/7626](https://togithub.com/cli/cli/pull/7626) - fix pr create crash on interactive milestone selection by [@vilmibm](https://togithub.com/vilmibm) in [https://github.com/cli/cli/pull/7666](https://togithub.com/cli/cli/pull/7666) - Fix issue develop command by [@samcoe](https://togithub.com/samcoe) in [https://github.com/cli/cli/pull/7656](https://togithub.com/cli/cli/pull/7656) #### Docs & Dev - docs: example of setting multiple vars using stdin by [@iloveitaly](https://togithub.com/iloveitaly) in [https://github.com/cli/cli/pull/7683](https://togithub.com/cli/cli/pull/7683) - Remove old code paths and improve code comments for `repo sync` by [@samcoe](https://togithub.com/samcoe) in [https://github.com/cli/cli/pull/7610](https://togithub.com/cli/cli/pull/7610) - Add tenancy support by [@samcoe](https://togithub.com/samcoe) in [https://github.com/cli/cli/pull/7636](https://togithub.com/cli/cli/pull/7636) - Update httpretty to released version by [@josebalius](https://togithub.com/josebalius) in [https://github.com/cli/cli/pull/7654](https://togithub.com/cli/cli/pull/7654) - build(deps): bump github.com/henvic/httpretty from 0.1.1 to 0.1.2 by [@dependabot](https://togithub.com/dependabot) in [https://github.com/cli/cli/pull/7657](https://togithub.com/cli/cli/pull/7657) - build(deps): bump google.golang.org/grpc from 1.49.0 to 1.53.0 by [@dependabot](https://togithub.com/dependabot) in [https://github.com/cli/cli/pull/7665](https://togithub.com/cli/cli/pull/7665) - Use SmartBaseRepoFunc for api command by [@whi-tw](https://togithub.com/whi-tw) in [https://github.com/cli/cli/pull/7594](https://togithub.com/cli/cli/pull/7594) - Clean up style nits and simplify some logic by [@samcoe](https://togithub.com/samcoe) in [https://github.com/cli/cli/pull/7687](https://togithub.com/cli/cli/pull/7687) - add gh cache to Actions explainer by [@vilmibm](https://togithub.com/vilmibm) in [https://github.com/cli/cli/pull/7689](https://togithub.com/cli/cli/pull/7689) #### New Contributors - [@leavesster](https://togithub.com/leavesster) made their first contribution in [https://github.com/cli/cli/pull/7330](https://togithub.com/cli/cli/pull/7330) - [@kamaz](https://togithub.com/kamaz) made their first contribution in [https://github.com/cli/cli/pull/7621](https://togithub.com/cli/cli/pull/7621) - [@shion1305](https://togithub.com/shion1305) made their first contribution in [https://github.com/cli/cli/pull/7589](https://togithub.com/cli/cli/pull/7589) - [@jgrumboe](https://togithub.com/jgrumboe) made their first contribution in [https://github.com/cli/cli/pull/7451](https://togithub.com/cli/cli/pull/7451) - [@whi-tw](https://togithub.com/whi-tw) made their first contribution in [https://github.com/cli/cli/pull/7594](https://togithub.com/cli/cli/pull/7594) - [@seanbright](https://togithub.com/seanbright) made their first contribution in [https://github.com/cli/cli/pull/7645](https://togithub.com/cli/cli/pull/7645) - [@iloveitaly](https://togithub.com/iloveitaly) made their first contribution in [https://github.com/cli/cli/pull/7683](https://togithub.com/cli/cli/pull/7683) **Full Changelog**: cli/cli@v2.31.0...v2.32.0 </details> <details> <summary>fluxcd/flux2 (fluxcd/flux2)</summary> ### [`v2.0.1`](https://togithub.com/fluxcd/flux2/releases/tag/v2.0.1) [Compare Source](https://togithub.com/fluxcd/flux2/compare/v2.0.0...v2.0.1) #### Highlights Flux `v2.0.1` is a patch release which comes with various fixes. Users are encouraged to upgrade for the best experience. :bulb: For upgrading from Flux `v0.x`, please see [the procedure documented in 2.0.0](https://togithub.com/fluxcd/flux2/releases/tag/v2.0.0). ##### Fixes - Fix AWS auth for cross-region ECR repositories (`source-controller`, `image-reflector-controller`). - Prevent spurious alerts for skipped resources (`kustomize-controller`). - List removed resources for `flux diff ks --kustomization-file` (`flux` CLI). - Fix SLSA provenance generation for the Flux CLI binaries. #### Components changelog - source-controller [v1.0.1](https://togithub.com/fluxcd/source-controller/blob/v1.0.1/CHANGELOG.md) - kustomize-controller [v1.0.1](https://togithub.com/fluxcd/kustomize-controller/blob/v1.0.1/CHANGELOG.md) - image-reflector-controller [v0.29.1](https://togithub.com/fluxcd/image-reflector-controller/blob/v0.29.1/CHANGELOG.md) #### CLI Changelog - PR [#4068](https://togithub.com/fluxcd/flux2/issues/4068) - [@stefanprodan](https://togithub.com/stefanprodan) - Update dependencies - PR [#4065](https://togithub.com/fluxcd/flux2/issues/4065) - [@hiddeco](https://togithub.com/hiddeco) - action: support `openssl` and `sha256sum` - PR [#4062](https://togithub.com/fluxcd/flux2/issues/4062) - [@souleb](https://togithub.com/souleb) - diff: Take into account the server-side inventory for local Flux Kustomizations - PR [#4061](https://togithub.com/fluxcd/flux2/issues/4061) - [@hiddeco](https://togithub.com/hiddeco) - action: re-allow configuration of non-default token - PR [#4057](https://togithub.com/fluxcd/flux2/issues/4057) - [@fluxcdbot](https://togithub.com/fluxcdbot) - Update toolkit components - PR [#4052](https://togithub.com/fluxcd/flux2/issues/4052) - [@stefanprodan](https://togithub.com/stefanprodan) - docs: Link to the Flux GitHub Action documentation - PR [#4051](https://togithub.com/fluxcd/flux2/issues/4051) - [@hiddeco](https://togithub.com/hiddeco) - action: use `$RUNNER_TOOL_CACHE`, support MacOS and Windows, validate checksum - PR [#4046](https://togithub.com/fluxcd/flux2/issues/4046) - [@stefanprodan](https://togithub.com/stefanprodan) - ci: backport: set write permissions - PR [#4043](https://togithub.com/fluxcd/flux2/issues/4043) - [@stefanprodan](https://togithub.com/stefanprodan) - ci: release: extract the image tag from GITHUB_REF - PR [#4041](https://togithub.com/fluxcd/flux2/issues/4041) - [@hiddeco](https://togithub.com/hiddeco) - ci: release: disable interpretation backslash esc #### New Documentation - [Flux GitHub Action](https://fluxcd.io/flux/flux-gh-action/) - [SLSA provenance verification](https://fluxcd.io/flux/security/slsa-assessment/#provenance-verification) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/scottames/dots).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

Follow up of cli#7612 The `missingWorkflowScopeRE` is defined to capture the error message when the `GH_TOKEN` does not have `workflow` scope in `gh repo sync <remote>`, but this is only intended for error messages for OAuth Apps and does not work with GitHub Apps. In GitHub App, you will get the following error: ``` { "message": "refusing to allow a GitHub App to create or update workflow `.github/workflows/teamcity-pr-checks.yml` without `workflows` permission", "documentation_url": "https://docs.github.com/rest/branches/branches#sync-a-fork-branch-with-the-upstream-repository", "status": "422" } ``` As you can see above, the existing regexp does not match the "`workflows` permission". This change modifies the regexp to return the user-friendly error message when the `workflow` permission is missing, even in the case of a GitHub App.

Early exit repo sync if merge-upstream requires workflow scope

fa8b514

williammartin requested a review from a team as a code owner June 21, 2023 14:03

williammartin requested review from samcoe and removed request for a team June 21, 2023 14:03

cliAutomation assigned williammartin Jun 21, 2023

williammartin commented Jun 21, 2023

View reviewed changes

williammartin mentioned this pull request Jun 21, 2023

HttpError: There was a problem updating the branch issue mislav/bump-homebrew-formula-action#52

Closed

samcoe approved these changes Jun 22, 2023

View reviewed changes

williammartin merged commit ddd27a5 into trunk Jun 23, 2023

williammartin deleted the wm/repo-sync-workflow-scope branch June 23, 2023 06:52

williammartin mentioned this pull request Feb 7, 2024

gh repo sync not working #7574

Closed

williammartin mentioned this pull request Mar 10, 2025

gh repo sync fails with surprising error if workflow scope is missing and token is from GitHub App #10573

Closed

wata727 mentioned this pull request Mar 10, 2025

[gh repo sync] Make missing workflow regexp aware of GitHub App #10574

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Early exit repo sync if merge-upstream requires workflow scope #7612

Early exit repo sync if merge-upstream requires workflow scope #7612

Uh oh!

williammartin commented Jun 21, 2023 •

edited

Loading

Uh oh!

williammartin Jun 21, 2023

Uh oh!

williammartin Jun 21, 2023

Uh oh!

samcoe left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Early exit repo sync if merge-upstream requires workflow scope #7612

Early exit repo sync if merge-upstream requires workflow scope #7612

Uh oh!

Conversation

williammartin commented Jun 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Demonstration

Before

After

Reproduction

Limitations

Uh oh!

williammartin Jun 21, 2023

Choose a reason for hiding this comment

Uh oh!

williammartin Jun 21, 2023

Choose a reason for hiding this comment

Uh oh!

samcoe left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

williammartin commented Jun 21, 2023 •

edited

Loading