Check extension for latest version when executed #9866

andyfeller · 2024-11-02T17:20:37Z

This PR applies the same approach used for checking and informing users of new gh releases to extensions when they are executed. The benefit being that GitHub CLI users can be notified about new versions as they use the extension without putting that effort onto GitHub CLI extension authors.

Note

This is initially created as a draft for feedback from core maintainers and the community before investing more effort to polish and complete the work.

Demo

andrewfeller@Andrews-MacBook-Pro cli % gh ext install dlvhdr/gh-dash --pin v4.6.0
✓ Installed extension dlvhdr/gh-dash
✓ Pinned extension at v4.6.0

andrewfeller@Andrews-MacBook-Pro cli % ./bin/gh dash

A new release of dash is available: 4.6.0 → 4.7.0
To upgrade, run: gh extension upgrade dash --force
https://github.com/dlvhdr/gh-dash

Open questions

How frequently do we check for new extension releases?

The gh version checking only happens at minimum every 24 hours, relying upon information in XDG state directory on when it was last checked. If the similar approach is desired here, then either the existing state file will be expanded OR gh maintains state information per extension.
```
% cat ~/.local/state/gh/state.yml 
checked_for_update_at: 2024-11-02T11:07:12.423561-04:00
latest_release:
    version: v2.60.1
    url: https://github.com/cli/cli/releases/tag/v2.60.1
    publishedat: 2024-10-25T17:15:00Z
```
What degree of testing is sufficient?

Being based on the existing behavior as Cobra command hooks, it is possible but want to solicit feedback before investing the time to understand any changes to the implementation.

Building on logic from the `gh ext list` for retrieving and assessing extension release information, this commit enhances the logic around invoking extensions to check for new releases. Using the same user experience from checking `gh` version, this should only output information when the extension is used and gives the user information on how to upgrade depending on the type of extension and whether it is pinned or not. ```shell andrewfeller@Andrews-MacBook-Pro cli % gh ext install dlvhdr/gh-dash --pin v4.6.0 ✓ Installed extension dlvhdr/gh-dash ✓ Pinned extension at v4.6.0 andrewfeller@Andrews-MacBook-Pro cli % ./bin/gh dash A new release of dash is available: 4.6.0 → 4.7.0 To upgrade, run: gh extension upgrade dash --force https://github.com/dlvhdr/gh-dash ```

pkg/cmd/root/extension.go

jtmcg

I like it! I'd ship the version-check as-is, I think, but you were looking for feedback so I've left a couple of questions.

Additionally, it's not clear to me what the inclusion of licenses.tmpl and licenses.yml files are for, here. Can you explain them to me?

pkg/cmd/root/extension.go

This commit expands upon the previous work by creating tests around extension command execution and how various extension update scenarios are handled. Along the way, the logic handling formatting update messaging has been switched to use `ColorScheme` in order to honor color behavior flags.

andyfeller · 2024-11-07T04:15:56Z

After taking a second pass at these changes, I'll admit I'm second guessing how much we attempt to apply the same update caching logic used below around gh. I'll explain my reasoning below but here is the code in question:

cli/internal/update/update.go

Lines 22 to 111 in 215e4f4

    
           // ReleaseInfo stores information about a release 
        
           type ReleaseInfo struct { 
        
           	Version     string    `json:"tag_name"` 
        
           	URL         string    `json:"html_url"` 
        
           	PublishedAt time.Time `json:"published_at"` 
        
           } 
        
           type StateEntry struct { 
        
           	CheckedForUpdateAt time.Time   `yaml:"checked_for_update_at"` 
        
           	LatestRelease      ReleaseInfo `yaml:"latest_release"` 
        
           } 
        
           // CheckForUpdate checks whether this software has had a newer release on GitHub 
        
           func CheckForUpdate(ctx context.Context, client *http.Client, stateFilePath, repo, currentVersion string) (*ReleaseInfo, error) { 
        
           	stateEntry, _ := getStateEntry(stateFilePath) 
        
           	if stateEntry != nil && time.Since(stateEntry.CheckedForUpdateAt).Hours() < 24 { 
        
           		return nil, nil 
        
           	} 
        
           	releaseInfo, err := getLatestReleaseInfo(ctx, client, repo) 
        
           	if err != nil { 
        
           		return nil, err 
        
           	} 
        
           	err = setStateEntry(stateFilePath, time.Now(), *releaseInfo) 
        
           	if err != nil { 
        
           		return nil, err 
        
           	} 
        
           	if versionGreaterThan(releaseInfo.Version, currentVersion) { 
        
           		return releaseInfo, nil 
        
           	} 
        
           	return nil, nil 
        
           } 
        
           func getLatestReleaseInfo(ctx context.Context, client *http.Client, repo string) (*ReleaseInfo, error) { 
        
           	req, err := http.NewRequestWithContext(ctx, "GET", fmt.Sprintf("https://api.github.com/repos/%s/releases/latest", repo), nil) 
        
           	if err != nil { 
        
           		return nil, err 
        
           	} 
        
           	res, err := client.Do(req) 
        
           	if err != nil { 
        
           		return nil, err 
        
           	} 
        
           	defer func() { 
        
           		_, _ = io.Copy(io.Discard, res.Body) 
        
           		res.Body.Close() 
        
           	}() 
        
           	if res.StatusCode != 200 { 
        
           		return nil, fmt.Errorf("unexpected HTTP %d", res.StatusCode) 
        
           	} 
        
           	dec := json.NewDecoder(res.Body) 
        
           	var latestRelease ReleaseInfo 
        
           	if err := dec.Decode(&latestRelease); err != nil { 
        
           		return nil, err 
        
           	} 
        
           	return &latestRelease, nil 
        
           } 
        
           func getStateEntry(stateFilePath string) (*StateEntry, error) { 
        
           	content, err := os.ReadFile(stateFilePath) 
        
           	if err != nil { 
        
           		return nil, err 
        
           	} 
        
           	var stateEntry StateEntry 
        
           	err = yaml.Unmarshal(content, &stateEntry) 
        
           	if err != nil { 
        
           		return nil, err 
        
           	} 
        
           	return &stateEntry, nil 
        
           } 
        
           func setStateEntry(stateFilePath string, t time.Time, r ReleaseInfo) error { 
        
           	data := StateEntry{CheckedForUpdateAt: t, LatestRelease: r} 
        
           	content, err := yaml.Marshal(data) 
        
           	if err != nil { 
        
           		return err 
        
           	} 
        
           	err = os.MkdirAll(filepath.Dir(stateFilePath), 0755) 
        
           	if err != nil { 
        
           		return err 
        
           	} 
        
           	err = os.WriteFile(stateFilePath, content, 0600) 
        
           	return err 
        
           }

Reasoning

The gh update checking logic is simple and straightforward:

gh has releases so we can make API calls to pull information
We're only checking a single GitHub repo for releases

Contrast that with extensions:

There are 3 types of extensions with very different mechanisms around version handling
1. Local extension
2. Remote script-based extension
3. Remote binary-based extension
Local + remote binary-based extensions have manifests similar to gh update state information however not exactly the same
These manifests are are the domain of the extension manager, possibly requiring a major refactor if we want to cache information per extension

With all of that said, I think there's enough done on the PR to bring it out of draft for further feedback.

andyfeller · 2024-11-07T04:17:52Z

I like it! I'd ship the version-check as-is, I think, but you were looking for feedback so I've left a couple of questions.

Additionally, it's not clear to me what the inclusion of licenses.tmpl and licenses.yml files are for, here. Can you explain them to me?

I'll be honest: I have no idea how that landed here.

jtmcg

Looks good to me! I've a tiny comment, but it's non-blocking 🚀

pkg/cmd/root/extension_test.go

pkg/cmd/root/extension.go

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [cli/cli](https://github.com/cli/cli) | minor | `v2.59.0` -> `v2.62.0` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>cli/cli (cli/cli)</summary> ### [`v2.62.0`](https://github.com/cli/cli/releases/tag/v2.62.0): GitHub CLI 2.62.0 [Compare Source](cli/cli@v2.61.0...v2.62.0) #### What's Changed - Update monotonic verification logic and testing by [@malancas](https://github.com/malancas) in cli/cli#9856 - Check extension for latest version when executed by [@andyfeller](https://github.com/andyfeller) in cli/cli#9866 - Shorten extension release checking from 3s to 1s by [@andyfeller](https://github.com/andyfeller) in cli/cli#9914 - Mention GitHub CLI team on discussion issues by [@andyfeller](https://github.com/andyfeller) in cli/cli#9920 **Full Changelog**: cli/cli@v2.61.0...v2.62.0 #### Security - A security vulnerability has been identified in GitHub CLI that could allow remote code execution (RCE) when users connect to a malicious Codespace SSH server and use the `gh codespace ssh` or `gh codespace logs` commands. For more information, see GHSA-p2h2-3vg9-4p87 #### GitHub CLI notifies users about latest extension upgrades Similar to the notification of latest `gh` releases, the `v2.62.0` version of GitHub CLI will notify users about latest extension upgrades when the extension is used: ```shell $ gh ado2gh ... A new release of ado2gh is available: 1.7.0 → 1.8.0 To upgrade, run: gh extension upgrade ado2gh --force https://github.com/github/gh-ado2gh ``` ##### Why does this matter? This removes a common pain point of extension authors as they have had to reverse engineer and implement a similar mechanism within their extensions directly. With this quality of life improvement, there are 2 big benefits: 1. Extension authors will hopefully see increased adoption of newer releases while having lower bar to maintaining their extensions. 2. GitHub CLI users will have greater awareness of new features, bug fixes, and security fixes to the extensions used. ##### What do you need to do? Extension authors should review their extensions and consider removing any custom logic previously implemented to notify users of new releases. ### [`v2.61.0`](https://github.com/cli/cli/releases/tag/v2.61.0): GitHub CLI 2.61.0 [Compare Source](cli/cli@v2.60.1...v2.61.0) #### Ensure users understand consequences before making repository visibility changes In `v2.61.0`, `gh repo edit` command has been enhanced to inform users about [consequences of changing visibility](https://gh.io/setting-repository-visibility) and ensure users are intentional before making irreversible changes: 1. Interactive `gh repo edit` visibility change requires confirmation when changing from `public`, `private`, or `internal` 2. Non-interactive `gh repo edit --visibility` change requires new `--accept-visibility-change-consequences` flag to confirm 3. New content to inform users of consequences - Incorporate [GitHub Docs content](https://gh.io/setting-repository-visibility) into help usage and interactive `gh repo edit` experience - Expanded help usage to call out most concerning consequences - Display repository star and watcher counts to understand impact before confirming #### What's Changed - Add acceptance test for `project` command by [@jtmcg](https://github.com/jtmcg) in cli/cli#9816 - Add comprehensive testscript for `gh ruleset` by [@andyfeller](https://github.com/andyfeller) in cli/cli#9815 - Add comprehensive testscript for gh ext commandset by [@andyfeller](https://github.com/andyfeller) in cli/cli#9810 - Require visibility confirmation in `gh repo edit` by [@andyfeller](https://github.com/andyfeller) in cli/cli#9845 - Clean up skipped online tests for `gh attestation verify` by [@malancas](https://github.com/malancas) in cli/cli#9838 - `gh attestation verify` should only verify provenance attestations by default by [@malancas](https://github.com/malancas) in cli/cli#9825 - Set `dnf5` commands as default by [@its-miroma](https://github.com/its-miroma) in cli/cli#9844 - Fix verbiage for deleting workflow runs by [@akx](https://github.com/akx) in cli/cli#9876 - Bump github.com/creack/pty from 1.1.23 to 1.1.24 by [@dependabot](https://github.com/dependabot) in cli/cli#9862 - `gh attestation verify` policy enforcement refactor by [@malancas](https://github.com/malancas) in cli/cli#9848 - Simplify Sigstore verification result handling in `gh attestation verify` by [@malancas](https://github.com/malancas) in cli/cli#9877 - Print empty array for `gh cache list` when `--json` is provided by [@williammartin](https://github.com/williammartin) in cli/cli#9883 - Bump actions/attest-build-provenance from 1.4.3 to 1.4.4 by [@dependabot](https://github.com/dependabot) in cli/cli#9884 - Create the automatic key when specified with -i by [@cmbrose](https://github.com/cmbrose) in cli/cli#9881 - fix: `gh pr create -w` ignore template flag by [@nilvng](https://github.com/nilvng) in cli/cli#9863 #### New Contributors - [@akx](https://github.com/akx) made their first contribution in cli/cli#9876 - [@nilvng](https://github.com/nilvng) made their first contribution in cli/cli#9863 **Full Changelog**: cli/cli@v2.60.1...v2.61.0 ### [`v2.60.1`](https://github.com/cli/cli/releases/tag/v2.60.1): GitHub CLI 2.60.1 [Compare Source](cli/cli@v2.60.0...v2.60.1) This is a small patch release to fix installing `gh` via `go install` which was broken with v2.60.0. #### What's Changed - Update testscript to use hard fork by [@williammartin](https://github.com/williammartin) in cli/cli#9821 **Full Changelog**: cli/cli@v2.60.0...v2.60.1 ### [`v2.60.0`](https://github.com/cli/cli/releases/tag/v2.60.0): GitHub CLI 2.60.0 [Compare Source](cli/cli@v2.59.0...v2.60.0) #### What's Changed - Add ArchivedAt field by [@tsukasaI](https://github.com/tsukasaI) in cli/cli#9790 - Include startedAt, completedAt in run steps data by [@andyfeller](https://github.com/andyfeller) in cli/cli#9774 - Adjust environment help for host and tokens by [@williammartin](https://github.com/williammartin) in cli/cli#9809 - Add handling of empty titles for Issues and MRs by [@jtmcg](https://github.com/jtmcg) in cli/cli#9701 - `LiveSigstoreVerifier.Verify` should error if no attestations are present by [@phillmv](https://github.com/phillmv) in cli/cli#9742 - `gh at verify` retries fetching attestations if it receives a 5xx by [@phillmv](https://github.com/phillmv) in cli/cli#9797 - Prevent local extension installations with invalid names and conflicts with core commands and other extensions by [@BagToad](https://github.com/BagToad) in cli/cli#9794 - Rewrite a sentence in CONTRIBUTING.md by [@muzimuzhi](https://github.com/muzimuzhi) in cli/cli#9772 - Use new GitHub preview terms in `working-with-us.md` by [@BagToad](https://github.com/BagToad) in cli/cli#9800 - Use new GitHub previews terminology in attestation commands' help docs by [@BagToad](https://github.com/BagToad) in cli/cli#9799 - Clarify in README that `gh` is supported on GitHub Enterprise Cloud by [@BagToad](https://github.com/BagToad) in cli/cli#9805 - build(deps): bump github.com/gabriel-vasile/mimetype from 1.4.5 to 1.4.6 by [@dependabot](https://github.com/dependabot) in cli/cli#9752 ##### Acceptance Test Changes - Add acceptance tests for `workflow`, `run`, and `cache` commands by [@BagToad](https://github.com/BagToad) in cli/cli#9766 - Add basic `api` acceptance tests by [@BagToad](https://github.com/BagToad) in cli/cli#9770 - Add acceptance tests for `release` commands by [@BagToad](https://github.com/BagToad) in cli/cli#9771 - Add acceptance tests for `org` and `ssh-key` commands by [@BagToad](https://github.com/BagToad) in cli/cli#9812 - Add acceptance tests for `gh auth` commands by [@jtmcg](https://github.com/jtmcg) in cli/cli#9787 - Add acceptance tests for `repo` commands by [@jtmcg](https://github.com/jtmcg) in cli/cli#9783 - Add acceptance tests for `search` command by [@BagToad](https://github.com/BagToad) in cli/cli#9786 - Add acceptance tests for `variable` commands by [@andyfeller](https://github.com/andyfeller) in cli/cli#978 - Add testscripts for gpg-key and label commands by [@williammartin](https://github.com/williammartin) in cli/cli#9811 - Use forked testscript for token redaction by [@williammartin](https://github.com/williammartin) in cli/cli#9804 - Add acceptance tests for `secret` commands by [@andyfeller](https://github.com/andyfeller) in cli/cli#9782 - Note token redaction in Acceptance test README by [@williammartin](https://github.com/williammartin) in cli/cli#9813 #### New Contributors - [@tsukasaI](https://github.com/tsukasaI) made their first contribution in cli/cli#9790 **Full Changelog**: cli/cli@v2.59.0...v2.60.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

andyfeller and others added 10 commits August 20, 2024 23:42

Prototype licensing workflow

36d622e

Fix working directory error

e42d449

Fix working directory error

67d7b07

Fix go-licenses package targeting

98294a6

Fix Go root path issue

df7c38f

Fix go-licenses template path error

341864e

Fix go-licenses ordering

0ca266b

Fix gh token usage

23acba6

Merge branch 'cli:trunk' into trunk

bb1a6f2

y-me-y reviewed Nov 2, 2024

View reviewed changes

pkg/cmd/root/extension.go Outdated Show resolved Hide resolved

jtmcg reviewed Nov 5, 2024

View reviewed changes

pkg/cmd/root/extension.go Outdated Show resolved Hide resolved

pkg/cmd/root/extension.go Show resolved Hide resolved

pkg/cmd/root/extension.go Outdated Show resolved Hide resolved

pkg/cmd/root/extension.go Outdated Show resolved Hide resolved

andyfeller marked this pull request as ready for review November 7, 2024 04:15

andyfeller requested a review from a team as a code owner November 7, 2024 04:15

andyfeller requested review from jtmcg and williammartin November 7, 2024 04:16

cliAutomation added the external pull request originating outside of the CLI core team label Nov 7, 2024

andyfeller added 2 commits November 6, 2024 23:18

Delete .github/licenses.tmpl

e629443

Delete .github/workflows/licenses.yml

54c0739

jtmcg approved these changes Nov 7, 2024

View reviewed changes

pkg/cmd/root/extension_test.go Show resolved Hide resolved

pkg/cmd/root/extension.go Show resolved Hide resolved

Enhance extension upgrade tests for URL

3ec657d

jtmcg merged commit 9b9e654 into cli:trunk Nov 7, 2024

jtmcg mentioned this pull request Nov 7, 2024

ghx (inspired by npx) to combine gh install owner/gh-extension && gh exec extension into ghx owner/gh-extension #9880

Closed

LangLangBart mentioned this pull request Nov 15, 2024

Add Environment Variable to Skip Latest Extension Version Check #9925

Closed

jtmcg mentioned this pull request Nov 22, 2024

Significant Zsh startup delay after adding gh copilot alias #9950

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Check extension for latest version when executed #9866

Check extension for latest version when executed #9866

Uh oh!

andyfeller commented Nov 2, 2024 •

edited

Loading

Uh oh!

Uh oh!

jtmcg left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andyfeller commented Nov 7, 2024

Uh oh!

andyfeller commented Nov 7, 2024

Uh oh!

jtmcg left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Check extension for latest version when executed #9866

Check extension for latest version when executed #9866

Uh oh!

Conversation

andyfeller commented Nov 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Demo

Open questions

Uh oh!

Uh oh!

jtmcg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andyfeller commented Nov 7, 2024

Reasoning

Uh oh!

andyfeller commented Nov 7, 2024

Uh oh!

jtmcg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

andyfeller commented Nov 2, 2024 •

edited

Loading