Codestin Search App

williammartin · 2026-04-17T12:25:55Z

Description

This PR adds telemetry specific to skills. It is not meant to be in any way comprehensive, just a base line set of interesting data. Most of this was already reviewed on the skills private fork, with the addition of an event when a search has converted to an installation.

Reviewer Notes

I need to review the async fetch repo visibility stuff first, I forgot I'd asked copilot to do it the first time around, so maybe hold off.

Copilot

Pull request overview

This PR adds skills-specific telemetry to the gh skill command suite, including raising telemetry sampling only when skill subcommands actually run and emitting events for skill search/preview/install.

Changes:

Plumb a ghtelemetry.CommandRecorder from NewCmdRoot → NewCmdSkills into skills subcommands, and bump sample rate to SAMPLE_ALL at subcommand RunE time.
Emit new telemetry events: skill_search, skill_search_install, skill_preview, and skill_install, with repo visibility used to gate recording of skill names.
Add unit + acceptance coverage for sampling behavior and telemetry payload contents.

Show a summary per file

File	Description
pkg/cmd/skills/skills_test.go	Adds regression tests ensuring sample rate isn’t set during command construction and is set during actual subcommand execution.
pkg/cmd/skills/skills.go	Updates `NewCmdSkills` to accept and pass through a telemetry recorder to subcommands that emit telemetry.
pkg/cmd/skills/search/search_test.go	Updates constructor usage and adds tests validating `skill_search` and `skill_search_install` events.
pkg/cmd/skills/search/search.go	Adds telemetry dependency, raises sample rate on run, and records search + install-from-results events.
pkg/cmd/skills/preview/preview_test.go	Updates constructor usage and adds tests validating `skill_preview` dimensions and visibility-gated skill name recording.
pkg/cmd/skills/preview/preview.go	Adds telemetry recording for preview, including async repo visibility lookup to decide whether to include skill names.
pkg/cmd/skills/install/install_test.go	Updates constructor usage and adds tests validating `skill_install` dimensions and visibility-gated skill name recording (single + multi-skill).
pkg/cmd/skills/install/install.go	Adds telemetry recording for install, raises sample rate on run, and includes async repo visibility lookup to gate skill name recording.
pkg/cmd/root/root.go	Passes the root telemetry recorder down into `NewCmdSkills`.
internal/telemetry/fake.go	Introduces `CommandRecorderSpy` test double to assert `SetSampleRate` calls and sample rate values.
internal/skills/discovery/discovery_test.go	Adds tests for `FetchRepoVisibility`.
internal/skills/discovery/discovery.go	Adds repo visibility fetching + async “future” helper; refactors a few response structs to inline structs.
acceptance/testdata/skills/skills-search.txtar	Adjusts search invocation and adds acceptance assertions for `skill_search` telemetry payload.
acceptance/testdata/skills/skills-search-page.txtar	Adjusts paginated search invocation to include `--owner`.
acceptance/testdata/skills/skills-preview.txtar	Adds acceptance assertions for `skill_preview` telemetry payload.
acceptance/testdata/skills/skills-install.txtar	Adds acceptance assertions for `skill_install` telemetry payload.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 16/16 changed files
Comments generated: 1

The VisibilityFuture/FetchRepoVisibilityAsync/Wait wrapper was an unidiomatic async abstraction built for a single pattern used in exactly two call sites. In Go the channel is already the future; wrapping it in a struct with a Wait(timeout) method adds no value. Delete the abstraction and inline a local visResult struct, buffered channel, goroutine, and select at each call site. Behavior is preserved exactly: err -> "unknown", timeout -> "unknown", success+public -> include skill_names. FetchRepoVisibility (synchronous) is kept as-is. Co-authored-by: Copilot <[email protected]>

babakks

LGTM, with some questions.

babakks · 2026-04-17T14:18:40Z

+	// Kick off the visibility fetch in parallel with the preview work so
+	// the extra API roundtrip doesn't add latency on the critical path.
+	// The result is consumed when the telemetry event is emitted below.
+	type visResult struct {
+		vis discovery.RepoVisibility
+		err error
+	}
+	visCh := make(chan visResult, 1)
+	go func() {
+		vis, err := discovery.FetchRepoVisibility(apiClient, hostname, owner, repoName)
+		visCh <- visResult{vis: vis, err: err}
+	}()
+


We should definitely extract this fire-and-wait implementation as a reusable function. I'd suggest us doing something like this:

type RepoVisibilityResult struct { Visibility discovery.Visibility Error error } func FetchRepoVisibility(ctx context.Context) <-chan RepoVisibilityResult { // make a channel (buffered with size 1), kick off a go routine and return the channel. // the go-routine should take care of the context and deadline }

On the caller side, we create a context with 2s timeout, call FetchRepoVisibility, grab the channel, and then do a blocking-fetch fro the channel at the end.

I know it'd be a bit weird (as you normally don't block on reading channels without a select) but I'm okay with it in favour of reusability.

This doesn't work, or at least, most of the value is lost because if you pass the context.WithTimeout here you bound the fetch totally. We don't care how long the fetch takes until it blocks our progress later on.

This conversation did highlight to me that we definitely don't need to give 2s grace period to that API request given all the stuff that happens in between, reducing it way down.

I don't see why it wouldn't work. I mean we just do:

ch := FetchRepoVisiblity(ctx) // ... // Grab the value (probably already available). Also it shouldn't block beyond the given timeout. visibility := <-ch

But I'm also happy to leave it as is.

Well, because the context.WithTimeout after 2 seconds is going to stop anything landing on the channel? Maybe if you sketch it out more.

SamMorrowDrums

One typo, nothing noticed that hasn't been asked. I think this pretty much good to go.

Production telemetry emission changed: - preview: skill_owner/skill_repo/skill_name (renamed from skill_names) are now emitted only when repo_visibility=public. - install: skill_owner/skill_repo/skill_names are now emitted only when repo_visibility=public. - search: the initial skill_search event was removed entirely; the skill_search_install event no longer carries query/owner dims. Update tests to match: rename skill_names -> skill_name in preview, make owner/repo assertions conditional on public visibility in both preview and install, and reduce the search test to a single event with explicit Empty assertions for the removed query/owner dims so a privacy regression cannot pass silently. Co-authored-by: Copilot <[email protected]>

Add TestCategorizeHost covering all four classification branches (github.com, ghes, tenancy, uncategorized) with cases verified against the real ghauth implementation rather than guessed. Update install and preview unit tests to assert the new skill_host_type dimension name, and fix a typo in the preview acceptance txtar (skill_hos_type -> skill_host_type). Co-authored-by: Copilot <[email protected]>

@babakks

The 2s visibilityWaitTimeout was wildly overprovisioned: by the time telemetry emission reaches the select, the command has already done several serial GitHub REST calls (and for install, a git sparse-checkout plus possibly interactive prompts), so the one-call visibility fetch has almost always completed. Drop the timeout to 200ms — a short safety net for a stalled REST call, not a wait budget for a healthy one. Also adds a table-driven case to TestFetchRepoVisibility covering an unknown/future visibility value from the API, addressing @babakks' review nitpick. Co-authored-by: Copilot <[email protected]>

babakks

LGTM! Thanks for the changes.

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [cli/cli](https://github.com/cli/cli) | minor | `v2.90.0` → `v2.92.0` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>cli/cli (cli/cli)</summary> ### [`v2.92.0`](https://github.com/cli/cli/releases/tag/v2.92.0): GitHub CLI 2.92.0 [Compare Source](cli/cli@v2.91.0...v2.92.0) #### Support GitHub Enterprise Cloud (GHEC) in `skill` commandset Now `gh skill` subcommands (`install`, `preview`, `publish`, `search`, `update`) are able to work with [GHEC](https://docs.github.com/en/enterprise-cloud@latest/admin/overview/about-github-enterprise-cloud) hosts with data residency. #### Add `--allow-hidden-dirs` flag to `skill preview` Following the addition of `--allow-hidden-dirs` to `skill install` in the previous release, now the flag is also supported in `skill preview`, allowing users to preview skills located in hidden (dot-prefixed) directories such as `.claude/skills/`, `.agents/skills/`, and `.github/skills/`. #### What's Changed ##### ✨ Features - feat(skills): add --allow-hidden-dirs flag to preview command by [@SamMorrowDrums](https://github.com/SamMorrowDrums) in [#13265](cli/cli#13265) - feat(skills): support GHEC with data residency hosts by [@SamMorrowDrums](https://github.com/SamMorrowDrums) in [#13264](cli/cli#13264) ##### 🐛 Fixes - Fix SetSampleRate not updating sample\_rate dimension by [@williammartin](https://github.com/williammartin) in [#13259](cli/cli#13259) - Fix log terminal injection by [@williammartin](https://github.com/williammartin) in [#13272](cli/cli#13272) - Add "Resource not accessible" to ProjectsV2IgnorableError by [@maxbeizer](https://github.com/maxbeizer) in [#13281](cli/cli#13281) ##### 📚 Docs & Chores - fix: using variable interpolation \`${{ in deployment.yml... by [@orbisai0security](https://github.com/orbisai0security) in [#13258](cli/cli#13258) - docs: correct typo in Linux Homebrew copy by [@cassidyjames](https://github.com/cassidyjames) in [#13273](cli/cli#13273) - Install skills flat by Name, not namespaced InstallName by [@SamMorrowDrums](https://github.com/SamMorrowDrums) in [#13266](cli/cli#13266) - chore: fix zsh completion on debian by [@babakks](https://github.com/babakks) in [#13274](cli/cli#13274) - Add trust disclaimer to extension help text by [@travellertales](https://github.com/travellertales) in [#13296](cli/cli#13296) - Bump Go to 1.26.2 by [@github-actions](https://github.com/github-actions)\[bot] in [#13301](cli/cli#13301) #####

Dependencies - chore(deps): bump github.com/mattn/go-isatty from 0.0.20 to 0.0.21 by [@dependabot](https://github.com/dependabot)\[bot] in [#13161](cli/cli#13161) - chore(deps): bump github.com/google/go-containerregistry from 0.21.4 to 0.21.5 by [@dependabot](https://github.com/dependabot)\[bot] in [#13162](cli/cli#13162) - chore(deps): bump charm.land/lipgloss/v2 from 2.0.2 to 2.0.3 by [@dependabot](https://github.com/dependabot)\[bot] in [#13163](cli/cli#13163) - chore(deps): bump charm.land/bubbletea/v2 from 2.0.2 to 2.0.6 by [@dependabot](https://github.com/dependabot)\[bot] in [#13206](cli/cli#13206) - chore(deps): bump github.com/gdamore/tcell/v2 from 2.13.8 to 2.13.9 by [@dependabot](https://github.com/dependabot)\[bot] in [#13241](cli/cli#13241) - chore(deps): bump github.com/mattn/go-isatty from 0.0.21 to 0.0.22 by [@dependabot](https://github.com/dependabot)\[bot] in [#13298](cli/cli#13298) #### New Contributors - [@orbisai0security](https://github.com/orbisai0security) made their first contribution in [#13258](cli/cli#13258) - [@cassidyjames](https://github.com/cassidyjames) made their first contribution in [#13273](cli/cli#13273) - [@travellertales](https://github.com/travellertales) made their first contribution in [#13296](cli/cli#13296) **Full Changelog**: <cli/cli@v2.91.0...v2.92.0> ### [`v2.91.0`](https://github.com/cli/cli/releases/tag/v2.91.0): GitHub CLI 2.91.0 [Compare Source](cli/cli@v2.90.0...v2.91.0) #### GitHub CLI now collects pseudonymous telemetry To better understand how features are used in practice, especially as agentic adoption grows, GitHub CLI now sends pseudonymous telemetry. See [Telemetry](https://cli.github.com/telemetry) for more details on what's collected, why, and how to opt out. #### Support more agents in `gh skill` Thanks to community feedback, `gh` now supports a large number of agent hosts. Run `gh skill install --help` for the list of available agents. #### Improve skill discovery `gh skill install` now adds the `--allow-hidden-dirs` flag to support discovering skills in hidden (dot-prefixed) directories such as `.claude/skills/`, `.agents/skills/`, and `.github/skills/`. #### Detect skills re-published from other sources GitHut CLI now detects if the skill to be installed is re-published from an upstream source and offers the option to install it from there. The `--upstream` flag is also added for non-interactive use cases. #### What's Changed ##### ✨ Features - Add support for installation in multiple agent hosts in `gh skills install` by [@tommaso-moro](https://github.com/tommaso-moro) in [#13209](cli/cli#13209) - Add --allow-hidden-dirs flag to gh skill install by [@SamMorrowDrums](https://github.com/SamMorrowDrums) in [#13213](cli/cli#13213) - Make skill discovery less strict: support nested `skills/` directories by [@SamMorrowDrums](https://github.com/SamMorrowDrums) in [#13235](cli/cli#13235) - feat(skills): detect re-published skills and offer upstream install by [@SamMorrowDrums](https://github.com/SamMorrowDrums) in [#13236](cli/cli#13236) ##### 🐛 Fixes - Fix `skills publish --fix` to not publish by [@SamMorrowDrums](https://github.com/SamMorrowDrums) in [#13237](cli/cli#13237) - fix(skills): match skills by install name in preview command by [@SamMorrowDrums](https://github.com/SamMorrowDrums) in [#13249](cli/cli#13249) ##### 📚 Docs & Chores - Remove misleading text by [@tommaso-moro](https://github.com/tommaso-moro) in [#13203](cli/cli#13203) - Add sampled command telemetry by [@williammartin](https://github.com/williammartin) in [#13191](cli/cli#13191) - Do not send telemetry for aliases by [@williammartin](https://github.com/williammartin) in [#13192](cli/cli#13192) - Add skills specific telemetry by [@williammartin](https://github.com/williammartin) in [#13204](cli/cli#13204) - Record CI context in telemetry by [@williammartin](https://github.com/williammartin) in [#13210](cli/cli#13210) - Record official extension telemetry by [@williammartin](https://github.com/williammartin) in [#13205](cli/cli#13205) - Add telemetry command by [@williammartin](https://github.com/williammartin) in [#13253](cli/cli#13253) - Log when there is no telemetry by [@williammartin](https://github.com/williammartin) in [#13255](cli/cli#13255) - docs(skills): add gh and gh-skill agent skills by [@BagToad](https://github.com/BagToad) in [#13244](cli/cli#13244) - Enable telemetry without env var by [@williammartin](https://github.com/williammartin) in [#13254](cli/cli#13254) **Full Changelog**: <cli/cli@v2.90.0...v2.91.0> </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate).

williammartin requested a review from a team April 17, 2026 12:25

williammartin requested a review from a team as a code owner April 17, 2026 12:25

williammartin requested review from BagToad and Copilot and removed request for a team April 17, 2026 12:25

Copilot started reviewing on behalf of williammartin April 17, 2026 12:26 View session

williammartin force-pushed the wm/no-alias-telemetry branch from a63a0e0 to 6709e31 Compare April 17, 2026 12:28

williammartin force-pushed the wm/skills-telemetry branch from cef878a to e0c2fd8 Compare April 17, 2026 12:28

Copilot AI reviewed Apr 17, 2026

View reviewed changes

Comment thread internal/skills/discovery/discovery.go Outdated

Base automatically changed from wm/no-alias-telemetry to trunk April 17, 2026 12:40

Add skills specific telemetry

d8a6a5c

williammartin force-pushed the wm/skills-telemetry branch from e0c2fd8 to d8a6a5c Compare April 17, 2026 12:40

williammartin requested a review from tommaso-moro April 17, 2026 12:41

williammartin assigned SamMorrowDrums Apr 17, 2026

williammartin marked this pull request as draft April 17, 2026 12:42

williammartin and others added 2 commits April 17, 2026 14:57

Fix nonsense copilot tests

ebb4a77

williammartin marked this pull request as ready for review April 17, 2026 13:12

williammartin mentioned this pull request Apr 17, 2026

Record official extension telemetry #13205

Merged

babakks reviewed Apr 17, 2026

View reviewed changes

SamMorrowDrums approved these changes Apr 17, 2026

View reviewed changes

Comment thread internal/telemetry/fake.go Outdated

tommaso-moro approved these changes Apr 17, 2026

View reviewed changes

williammartin and others added 3 commits April 17, 2026 17:09

williammartin requested a review from babakks April 17, 2026 15:54

babakks approved these changes Apr 17, 2026

View reviewed changes

williammartin merged commit 998b621 into trunk Apr 17, 2026
11 checks passed

williammartin deleted the wm/skills-telemetry branch April 17, 2026 17:59

SamMorrowDrums mentioned this pull request Apr 17, 2026

Add --allow-hidden-dirs flag to gh skill install #13213

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add skills specific telemetry#13204

Add skills specific telemetry#13204
williammartin merged 6 commits into
trunkfrom
wm/skills-telemetry

williammartin commented Apr 17, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

babakks left a comment

Uh oh!

Uh oh!

babakks Apr 17, 2026 •

edited

Loading

Uh oh!

williammartin Apr 17, 2026

Uh oh!

williammartin Apr 17, 2026

Uh oh!

babakks Apr 17, 2026

Uh oh!

babakks Apr 17, 2026

Uh oh!

williammartin Apr 17, 2026

Uh oh!

SamMorrowDrums left a comment

Uh oh!

Uh oh!

babakks left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

williammartin commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Reviewer Notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

babakks left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

babakks Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

williammartin Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

williammartin Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

babakks Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

babakks Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

williammartin Apr 17, 2026

Choose a reason for hiding this comment

Uh oh!

SamMorrowDrums left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

babakks left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

williammartin commented Apr 17, 2026 •

edited

Loading

babakks Apr 17, 2026 •

edited

Loading