Give Dependabot triage a real reachability check - #14087
Merged
Merged
Conversation
The triager was asked whether an upstream change can reach this repository, and answered it by grepping our own source for the module's import path. For an indirect dependency that grep always finds nothing, because "indirect" means precisely that we do not import it. Treating the silence as safety is a tautology. It produced a wrong assessment on #14066, where a github.com/docker/cli bump was reported as carrying no risk at High confidence. Five docker/cli packages are compiled into the shipped binary, reached through go-containerregistry/pkg/authn, which the same comment flagged as changed. Replace the inference with the build graph. The pre-flight step now runs `go mod vendor`, which resolves what the module graph actually needs and so is indifferent to who writes the import. Its vendor/modules.txt is a per-module list of the exact packages required, and a union of `go list -deps ./cmd/gh` across the platforms gh ships for separates packages in the binary from ones built only for tests. Vendoring also puts the dependency source in the workspace the agent already has mounted, so it can read the code a release changed instead of reasoning from release notes alone. Both steps are gated on the work list being non-empty, so an idle run still downloads nothing. Missing artifacts degrade the assessment to Medium confidence rather than failing the run, matching how this workflow already handles unreadable CI state. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 18bc01f9-9498-4bd8-9fd0-70308491b695
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request improves the Dependabot triage workflow’s “reachability” evidence by switching from source-tree import greps (which are unreliable for indirect deps) to build-graph-derived artifacts created via go mod vendor and go list -deps, and updates the triage skill to use that evidence when forming recommendations and confidence.
Changes:
- Add a pre-agent vendoring + production dependency enumeration step gated on a non-empty worklist.
- Expose the worklist count as a step output so downstream steps can be conditionally skipped on idle runs.
- Update the
dependabot-triagerskill to classify updated modules usingvendor/modules.txtand the production package list instead of import-site greps.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/dependabot-triage.md | Adds gated vendoring and production package listing to provide reachability evidence to the agent. |
| .github/workflows/dependabot-triage.lock.yml | Regenerates the compiled workflow to include the new step(s) and outputs wiring. |
| .github/skills/dependabot-triager/SKILL.md | Updates required evidence and rubric guidance to use vendored build-graph artifacts for reachability. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 3/3 changed files
- Comments generated: 2
- Review effort level: Lite
The vendored Go artifacts only answer the reachability question for Go module bumps, but most Dependabot traffic in this repository bumps GitHub Actions. As written, an Actions bump would find the action absent from both artifacts and could report it as "not built at all", which is the same false-safety inference this change set exists to remove, just reached by a different route. Split evidence item 3 by ecosystem. Go modules use the build graph. Actions use a grep of `.github/` for `uses:` lines, which is a valid method there precisely because a workflow can only reach an action by naming it in our own files, so there is no equivalent of an indirect dependency our source never mentions. Anything else has no mechanical method, and says so at Medium confidence. Gate the vendoring on a Go manifest actually moving, read from the PR's file list, so an Actions-only run no longer downloads tens of megabytes to produce evidence the agent is told to ignore. An unreadable file list falls back to vendoring, so a transient API failure costs wasted work rather than missing evidence. Co-authored-by: Copilot App <[email protected]> Copilot-Session: 18bc01f9-9498-4bd8-9fd0-70308491b695
Co-authored-by: Copilot App <[email protected]> Copilot-Session: 18bc01f9-9498-4bd8-9fd0-70308491b695
tmeijn
pushed a commit
to tmeijn/dotfiles
that referenced
this pull request
Aug 21, 2026
This MR contains the following updates: | Package | Update | Change | |---|---|---| | [cli/cli](https://github.com/cli/cli) | minor | `v2.97.0` → `v2.98.0` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>cli/cli (cli/cli)</summary> ### [`v2.98.0`](https://github.com/cli/cli/releases/tag/v2.98.0): GitHub CLI 2.98.0 [Compare Source](cli/cli@v2.97.0...v2.98.0) #### Security A security vulnerability has been identified, and fixed, that binds the local forwarded port to all available network interfaces by default. Users of `gh codespace ports forward` are advised to update `gh` to version `v2.98.0` as soon as possible. For more information see: <GHSA-vfhh-p7hm-pxfh> #### Support worktrees in `pr checkout` Users can now checkout a pull request into a git worktree by using the new `--worktree PATH` flag in `gh pr checkout`: ```shell gh pr checkout 12 --worktree ../wt-feature ``` #### Add semantic search to `search issues` The `gh search issues` command now supports semantic search for issues. Users can select the search type by passing the `--search-type` flag: ```shell gh search issues --search-type semantic ... gh search issues --search-type hybrid ... ``` For more information about semantic search see: ["Improved Search for github issues is now generally available"](https://github.blog/changelog/2026-04-02-improved-search-for-github-issues-is-now-generally-available/). #### What's Changed ##### ✨ Features - Add --worktree flag to gh pr checkout by [@​tidy-dev](https://github.com/tidy-dev) in [#​13946](cli/cli#13946) - Set GH\_EXTENSION=1 when gh invokes an extension by [@​williammartin](https://github.com/williammartin) in [#​14072](cli/cli#14072) - Add --search-type flag for semantic and hybrid issue search by [@​michaeljacholke](https://github.com/michaeljacholke) in [#​14006](cli/cli#14006) ##### 🐛 Fixes - Fix `RESTWithNext` error type, repairing `gh status` and attestation retries by [@​williammartin](https://github.com/williammartin) in [#​13988](cli/cli#13988) - Trim spaces when parsing X-Oauth-Scopes in `gh release create` by [@​williammartin](https://github.com/williammartin) in [#​14065](cli/cli#14065) - Fix project item-add output for non-TTY by [@​zwick](https://github.com/zwick) in [#​14056](cli/cli#14056) ##### 📚 Docs & Chores - Slim down dependabot triage comments by [@​williammartin](https://github.com/williammartin) in [#​14019](cli/cli#14019) - Require explicit MR review ownership by [@​williammartin](https://github.com/williammartin) in [#​14028](cli/cli#14028) - Collapse spam triage into the agentic issue-triage workflow by [@​williammartin](https://github.com/williammartin) in [#​14027](cli/cli#14027) - Run Dependabot triage every hour by [@​sergiou87](https://github.com/sergiou87) in [#​14030](cli/cli#14030) - Route deploy key requests through api.Client by [@​williammartin](https://github.com/williammartin) in [#​13989](cli/cli#13989) - Route ssh key requests through api.Client by [@​williammartin](https://github.com/williammartin) in [#​13994](cli/cli#13994) - Route gpg key requests through api.Client by [@​williammartin](https://github.com/williammartin) in [#​13997](cli/cli#13997) - Route autolink requests through api.Client by [@​williammartin](https://github.com/williammartin) in [#​14013](cli/cli#14013) - Route extension requests through api.Client by [@​williammartin](https://github.com/williammartin) in [#​14059](cli/cli#14059) - Route release creation through api.Client by [@​williammartin](https://github.com/williammartin) in [#​14062](cli/cli#14062) - Tell agents to use the MR template in AGENTS.md by [@​williammartin](https://github.com/williammartin) in [#​14074](cli/cli#14074) - Make Dependabot triage cheaper and more decisive by [@​williammartin](https://github.com/williammartin) in [#​14079](cli/cli#14079) - Route release deletions through api.Client by [@​williammartin](https://github.com/williammartin) in [#​14077](cli/cli#14077) - Give Dependabot triage a real reachability check by [@​williammartin](https://github.com/williammartin) in [#​14087](cli/cli#14087) - Restore automatic spam issue closure by [@​williammartin](https://github.com/williammartin) in [#​14088](cli/cli#14088) - Add a scheduled tech debt burndown skill by [@​williammartin](https://github.com/williammartin) in [#​14095](cli/cli#14095) - Use reflect.Pointer instead of deprecated reflect.Ptr by [@​williammartin](https://github.com/williammartin) in [#​14098](cli/cli#14098) - Clarify what belongs in the MR template's testing section by [@​williammartin](https://github.com/williammartin) in [#​14103](cli/cli#14103) - Rename cli-code-reviewer skill to code-review by [@​BagToad](https://github.com/BagToad) in [#​14116](cli/cli#14116) - Add aw-actions group to dependabot configuration by [@​babakks](https://github.com/babakks) in [#​14123](cli/cli#14123) - Isolate tests from local machine's auth and git configuration by [@​BagToad](https://github.com/BagToad) in [#​14128](cli/cli#14128) - Don't ask for feature detection cleanup comments when not needed by [@​babakks](https://github.com/babakks) in [#​14139](cli/cli#14139) - Accept pre-release tags in deployment validation by [@​BagToad](https://github.com/BagToad) in [#​14193](cli/cli#14193) - ci: add temporary step to verify Linux repo signing keys by [@​babakks](https://github.com/babakks) in [#​14202](cli/cli#14202) - Revert "ci: add temporary step to verify Linux repo signing keys" by [@​babakks](https://github.com/babakks) in [#​14203](cli/cli#14203) - Fix issue triage action compatibility \[skip changelog] by [@​tidy-dev](https://github.com/tidy-dev) in [#​14207](cli/cli#14207) #####Dependencies - chore(deps): bump github.com/sigstore/sigstore-go from 1.2.2 to 1.3.0 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14047](cli/cli#14047) - chore(deps): bump the codeql-actions group across 1 directory with 3 updates by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14049](cli/cli#14049) - chore(deps): bump google.golang.org/grpc from 1.82.1 to 1.83.0 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14048](cli/cli#14048) - chore(deps): bump github.com/google/go-containerregistry from 0.21.7 to 0.21.8 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14066](cli/cli#14066) - chore(deps): bump actions/attest from 4.2.1 to 4.2.2 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14100](cli/cli#14100) - chore(deps): bump azure/login from 3.0.0 to 3.0.1 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14101](cli/cli#14101) - chore(deps): bump the codeql-actions group across 1 directory with 3 updates by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14091](cli/cli#14091) - chore(deps): bump github/gh-aw-actions/setup-cli from 0.83.4 to 0.85.4 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14068](cli/cli#14068) - chore(deps): bump github.com/google/go-containerregistry from 0.21.8 to 0.21.9 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14119](cli/cli#14119) - chore(deps): bump github.com/klauspost/compress from 1.19.1 to 1.19.2 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14120](cli/cli#14120) - chore(deps): bump the aw-actions group with 2 updates by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14147](cli/cli#14147) - chore: sign APT repository with both keys by [@​babakks](https://github.com/babakks) in [#​13271](cli/cli#13271) - chore(deps): bump github.com/yuin/goldmark from 1.8.4 to 1.8.5 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14029](cli/cli#14029) - chore(deps): bump actions/attest from 4.2.0 to 4.2.1 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14050](cli/cli#14050) - Bump golangci-lint in CI to v2.12.2 by [@​williammartin](https://github.com/williammartin) in [#​14102](cli/cli#14102) - chore(deps): bump the aw-actions group with 2 updates by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14124](cli/cli#14124) - chore(deps): bump google.golang.org/protobuf from 1.36.11 to 1.36.12 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14140](cli/cli#14140) - Upgrade gh-aw workflows to v0.85.4 by [@​tidy-dev](https://github.com/tidy-dev) in [#​14141](cli/cli#14141) - Bump Go to 1.26.6 by [@​github-actions](https://github.com/github-actions)\[bot] in [#​14143](cli/cli#14143) - chore: bump go to 1.26.7 by [@​babakks](https://github.com/babakks) in [#​14205](cli/cli#14205) - chore(deps): bump github.com/stretchr/testify from 1.11.1 to 1.12.1 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14204](cli/cli#14204) - chore(deps): bump the codeql-actions group across 1 directory with 3 updates by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14169](cli/cli#14169) - chore(deps): bump golang.org/x/crypto from 0.54.0 to 0.55.0 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14164](cli/cli#14164) - chore(deps): bump charm.land/lipgloss/v2 from 2.0.5 to 2.0.6 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​14166](cli/cli#14166) - Bump gh-aw-actions to v0.87.1 and recompile agentic workflows by [@​BagToad](https://github.com/BagToad) in [#​14210](cli/cli#14210) #### New Contributors - [@​sergiou87](https://github.com/sergiou87) made their first contribution in [#​14030](cli/cli#14030) - [@​michaeljacholke](https://github.com/michaeljacholke) made their first contribution in [#​14006](cli/cli#14006) **Full Changelog**: <cli/cli@v2.97.0...v2.98.0> </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Mend Renovate](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4yODguMCIsInVwZGF0ZWRJblZlciI6IjQzLjI4OC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiLCJhdXRvbWF0aW9uOmJvdC1hdXRob3JlZCIsImRlcGVuZGVuY3ktdHlwZTo6bWlub3IiXX0=-->
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #14079.
Description
The Dependabot triage workflow posts an advisory comment on each open Dependabot PR. Part of what it is asked to judge is whether the upstream change can actually reach this repository, since a bump to something we never build is not worth a maintainer's attention and a bump to something we do build is.
It answered that question by grepping our own source tree for the module's import path. That works for a direct dependency. For an indirect one it is a tautology:
// indirectmeans precisely that no code we wrote imports it, so the grep is guaranteed to find nothing, and the skill invited reading that silence as evidence of safety.That is not hypothetical. On #14066 the workflow reported, at
Highconfidence:The
docker/clihalf is wrong. Five of its packages are compiled into the shippedghbinary, reached viago-containerregistry/pkg/authn- the very package that same comment flagged as having changed:The
x/toolshalf happened to be right, but by the same reasoning that gotdocker/cliwrong.This replaces the inference with evidence, per ecosystem.
Go modules get the build graph. The pre-flight step runs
go mod vendor, which resolves what the module graph actually needs and is therefore indifferent to who writes the import. That yieldsvendor/modules.txt(the exact packages each module contributes to the build) and a union ofgo list -deps ./cmd/ghacross linux, darwin and windows, separating packages in the shipped binary from ones built only for tests. Vendoring also drops the dependency source into the workspace the agent already has mounted, so when a change is reachable it can read the affected code instead of reasoning from release notes.GitHub Actions get a
uses:grep of.github/. Grepping is the right method there, for the reason it is the wrong one for Go: a workflow can only reach an action by naming it in auses:line in our own files, so there is no equivalent of an indirect dependency our source never mentions.Authorship and follow-up
Who wrote this:
Who answers review comments: