Codestin Search App

yeelam-gordon · 2026-06-09T00:41:00Z

Adds skills/copilot-pr-autopilot/ — runs the Copilot Code Review → fix → reply+resolve → re-trigger loop on a PR until HEAD is reviewed with zero open Copilot threads.

Repo-agnostic (discovers build/test/lint from CONTRIBUTING/AGENTS/package.json/Makefile). Bundles 10 PowerShell scripts (one per step) + per-step reference docs.

Permissions: the full multi-round autopilot needs repo Triage or Write (GitHub's requestReviewsByLogin is the only public API for bot reviewers and is gated on that). External PR authors get a documented -SingleIteration mode plus manual re-trigger between iterations (UI 🔄 button or synchronize-event push). See SKILL.md → Prerequisites → Permissions for the full matrix.

Validation: npm run skill:validate ✅, npm run build ✅, bash eng/fix-line-endings.sh ✅.

github-actions · 2026-06-09T00:41:26Z

🔍 Skill Validator Results

⚠️ Warnings or advisories found

Scope	Checked
Skills	1
Agents	0
Total	1

Severity	Count
❌ Errors	0
⚠️ Warnings	2
ℹ️ Advisories	0

Summary

Level	Finding
ℹ️	Found 1 skill(s)
ℹ️	[copilot-pr-autopilot] 📊 copilot-pr-autopilot: 3,286 BPE tokens [chars/4: 3,251] (standard ~), 9 sections, 0 code blocks
ℹ️	[copilot-pr-autopilot] ⚠ Skill is 3,286 BPE tokens (chars/4 estimate: 3,251) — approaching "comprehensive" range where gains diminish.
ℹ️	[copilot-pr-autopilot] ⚠ No code blocks — agents perform better with concrete snippets and commands.
ℹ️	✅ All checks passed (1 skill(s))

Full validator output

Found 1 skill(s)
[copilot-pr-autopilot] 📊 copilot-pr-autopilot: 3,286 BPE tokens [chars/4: 3,251] (standard ~), 9 sections, 0 code blocks
[copilot-pr-autopilot]    ⚠  Skill is 3,286 BPE tokens (chars/4 estimate: 3,251) — approaching "comprehensive" range where gains diminish.
[copilot-pr-autopilot]    ⚠  No code blocks — agents perform better with concrete snippets and commands.
✅ All checks passed (1 skill(s))

Copilot

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds the new copilot-pr-autopilot skill, including PowerShell automation scripts and reference docs to drive a “request review → wait → triage → fix → reply/resolve → converge” loop using the gh CLI + GitHub GraphQL.

Changes:

Introduces shared PowerShell helpers (_lib.ps1) and several workflow scripts (request review, snapshot status, list threads, reply/resolve, cleanup outdated).
Adds detailed operational documentation (workflow, API quirks, triage rubric, reply templates) for consistent agent behavior.
Registers the new skill in docs/README.skills.md.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
skills/copilot-pr-autopilot/scripts/_lib.ps1	Adds shared `gh` prerequisite checks and wrappers (`Invoke-Gh`, `Invoke-GhGraphQL`, `Resolve-RepoCoords`) used by all scripts.
skills/copilot-pr-autopilot/scripts/01-request-review.ps1	Requests Copilot review via GraphQL and verifies via `copilot_work_started` event polling.
skills/copilot-pr-autopilot/scripts/02-check-review-status.ps1	Produces a JSON snapshot of PR review state + convergence signals for polling logic.
skills/copilot-pr-autopilot/scripts/03-list-open-threads.ps1	Lists unresolved review threads (all reviewers) for downstream triage.
skills/copilot-pr-autopilot/scripts/08-reply-and-resolve.ps1	Posts a reply to a review thread and optionally resolves it.
skills/copilot-pr-autopilot/scripts/10-cleanup-outdated.ps1	Bulk-resolves outdated Copilot threads as a post-convergence safety net.
skills/copilot-pr-autopilot/references/workflow.md	Documents the loop, delegation map, budgets, and operational contracts.
skills/copilot-pr-autopilot/references/api-quirks.md	Captures verified GitHub API/`gh` quirks that drive implementation choices.
skills/copilot-pr-autopilot/references/03-triage-criteria.md	Defines fix/decline/escalate rubric and reviewer-type policy.
skills/copilot-pr-autopilot/references/06-reply-templates.md	Provides reply templates and anti-patterns for thread responses.
skills/copilot-pr-autopilot/SKILL.md	Defines the skill, prerequisites, workflow, and troubleshooting guidance.
docs/README.skills.md	Adds the skill entry and links to its references/scripts.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 5 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 5 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 5 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

skills/copilot-pr-autopilot/scripts/01-request-review.ps1:1

The thrown/printed error text includes raw gh stderr/body strings ($rawMsg, $r.Stderr, $rb.Stderr) without passing through the credential redactor. If an operator runs with gh --verbose / GH_DEBUG=api or a GitHub error response echoes authorization-like values, this can leak secrets into CI logs. Fix by running these fields through Protect-CopilotPrAutopilotCredentials (or the shared Format-CopilotPrAutopilotErrorPreview) before embedding them in throw/Write-Warning messages.

<#

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated no new comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated no new comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 11 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Copilot

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Skill that drives any GitHub pull request through repeated rounds of Copilot Code Review until the agent has either resolved every thread or explicitly escalated it to the human. Triggered via GraphQL (no @copilot mention needed), triages every open thread with a fix / decline / escalate rubric, replies and resolves each thread citing the pushed SHA, then re-triggers until HEAD is reviewed with zero threads awaiting the agent's reply. Includes step scripts (01 request-review, 02 check-review-status, 03 list-open-threads, 08 reply-and-resolve, 10 cleanup-outdated), shared library (_lib.ps1) with gh-CLI wrappers (Invoke-Gh, Invoke-GhGraphQL, ConvertFrom-GhJson, Assert-GhReady), reply templates, and reference docs for each step. Repo-agnostic. Requires gh CLI on PATH and repo Triage/Write for full autopilot; external PR authors get single-iteration mode with manual re-trigger via the UI re-request button or a substantive push. Co-authored-by: Copilot <[email protected]>

aaronpowell

Interesting idea, but a couple of observations:

Why are some of the steps in the orchestration.md file and others are on their own? Wouldn't it be clearer if they are all separate?
templates folder should be assets
I worry about turn count - doesn't this run the risk of getting itself into a loop?

Copilot AI review requested due to automatic review settings June 9, 2026 00:41

yeelam-gordon requested a review from aaronpowell as a code owner June 9, 2026 00:41

github-actions Bot added new-submission PR adds at least one new contribution skills PR touches skills skill-check-warning Skill validator reported warnings labels Jun 9, 2026

Copilot AI reviewed Jun 9, 2026

View reviewed changes

yeelam-gordon requested a review from Copilot June 9, 2026 12:06

Copilot AI reviewed Jun 9, 2026

View reviewed changes

yeelam-gordon requested a review from Copilot June 9, 2026 12:32

Copilot AI reviewed Jun 9, 2026

View reviewed changes

yeelam-gordon requested a review from Copilot June 9, 2026 12:47

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread skills/copilot-pr-autopilot/scripts/_lib.ps1

Comment thread skills/copilot-pr-autopilot/scripts/01-request-review.ps1 Outdated

Comment thread skills/copilot-pr-autopilot/SKILL.md Outdated

yeelam-gordon requested a review from Copilot June 9, 2026 13:02

github-actions Bot added the skill-check-error Skill validator reported errors label Jun 9, 2026

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread docs/README.skills.md Outdated

Comment thread skills/copilot-pr-autopilot/references/09-convergence.md Outdated

yeelam-gordon force-pushed the add-copilot-review-autopilot-skill branch from c7b2a60 to b73d67f Compare June 9, 2026 13:06

github-actions Bot removed the skill-check-error Skill validator reported errors label Jun 9, 2026

yeelam-gordon requested a review from Copilot June 9, 2026 13:10

yeelam-gordon force-pushed the add-copilot-review-autopilot-skill branch from 3d3642a to 04067ae Compare June 9, 2026 13:11

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread skills/copilot-pr-autopilot/scripts/03-list-open-threads.ps1

Comment thread agency.toml Outdated

Comment thread skills/copilot-pr-autopilot/scripts/10-cleanup-outdated.ps1 Outdated

yeelam-gordon requested a review from Copilot June 9, 2026 13:16

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread skills/copilot-pr-autopilot/scripts/03-list-open-threads.ps1 Outdated

Comment thread skills/copilot-pr-autopilot/scripts/10-cleanup-outdated.ps1

yeelam-gordon requested a review from Copilot June 9, 2026 13:44

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread skills/copilot-pr-autopilot/scripts/01-request-review.ps1 Outdated

Comment thread skills/copilot-pr-autopilot/templates/reply-drift.md Outdated

Comment thread skills/copilot-pr-autopilot/templates/reply-fix.md Outdated

yeelam-gordon requested a review from Copilot June 9, 2026 13:51

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread skills/copilot-pr-autopilot/scripts/10-cleanup-outdated.ps1

Comment thread skills/copilot-pr-autopilot/references/03-list-threads.md Outdated

Comment thread skills/copilot-pr-autopilot/scripts/_lib.ps1 Outdated

yeelam-gordon requested a review from Copilot June 9, 2026 23:20

Copilot AI reviewed Jun 9, 2026

View reviewed changes

Comment thread skills/copilot-pr-autopilot/scripts/03-list-open-threads.ps1

Comment thread skills/copilot-pr-autopilot/scripts/10-cleanup-outdated.ps1

Comment thread skills/copilot-pr-autopilot/scripts/_lib.ps1 Outdated

yeelam-gordon requested a review from Copilot June 10, 2026 00:03