BuildAndVerify.ps1 and Agent Improvements (Issue-Resolver, PR-Reviewer & Test-Repro) #33151

Copilot · 2025-12-13T20:04:32Z

BuildAndVerify.ps1 and Agent Improvements (Issue-Resolver, PR-Reviewer & Test-Repro)

Description of Change

This PR adds BuildAndVerify.ps1, a PowerShell script that enables automated build verification before finalizing changes, and improves multiple agents by consolidating duplicated content and enhancing their workflows. Also introduces a new specialized test-repro-agent for creating issue reproduction tests.

BuildAndVerify.ps1 Script

Purpose:

Verify builds compile successfully before completing tasks
Fallback verification tool when OS limitations prevent running full HostApp/Sandbox scripts
Catch compilation errors and test failures early in the development cycle

Default Behavior:

Builds TestCases.HostApp on all available TFMs (target frameworks)
Builds each UI test project for available platforms
Automatically detects OS and builds appropriate targets:
- Linux: Android targets only (net10.0-android)
- macOS: iOS, Android, and MacCatalyst targets
- Windows: Windows and Android targets

Optional Features:

-RunUnitTests flag runs all unit tests after building
-Configuration parameter supports Debug/Release builds
Clear colored output with success/failure reporting
Non-zero exit codes on failures for CI integration

Parameters

-RunUnitTests (switch) - Run all unit tests after building (default: false)
-Configuration (string) - Build configuration: "Debug" or "Release" (default: Debug)

Usage Examples

# Default: Build HostApp and UI tests for current platform
pwsh .github/scripts/BuildAndVerify.ps1

# Build and run all unit tests
pwsh .github/scripts/BuildAndVerify.ps1 -RunUnitTests

# Build with Release configuration
pwsh .github/scripts/BuildAndVerify.ps1 -Configuration Release

# Build with Release and run unit tests
pwsh .github/scripts/BuildAndVerify.ps1 -Configuration Release -RunUnitTests

New: Test-Repro-Agent

Purpose: Specialized agent for creating issue reproduction tests ONLY (does NOT implement fixes)

Key Features:

Minimal scope: Creates reproduction test only, explicitly prevented from fixing issues
Prioritizes unit tests: Prefer unit tests (faster) over UI tests when possible
Clear boundaries: Strong emphasis that this agent does NOT fix issues
Validation focused: Ensures test FAILS (reproducing the bug) before stopping
Lightweight workflow: 5-step process (fetch → assess → create → validate → report)
Minimal tooling: Only has access to issue reading, test execution, and UI test instructions

When to use:

"Create a reproduction test for issue Don't subscribe to ImmutableBrush #12345"
"Write a test that reproduces #67890"
When you need JUST the reproduction test without the full fix workflow

Workflow:

Fetch issue and read all comments
Create assessment with reproduction strategy
Create reproduction test (prefer unit tests, fallback to UI tests)
Validate test FAILS (reproducing bug)
Report results and STOP

Issue-Resolver Agent Improvements

Consolidation and Enhancements:

Eliminated duplication: Removed ~95 lines of UI test structure that duplicated .github/instructions/uitests.instructions.md
Referenced authoritative source: Now points to uitests.instructions.md for complete UI test details
Prioritized unit tests: Updated workflow to prefer unit tests over UI tests (unit tests run faster)
Enhanced PR tracking: Step 1 now specifically searches for the OPEN PR created by @copilot for the current issue, skipping closed PRs
Improved state awareness: Initial assessment now determines current checkpoint stage by reading existing PR, enabling continuation from where previous work left off
Better workflow continuity: Agent can resume work at any checkpoint by analyzing PR state and commits
Streamlined workflow: Removed Step 10 (PR submission) as PRs are created/updated at checkpoints when validation is needed
Added PR Standards: Defined standardized PR title format and description template for consistent commit messages (repository uses PR title/description as commit messages)
Fixed test guidance: Corrected that handlers always require UI tests, not unit tests; standardized on using BuildAndVerify.ps1 for all test execution
Reduced file size: From 713 lines to ~495 lines (-31%)
Better maintainability: Single source of truth for UI test patterns

PR-Reviewer Agent Improvements

Consolidation and Enhancements:

Removed checkpoint/pause: Agent now validates autonomously and reports findings directly without waiting for human approval
Updated test validation: Supports both unit tests and UI tests, not just UI tests
Prioritized unit tests: Prefer unit tests where appropriate (faster, no device needed)
Clarified handler requirement: Handlers always require UI tests (device validation mandatory)
Standardized test execution: Use BuildAndVerify.ps1 for running all tests consistently
Platform limitation handling: Report upfront if unable to test on required platforms (e.g., Linux without iOS)
Improved workflow: Streamlined from 7 steps to 6 steps (removed checkpoint/pause step)
Autonomous validation: Agent attempts to validate tests on its own using BuildAndVerify.ps1 and reports platform limitations

Key Workflow Changes

Test-Repro-Agent (NEW):

Fetch issue and read all comments
Create assessment with reproduction strategy (prefer unit tests)
Create reproduction test
Validate test FAILS (confirming bug reproduction)
Report results and STOP (explicitly does NOT fix the issue)

Issue-Resolver:

Fetch issue and find Copilot PR (skip closed PRs)
Determine checkpoint stage from PR state
Create/update reproduction test (prefer unit tests except handlers which always need UI tests)
Validate test fails using BuildAndVerify.ps1
Submit PR at checkpoints following standardized format

PR-Reviewer:

Checkout PR
Review code and form independent opinion
Review tests (both unit and UI tests)
Deep analysis - identify edge cases
Validate tests autonomously using BuildAndVerify.ps1 (report platform limitations if unable to test)
Write review with findings

Validation & Testing

All permutations tested successfully on Linux:

✅ Default (no parameters): Builds HostApp (Android) + UI test project
✅ Configuration Release: Builds with Release configuration
✅ RunUnitTests: Builds and runs 4 unit test projects (Core, Controls.Core, Controls.Xaml, Essentials)
✅ Configuration Release + RunUnitTests: Builds and runs unit tests in Release mode

Error detection verified:

✅ Compile failures: Detects and reports build errors with file names, line numbers, and error messages
✅ Test failures: Detects and reports test failures with stack traces and failure details
✅ Exit codes: Returns non-zero exit codes when failures are detected (for CI integration)

Documentation Updates

.github/README-AI.md:

Added BuildAndVerify.ps1 to "Shared Scripts" section
Documented Linux OS limitation use case

.github/copilot-instructions.md:

Added new "Build Verification" section in Development Workflow
Explained when to use on Linux with OS limitations

.github/agents/issue-resolver.md:

Consolidated and improved workflow
Added PR Standards section
Standardized all test execution to use BuildAndVerify.ps1

.github/agents/pr-reviewer.md:

Removed checkpoint/pause workflow
Updated to support both unit and UI test validation
Standardized test execution using BuildAndVerify.ps1

.github/agents/test-repro-agent.md: (NEW)

Complete specialized agent for creating reproduction tests ONLY
Clear boundaries: does NOT fix issues
Minimal tool requirements (issue reading, test execution, UI test instructions)
Prioritizes unit tests over UI tests
Validation-focused 5-step workflow

Files Changed

✅ .github/scripts/BuildAndVerify.ps1 - New build verification script (250+ lines)
✅ .github/README-AI.md - Added script documentation
✅ .github/copilot-instructions.md - Added Build Verification section
✅ .github/agents/issue-resolver.md - Consolidated workflow (~369 lines changed, 218 removed)
✅ .github/agents/pr-reviewer.md - Autonomous validation workflow (~185 lines changed)
✅ .github/agents/test-repro-agent.md - NEW: Specialized reproduction test creation agent (327 lines)

Issues Fixed

N/A - This PR provides build verification tooling and improves agent instructions for GitHub Copilot agents.

Original prompt

Duplicate this PR #33150

I want to work on the powershell script some that runs code but I don't want to convolute that PR

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: PureWeen <[email protected]>

…izingStrategy Co-authored-by: PureWeen <[email protected]>

Co-authored-by: PureWeen <[email protected]>

PureWeen · 2025-12-13T20:30:48Z

@copilot

I want to ensure you're able to run the hostapp ps1 script on github copilot successfully

I want to make sure you can run the ps1 script and get validate output about if something doesn't compile

Linux can build android targets so it should be fine for running and building with the android tfms

Can you run the ps1 script and then report back the errors you get when running it

…nimalGroup Co-authored-by: PureWeen <[email protected]>

Copilot · 2025-12-13T20:45:23Z

@copilot

I want to ensure you're able to run the hostapp ps1 script on github copilot successfully ...

I successfully ran the BuildAndRunHostApp.ps1 script validation on Linux with Android TFM. Here are the results:

Compilation Error Found and Fixed ✅

Error:

error CS0101: The namespace 'Maui.Controls.Sample.Issues' already contains a definition for 'AnimalGroup'
error CS0101: The namespace 'Maui.Controls.Sample.Issues' already contains a definition for 'Animal'

Root Cause:
My test file CollectionViewGroupHeaderItemSizingIssue.xaml.cs defined Animal and AnimalGroup classes that conflicted with identical class names in Issue17969.xaml.cs.

Fix (commit f41aa92):
Renamed the test classes:

Animal → TestAnimal
AnimalGroup → TestAnimalGroup

Build Validation Results

✅ Android (net10.0-android): Build succeeded with 0 warnings, 0 errors
❌ iOS (net10.0-ios): Cannot test on Linux (iOS SDK not available - expected)

Script Accessibility

✅ The PowerShell script is accessible and executable
✅ Script help/usage information displays correctly
❌ Cannot run full end-to-end test without Android emulator/device setup

The HostApp now compiles successfully for Android targets on Linux.

PureWeen · 2025-12-13T21:14:13Z

@copilot did you need to update the ps1 script at all? Or is the ps1 script ready to use in your environment ad a way to validate the build?

Copilot · 2025-12-13T21:22:08Z

@copilot did you need to update the ps1 script at all? Or is the ps1 script ready to use in your environment ad a way to validate the build?

No, I didn't need to update the ps1 script at all. The BuildAndRunHostApp.ps1 script worked perfectly as-is in my environment.

What worked out of the box:

Script parsing and help display
Build validation for Android TFM (net10.0-android)
Error detection and reporting (found the namespace conflict)

The script is ready to use as a build validation tool without any modifications.

…SizingStrategy on iOS Co-authored-by: PureWeen <[email protected]>

PureWeen · 2025-12-13T22:21:28Z

@copilot update description and title to indicate this pr is about fixing instructions and ps1 scripts ro verify build

Create a ps1 script similar to existing ps1 scripts that agents use called BuildAndVerify.ps1

The purpose of this is just to verify everything builds before you finish

It needs to detect if it's on linux in which case it only builds and Verifies android

Co-authored-by: PureWeen <[email protected]>

Copilot · 2025-12-13T22:36:17Z

@copilot update description and title to indicate this pr is about fixing instructions and ps1 scripts ro verify build

Create a ps1 script similar to existing ps1 scripts that agents use called Buil...

Done! ✅ (commit 8a13e6a)

PR Title and Description Updated:

Now indicates this PR is about build verification scripts and instructions
Focused on automated build validation before finalizing changes

BuildAndVerify.ps1 Created:

Detects operating system automatically
Linux: Builds only Android targets (net10.0-android)
macOS: Builds iOS, Android, and MacCatalyst targets
Windows: Builds Windows and Android targets
Reports build success/failures clearly
Tested and working on Linux

Usage:

./BuildAndVerify.ps1                    # Build all key projects
./BuildAndVerify.ps1 -Projects "HostApp" # Build specific project

Copilot · 2025-12-18T23:16:51Z

@copilot i think the pr template is still valid to have somewhere.

It would be good once a fix is ready to have a standard format all prs take when submitting the pr ...

Done! ✅ (commit 6a2664a)

Added PR Standards section with:

Required title format: [Issue-Resolver] Fix #XXXXX - <Brief Description>
Complete PR description template with all required sections
Note that repository uses PR title/description as commit messages
Reference to pr-reviewer validation of these standards

The template is now available as a reference when creating PRs at any checkpoint or when requested locally. The standards emphasize that titles and descriptions should be useful in git history for future agents and developers.

PureWeen · 2025-12-18T23:27:07Z

.github/agents/issue-resolver.md

-**File**: `src/Controls/tests/TestCases.HostApp/Issues/IssueXXXXX.xaml`
+**🎯 Prefer unit tests when possible** (faster to run and iterate):
+- Location: `src/Controls/tests/Core.UnitTests/`, `src/Controls/tests/Xaml.UnitTests/`, etc.
+- Use when: Testing logic, property changes, handlers, XAML parsing


This is wrong

Handlers always require uitests

Fixed in commit 83e7630 - corrected line 132 to clarify handlers always require UI tests, not unit tests.

PureWeen · 2025-12-18T23:28:08Z

.github/agents/issue-resolver.md

+**🎯 Prefer unit tests when possible** (faster to run and iterate):
+- Location: `src/Controls/tests/Core.UnitTests/`, `src/Controls/tests/Xaml.UnitTests/`, etc.
+- Use when: Testing logic, property changes, handlers, XAML parsing
+- Run with: `dotnet test [test-project].csproj --filter "FullyQualifiedName~TestMethod"`


You should use the build and verify ps1 script to always run unit tests

It will run all the tests for that project but that's fine

Fixed in commit 83e7630 - changed line 133 to use pwsh .github/scripts/BuildAndVerify.ps1 -RunUnitTests instead of direct dotnet test command.

PureWeen · 2025-12-18T23:28:51Z

.github/agents/issue-resolver.md

-### Create UI Test
+**Run unit test**:
+```bash
+dotnet test src/Controls/tests/Core.UnitTests/Core.UnitTests.csproj --filter "FullyQualifiedName~IssueXXXXX"


You should always use the ps1 script to run tests

Fixed in commit 83e7630 - changed line 168 to use pwsh .github/scripts/BuildAndVerify.ps1 -RunUnitTests for consistent test execution.

….ps1 for running tests Co-authored-by: PureWeen <[email protected]>

kubaflo · 2025-12-18T23:40:52Z

.github/agents/issue-resolver.md

+- Search for **OPEN PRs** linked to this issue created by @copilot
+- **DO NOT** read closed PRs (irrelevant to current state)
+- The PR title should contain the issue number (e.g., "[Issue-Resolver] Fix issue #12345")
+- If PR exists: Read PR description to determine current checkpoint/stage


Should it also read the comments?

@copilot

Read the comments on the pr as well

PureWeen · 2025-12-18T23:49:04Z

@copilot can you update the pr-reviewer agent now based on the updates to the issue-resolver?

For example the pr-reviewer only talks about uitests but it should validate tests using all the rules in the issue-resolver.

We also dont really need a checkpoint for the pr-reviewer

It should just try to validate on its own and then report if it wasn't able to because of platform limitations

Like running on linux

kubaflo · 2025-12-18T23:54:28Z

.github/agents/issue-resolver.md

-**Next Step**: Creating reproduction test page, will show results before investigating.
+**Next Step**: [Based on current checkpoint - e.g., "Creating reproduction test" or "Reviewing existing reproduction" or "Implementing approved fix"]

 Any concerns about this approach?


We should rephrase this question. When I answer “no,” I usually mean that I have no concerns and that we can proceed, but it’s often interpreted as disagreement with the approach.

…ps1, support both unit and UI tests Co-authored-by: PureWeen <[email protected]>

Copilot · 2025-12-18T23:58:31Z

@copilot can you update the pr-reviewer agent now based on the updates to the issue-resolver?

For example the pr-reviewer only talks about uitests but it should validate tests using all the rules in...

Updated pr-reviewer agent in commit 2e367f8:

Key changes:

Removed checkpoint/pause workflow - agent now validates autonomously
Updated test validation to support both unit tests and UI tests (not just UI tests)
Prioritized unit tests where appropriate (faster, no device needed)
Clarified that handlers always require UI tests
Standardized all test execution to use BuildAndVerify.ps1 script
Platform limitations reported upfront rather than requiring pause

Workflow streamlined from 7 steps to 6 steps (removed pause/wait step):

Checkout PR
Review code
Review tests (unit and UI)
Deep analysis
Validate autonomously using BuildAndVerify.ps1 (reports platform limitations)
Write review

kubaflo · 2025-12-19T00:04:51Z

.github/agents/pr-reviewer.md

-| Test logs | `CustomAgentLogsTmp/UITests/` |
+| Run unit tests | `pwsh .github/scripts/BuildAndVerify.ps1 -RunUnitTests` |
+| Build verification | `pwsh .github/scripts/BuildAndVerify.ps1` |
+| Run UI tests (handlers) | `pwsh .github/scripts/BuildAndRunHostApp.ps1 -Platform [platform] -TestFilter "..."` |


I think it went with handlers too much here

PureWeen · 2025-12-19T01:47:11Z

@copilot cam you create a new testing agent that is just the initial part of the issue resolver agent

The only purpose of this agent is to create the tests that reproduces the issue

Please figure out what minimal tools it needs

And make sure to emphasize that it shouldn't be used to fix the issue at all, just create tests

…on tests only Co-authored-by: PureWeen <[email protected]>

Copilot · 2025-12-19T01:56:14Z

@copilot cam you create a new testing agent that is just the initial part of the issue resolver agent

The only purpose of this agent is to create the tests that reproduces the issue ...

Created test-repro-agent in commit b4e827c.

Key design decisions:

Minimal scope: Only creates reproduction tests, explicitly prevented from fixing issues
5-step workflow: Fetch → Assess → Create → Validate → Report (no fix/PR steps)
Prioritizes unit tests: Emphasizes unit tests (faster) over UI tests when appropriate
Validation-focused: Ensures test FAILS (reproducing bug) before stopping
Minimal tools: Only needs GitHub issue reading, BuildAndVerify.ps1 for unit tests, BuildAndRunHostApp.ps1 for UI tests

The agent emphasizes throughout that it does NOT implement fixes - its only job is to create a test that proves the bug exists.

Initial plan

48b438c

Copilot AI assigned Copilot and PureWeen Dec 13, 2025

Copilot started work on behalf of PureWeen December 13, 2025 20:04 View session

Copilot AI and others added 3 commits December 13, 2025 20:12

Initial plan for duplicating PR #33150

b73cdc0

Co-authored-by: PureWeen <[email protected]>

Duplicate PR #33150 - Fix CollectionView group header size with ItemS…

5b55353

…izingStrategy Co-authored-by: PureWeen <[email protected]>

Apply code formatting to match repository style

02fe8f4

Co-authored-by: PureWeen <[email protected]>

Copilot AI changed the title ~~[WIP] Duplicate PR to work on powershell script separately~~ Duplicate PR #33150: Fix CollectionView group header sizing with ItemSizingStrategy on iOS Dec 13, 2025

Copilot AI requested a review from PureWeen December 13, 2025 20:22

Copilot finished work on behalf of PureWeen December 13, 2025 20:22

Copilot started work on behalf of PureWeen December 13, 2025 20:31 View session

Fix namespace conflict: Rename Animal/AnimalGroup to TestAnimal/TestA…

f41aa92

…nimalGroup Co-authored-by: PureWeen <[email protected]>

Copilot finished work on behalf of PureWeen December 13, 2025 20:46

Copilot started work on behalf of PureWeen December 13, 2025 21:14 View session

Duplicate PR #33150: Fix CollectionView group header sizing with Item…

4ab62b6

…SizingStrategy on iOS Co-authored-by: PureWeen <[email protected]>

Copilot finished work on behalf of PureWeen December 13, 2025 21:28

PureWeen closed this Dec 13, 2025

PureWeen reopened this Dec 13, 2025

PureWeen closed this Dec 13, 2025

PureWeen reopened this Dec 13, 2025

Copilot started work on behalf of PureWeen December 13, 2025 22:21 View session

Add BuildAndVerify.ps1 script for automated build verification

8a13e6a

Co-authored-by: PureWeen <[email protected]>

Copilot AI changed the title ~~Duplicate PR #33150: Fix CollectionView group header sizing with ItemSizingStrategy on iOS~~ Add BuildAndVerify.ps1 script for automated build verification Dec 13, 2025

Copilot finished work on behalf of PureWeen December 13, 2025 22:37

Copilot finished work on behalf of PureWeen December 18, 2025 23:18

PureWeen requested changes Dec 18, 2025

View reviewed changes

Copilot started work on behalf of PureWeen December 18, 2025 23:30 View session

Fix issue-resolver: Handlers need UI tests, always use BuildAndVerify…

83e7630

….ps1 for running tests Co-authored-by: PureWeen <[email protected]>

Copilot AI requested a review from PureWeen December 18, 2025 23:40

Copilot finished work on behalf of PureWeen December 18, 2025 23:40

kubaflo reviewed Dec 18, 2025

View reviewed changes

Copilot started work on behalf of PureWeen December 18, 2025 23:49 View session

kubaflo reviewed Dec 18, 2025

View reviewed changes

Update pr-reviewer: validate tests autonomously using BuildAndVerify.…

2e367f8

…ps1, support both unit and UI tests Co-authored-by: PureWeen <[email protected]>

Copilot AI changed the title ~~BuildAndVerify.ps1 and Issue-Resolver Agent Improvements~~ BuildAndVerify.ps1 and Agent Improvements (Issue-Resolver & PR-Reviewer) Dec 18, 2025

Copilot finished work on behalf of PureWeen December 18, 2025 23:59

Copilot started work on behalf of PureWeen December 19, 2025 00:00 View session

kubaflo reviewed Dec 19, 2025

View reviewed changes

Copilot finished work on behalf of PureWeen December 19, 2025 00:08

Copilot started work on behalf of PureWeen December 19, 2025 01:47 View session

Add test-repro-agent: specialized agent for creating issue reproducti…

b4e827c

…on tests only Co-authored-by: PureWeen <[email protected]>

Copilot AI changed the title ~~BuildAndVerify.ps1 and Agent Improvements (Issue-Resolver & PR-Reviewer)~~ BuildAndVerify.ps1 and Agent Improvements (Issue-Resolver, PR-Reviewer & Test-Repro) Dec 19, 2025

Copilot finished work on behalf of PureWeen December 19, 2025 01:57

PureWeen and others added 2 commits December 18, 2025 20:15

Update uitest-coding-agent.md

270f21f

Update uitest-coding-agent.md

b036cda

BuildAndVerify.ps1 and Agent Improvements (Issue-Resolver, PR-Reviewer & Test-Repro) #33151

Are you sure you want to change the base?

BuildAndVerify.ps1 and Agent Improvements (Issue-Resolver, PR-Reviewer & Test-Repro) #33151

Conversation

Copilot AI commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!