Make add/add-wizard and update manifest-aware for aw.yml package installs#34008
Conversation
Co-authored-by: pelikhan <[email protected]>
Co-authored-by: pelikhan <[email protected]>
Co-authored-by: pelikhan <[email protected]>
Co-authored-by: pelikhan <[email protected]>
add/add-wizard and update manifest-aware for aw.yml package installs
There was a problem hiding this comment.
Pull request overview
This PR makes workflow install/update behavior “manifest-aware” for repository packages defined by aw.yml, so workflows installed from a manifest are tracked and updated as a manifest-managed set rather than as individual file sources.
Changes:
- Switches
source:tracking for manifest-installed workflows to point at the manifest address (owner/repo[@ref]orowner/repo/<packagePath>[@ref]), via shared source-string generation and parsing. - Adds validation to reject
aw.ymlmanifests that contain duplicate workflow filenames (even across different folders). - Updates
updateto detect manifest-backed workflows, group them by manifest source, and reconcile each group (update existing, add new, remove no-longer-listed).
Show a summary per file
| File | Description |
|---|---|
| pkg/cli/update_workflows.go | Groups workflows by manifest source and dispatches group updates. |
| pkg/cli/update_merge.go | Normalizes source during local-modification detection to avoid false diffs. |
| pkg/cli/update_manifest.go | Implements manifest-group update orchestration (update/add/remove). |
| pkg/cli/update_manifest_test.go | Adds unit coverage for grouped manifest update behavior. |
| pkg/cli/spec.go | Adds FromRepositoryManifest and emits manifest-scoped source strings. |
| pkg/cli/spec_manifest_source_test.go | Adds tests for manifest source generation/parsing. |
| pkg/cli/add_workflow_resolution.go | Marks resolved package workflows as manifest-derived and preserves package path. |
| pkg/cli/add_package_manifest.go | Validates unique markdown workflow filenames within a manifest. |
| pkg/cli/add_package_manifest_test.go | Tests duplicate-filename manifest rejection. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 9/9 changed files
- Comments generated: 2
| baseContent, err := downloadWorkflowContentFn(ctx, repo, currentPath, currentRef, opts.Verbose) | ||
| if err != nil { | ||
| merge = false | ||
| } else { | ||
| currentContent, err := os.ReadFile(wf.Path) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to read current workflow: %w", err) | ||
| } | ||
| newSourceSpec := sourceSpecWithRef(&SourceSpec{Repo: repo, Path: latestPath}, latestRef) | ||
| mergedContent, conflicts, mergeErr := MergeWorkflowContent(string(baseContent), string(currentContent), string(newContent), sourceSpecCurrent, newSourceSpec, wf.Path, opts.Verbose) | ||
| if mergeErr != nil { |
| if repoSpec, ok, err := parseManifestSourceSpec(wf.SourceSpec); err != nil { | ||
| errMsg := err.Error() | ||
| if repoSpec != nil { | ||
| errMsg = fmt.Sprintf("%s (%s)", errMsg, repositoryPackageIdentifier(repoSpec.RepoSlug, repoSpec.PackagePath)) | ||
| } | ||
| failedUpdates = append(failedUpdates, updateFailure{ | ||
| Name: wf.Name, | ||
| Error: errMsg, |
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅ |
|
🧪 Test Quality Sentinel completed test quality analysis. |
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. |
|
✅ PR Code Quality Reviewer completed the code quality review. |
There was a problem hiding this comment.
Code Review Summary
I've reviewed this PR for manifest-aware workflow management. The implementation is solid overall with good test coverage. Found 4 issues requiring attention before merge:
📋 Issue breakdown (4 total)
Correctness issues (3):
- Empty workflow filenames silently ignored in validation
- Hardcoded "main" branch assumption (should query actual default branch)
- Unreachable error context branch in update_workflows.go (flagged by previous reviewer)
Maintainability issues (1):
4. Double TrimSpace() call on source spec (minor performance waste)
Already flagged by previous reviewer:
- Source normalization issue in MergeWorkflowContent (line 199, update_manifest.go)
- Unreachable error context in parseManifestSourceSpec handling (line 82, update_workflows.go)
✅ What's done well
- Excellent test coverage: New tests cover manifest source generation, parsing, duplicate detection, and add/update/remove operations
- Clear separation of concerns: Manifest-specific logic cleanly separated into update_manifest.go
- Thorough error handling: Most error paths are properly handled with descriptive messages
- Good use of existing patterns: Follows established conventions for validation and error messages
Recommendation
Please address the correctness issues (especially the hardcoded "main" branch assumption and empty filename handling) before merging. The maintainability issue is minor but worth fixing while touching the code.
🔎 Code quality review by PR Code Quality Reviewer · ● 1.3M
Comments that could not be inline-anchored
pkg/cli/add_package_manifest.go:406
Empty workflow filenames silently ignored: When a manifest entry has an empty filename after trimming (e.g., " .md" or "folder/.md"), the validation silently skips it without error, potentially allowing malformed manifest entries to pass through.
<details>
<summary>💡 Suggested fix</summary>
if key == "" {
return fmt.Errorf("invalid Agentic Workflow manifest %q: workflow filename cannot be empty or whitespace-only: %q", manifestPath, installPath)
}This ensures all `.m…
pkg/cli/update_manifest.go:285
Hardcoded default ref assumption: When repoSpec.Version is empty, the code assumes "main" as the default branch, which may not match the repository's actual default branch (could be master, develop, etc.).
<details>
<summary>💡 Suggested fix</summary>
currentRef := repoSpec.Version
if currentRef == "" {
defaultBranch, err := getRepositoryPackageDefaultBranch(repoSpec.RepoSlug, "")
if err != nil {
for _, wf := range grouped {
failures = append(fail…
</details>
<details><summary>pkg/cli/update_manifest.go:356</summary>
**Potential panic from empty grouped slice**: When adding new workflows from the manifest, `targetDir` is derived from `grouped[0].Path`, but this assumes `grouped` is non-empty. If all workflows in the manifest are new (no existing workflows), this will panic with an index out of range error.
<details>
<summary>💡 Suggested fix</summary>
```go
// Determine target directory from existing workflows or use a sensible default
targetDir := "."
if len(grouped) > 0 {
targetDir = filepath.Dir(gr…
</details>
<details><summary>pkg/cli/update_workflows.go:86</summary>
**Double TrimSpace potentially wasteful**: The source spec is trimmed twice — once in `parseManifestSourceSpec()` (line 232 of update_manifest.go) and again when used as the map key here. The second trim is redundant since `parseManifestSourceSpec` already trims the input.
<details>
<summary>💡 Suggested fix</summary>
```go
manifestGroups[wf.SourceSpec] = append(manifestGroups[wf.SourceSpec], wf)Since the source is already trimmed during parsing, the map key doesn't need additional trim…
🏗️ Design Decision Gate — ADR RequiredThis PR makes significant changes to core business logic (+583 lines under 📄 Draft ADR committed:
📋 What to do next
Once an ADR is linked in the PR body, this gate will re-run and verify the implementation matches the decision. ❓ Why ADRs Matter
ADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors (and your future self) will thank you. This PR introduces a dual interpretation of the 📋 Michael Nygard ADR Format ReferenceAn ADR must contain these four sections to be considered complete:
All ADRs are stored in
|
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /tdd and /zoom-out — requesting changes on test coverage gaps and error handling.
📋 Key Themes & Recommendations
Test Coverage Gaps (/tdd)
- Missing edge cases: Empty manifest handling, case-insensitive duplicate detection, partial package resolution failures
- Incomplete assertions: Lock file cleanup not verified in removal tests
- Error recovery paths: Network failures and partial manifest resolution need coverage
Architecture & Design (/zoom-out)
- ✅ Strong architectural fit: Manifest-scoped source tracking is the right abstraction level — it aligns with how users think about package updates
- ✅ Clean separation: Manifest vs. direct workflow routing is well-designed and maintainable
⚠️ Silent error suppression: Source normalization errors are ignored, which could hide frontmatter issues during local modification detection
Positive Highlights
- ✅ Comprehensive happy-path tests:
TestUpdateManifestWorkflowGroup_AddsUpdatesRemovescovers the core update/add/remove flow - ✅ Early validation: Duplicate filename detection happens before any downloads
- ✅ Backward compatibility: Existing direct workflow specs continue working unchanged
- ✅ Isolated logic: New manifest update behavior is cleanly separated in
update_manifest.go
Priority fixes:
- Add error handling/logging for source normalization failures (comment on
update_merge.go:75) - Add test coverage for edge cases: empty manifests, case-insensitive duplicates, partial failures
- Strengthen existing test assertions to verify lock file cleanup
The core design is solid — these are refinements to make the implementation more robust. 🚀
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · ● 1.1M
|
|
||
| // Normalize again after processing | ||
| sourceResolvedNormalized := stringutil.NormalizeWhitespace(sourceResolved) | ||
| if normalized, normalizeErr := UpdateFieldInFrontmatter(sourceResolvedNormalized, "source", "__gh_aw_source__"); normalizeErr == nil { |
There was a problem hiding this comment.
[/zoom-out] Silent error suppression during source normalization could hide frontmatter issues.
💡 Impact & Suggested Fix
If UpdateFieldInFrontmatter fails (e.g., malformed YAML), the comparison proceeds with unnormalized source values, potentially reporting false positives for local modifications.
Suggested approach (option 1 - log and continue):
if normalized, normalizeErr := UpdateFieldInFrontmatter(sourceResolvedNormalized, "source", "__gh_aw_source__"); normalizeErr == nil {
sourceResolvedNormalized = normalized
} else {
updateLog.Printf("Warning: failed to normalize source field: %v", normalizeErr)
}Option 2 - fail fast if normalization is critical:
normalized, err := UpdateFieldInFrontmatter(sourceResolvedNormalized, "source", "__gh_aw_source__")
if err != nil {
return false, fmt.Errorf("failed to normalize source field: %w", err)
}
sourceResolvedNormalized = normalizedThis prevents silent logic errors when frontmatter is malformed.
| return nil, nil | ||
| } | ||
|
|
||
| _, err := resolveRepositoryPackage(&RepoSpec{RepoSlug: "owner/repo"}, "") |
There was a problem hiding this comment.
[/tdd] Missing test case: case-insensitive duplicate detection.
💡 Suggested edge case test
The duplicate detection in validateUniqueManifestWorkflowFilenames uses strings.ToLower() for case-insensitive comparison, but this test only checks exact duplicates (triage.md in different folders).
Add a test for case variants:
t.Run("rejects case-insensitive duplicate filenames", func(t *testing.T) {
downloadPackageFileFromGitHubForHost = func(owner, repo, path, ref, host string) ([]byte, error) {
switch path {
case "aw.yml":
return []byte(`name: Case Duplicates
files:
- workflows/triage.md
- workflows/Triage.md
`), nil
case "README.md":
return []byte("# Case Duplicates\n"), nil
default:
return nil, createRepositoryPackageNotFoundError(path)
}
}
_, err := resolveRepositoryPackage(&RepoSpec{RepoSlug: "owner/repo"}, "")
require.Error(t, err)
assert.Contains(t, err.Error(), "duplicate workflow filename")
})This ensures the case-insensitive logic works correctly and prevents platform-specific filename collisions.
| if len(successes) != 3 { | ||
| t.Fatalf("expected 3 successful operations, got %d", len(successes)) | ||
| } | ||
|
|
There was a problem hiding this comment.
[/tdd] Test doesn't verify .lock.yml file cleanup.
💡 Strengthen test assertion
The test verifies that removed.md is deleted, but removeManifestManagedWorkflow also removes the corresponding .lock.yml file. The test should verify both:
if _, err := os.Stat(removedPath); !os.IsNotExist(err) {
t.Fatalf("expected removed workflow to be deleted, got err=%v", err)
}
removedLockPath := filepath.Join(tmpDir, "removed.lock.yml")
if _, err := os.Stat(removedLockPath); !os.IsNotExist(err) {
t.Fatalf("expected removed lock file to be deleted, got err=%v", err)
}Also create the lock file in test setup:
if err := os.WriteFile(removedLockPath, []byte("# lock file\n"), 0o644); err != nil {
t.Fatalf("write removed lock: %v", err)
}This ensures the cleanup logic works end-to-end.
| Version: currentRef, | ||
| }, "") | ||
| if err != nil { | ||
| for _, wf := range grouped { |
There was a problem hiding this comment.
[/tdd] Missing test: partial package resolution failure.
💡 Edge case that needs coverage
When resolveRepositoryPackage fails for currentPkg but succeeds for latestPkg (or vice versa), all workflows in the group fail. This error recovery path isn't tested.
Suggested test scenario:
t.Run("handles partial package resolution failure", func(t *testing.T) {
downloadPackageFileFromGitHubForHost = func(owner, repo, path, ref, host string) ([]byte, error) {
if ref == "v1.0.0" && path == "aw.yml" {
return nil, errors.New("network timeout") // Simulate failure
}
if ref == "v2.0.0" && path == "aw.yml" {
return []byte("name: Test\nfiles:\n - workflows/test.md\n"), nil
}
return nil, createRepositoryPackageNotFoundError(path)
}
successes, failures := updateManifestWorkflowGroup(...)
require.Empty(t, successes)
require.NotEmpty(t, failures)
assert.Contains(t, failures[0].Error, "failed to resolve current manifest package")
})This validates the error propagation behavior when one of the two package fetches fails.
| } | ||
| return successes, failures | ||
| } | ||
|
|
There was a problem hiding this comment.
[/tdd] Missing test: empty manifest in latest version.
💡 Boundary condition test
What happens when the latest manifest has zero workflows? The current test doesn't cover this scenario.
Suggested test:
t.Run("removes all workflows when latest manifest is empty", func(t *testing.T) {
downloadPackageFileFromGitHubForHost = func(owner, repo, path, ref, host string) ([]byte, error) {
switch path {
case "aw.yml":
if ref == "v1.0.0" {
return []byte("name: Test\nfiles:\n - workflows/old.md\n"), nil
}
if ref == "v2.0.0" {
return []byte("name: Test\nfiles: []\n"), nil // Empty manifest
}
case "README.md":
return []byte("# Test\n"), nil
}
return nil, createRepositoryPackageNotFoundError(path)
}
tmpDir := testutil.TempDir(t, "empty-manifest-*")
oldPath := filepath.Join(tmpDir, "old.md")
os.WriteFile(oldPath, []byte("---\nsource: owner/[email protected]\n---\n"), 0o644)
successes, failures := updateManifestWorkflowGroup(context.Background(), "owner/[email protected]", []*workflowWithSource{
{Name: "old", Path: oldPath, SourceSpec: "owner/[email protected]"},
}, UpdateWorkflowsOptions{NoCompile: true, DisableSecurityScanner: true})
require.Empty(t, failures)
require.Len(t, successes, 1)
_, err := os.Stat(oldPath)
assert.True(t, os.IsNotExist(err), "workflow should be removed when manifest becomes empty")
})This ensures graceful handling when a package stops declaring workflows.
| return "" | ||
| } | ||
|
|
||
| if workflow.FromRepositoryManifest { |
There was a problem hiding this comment.
[/zoom-out] Good architectural decision: manifest source format.
✅ Why this design works well
Using owner/repo/packages/repo-assist@ref as the manifest source (instead of individual file paths) is the right abstraction level:
- Semantic clarity: The source field now represents the unit of versioning — the manifest package, not individual files
- Update efficiency: All workflows from the same manifest update as a group with a single API call
- Conflict reduction: Source normalization (lines 75-79 in
update_merge.go) prevents false "local modification" detection when migrating from file-level to manifest-level sources
This aligns well with how users think about package updates: "update the repo-assist package" rather than "update 5 individual workflows."
| // Update each workflow | ||
| manifestGroups := make(map[string][]*workflowWithSource) | ||
| var directWorkflows []*workflowWithSource | ||
| for _, wf := range workflows { |
There was a problem hiding this comment.
[/zoom-out] Clean separation: manifest vs. direct workflows.
✅ Good design pattern
The separation logic here is clear and maintainable:
- Parse once, route correctly:
parseManifestSourceSpecdetermines whether a workflow is manifest-backed - Group for efficiency: Manifest workflows get batched by source for atomic updates
- Preserve existing behavior: Direct workflows continue through the original update path
This design avoids a common anti-pattern where manifest and direct workflows would be intermixed in a single update loop with complex conditionals.
Minor suggestion: Consider extracting the grouping logic into a helper function if this pattern repeats elsewhere:
func groupWorkflowsBySource(workflows []*workflowWithSource) (map[string][]*workflowWithSource, []*workflowWithSource, []updateFailure)
🧪 Test Quality Sentinel Report✅ Test Quality Score: 90/100 — Excellent
📊 Metrics & Test Classification (5 tests analyzed)
Test Classification Details
Language SupportTests analyzed:
|
|
@copilot review all comments |
Co-authored-by: pelikhan <[email protected]>
Co-authored-by: pelikhan <[email protected]>
|
@copilot add integration test that adds githubnext/agentic-ops v-1, then call update |
Co-authored-by: pelikhan <[email protected]>
Handled. I reviewed and addressed the actionable review threads in commit |
Co-authored-by: pelikhan <[email protected]>
Added in |
Workflows installed from
aw.ymlmanifests were tracked as per-file sources, soupdatecould not reason at manifest scope. This change moves source tracking to the manifest address (repo/repo/folder+ ref) and updates workflows as a manifest-managed set.Manifest-scoped source tracking
sourcenow points to the manifest location (not individual workflow files).addandadd-wizardpaths through shared source-string generation.Filename-keyed manifest safety
Manifest-aware update orchestration
source.Source-normalized merge/modification behavior
sourceduring local-modification comparison/merge paths so manifest-managed workflows don’t appear modified purely because source format changed.Coverage additions