Add toolsnaps for every tool #543
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Pull Request Overview
Backfill toolsnaps invocations into all GitHub tool tests and generate corresponding snapshot files to catch unintended schema changes.
- Insert
require.NoError(t, toolsnaps.Test(...))in every tool test. - Add a snapshot file under
pkg/github/__toolsnaps__/for each tool. - Update the toolsnaps library’s error message to include instructions for updating snapshots.
Reviewed Changes
Copilot reviewed 55 out of 55 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| internal/toolsnaps/toolsnaps.go | Augment error message to suggest UPDATE_TOOLSNAPS=true. |
| pkg/github/*_test.go | Invoke toolsnaps.Test in each tool’s test. |
| pkg/github/toolsnaps/*.snap | New snapshot files for every tool schema. |
Comments suppressed due to low confidence (2)
pkg/github/search_test.go:21
- The call to
require.NoErroris unqualified because therequirepackage isn't imported. Addimport "github.com/stretchr/testify/require"to avoid compile errors.
require.NoError(t, toolsnaps.Test(tool.Name, tool))
pkg/github/pullrequests_test.go:932
- Variable
toolis undefined in this scope (onlyhandlerwas assigned). Capture the tool return value before using it, e.g.tool, handler := UpdatePullRequestBranch(...).
require.NoError(t, toolsnaps.Test(tool.Name, tool))
522b2b1 to
40ab680
Compare
SamMorrowDrums
approved these changes
Jun 18, 2025
SamMorrowDrums
approved these changes
Jun 18, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Some time ago I introduced the idea of
toolsnapsto highlight unintended changes to our tool schemas. I also imagined that it might have the benefit of quickly validating conformance if/when the project swaps to the official Go supported SDK. This PR backfills the toolsnaps to all tools.Here's a reminder of what the error messages look like:
Future Thoughts
Like with the desire to produce README's automatically from tool schemas, it would be better if there were a single data structure representing our tools, and then we could iterate that rather than relying on remembering to add the toolsnap call in each test.