-
Notifications
You must be signed in to change notification settings - Fork 10.1k
refactor: implement DelegateToAgentTool with discriminated union #14769
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Summary of ChangesHello @abhipatel12, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly refactors the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request is a great refactoring that introduces a DelegateToAgentTool using a Zod discriminated union. This improves type safety and makes the agent delegation mechanism more scalable and maintainable. The changes are well-implemented across the codebase, including updates to configuration, tests, and prompts.
I've found one high-severity issue related to a missing validation check that could cause the application to crash at startup if a sub-agent is defined with a reserved parameter name. I've included a specific comment with a suggested fix for this.
|
Size Change: +2.82 kB (+0.01%) Total Size: 21.6 MB
ℹ️ View Unchanged
|
|
/gemini review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request is a great refactoring of the agent delegation mechanism. Using a Zod discriminated union to dynamically generate the delegate_to_agent tool's schema is a clever and robust approach that improves type safety and provides better guidance to the model. The simplification of the AgentRegistry and the dynamic tool description are also excellent improvements.
I have one suggestion to further improve the robustness of the new DelegateToAgentTool by ensuring the input type mapping is exhaustive, which will help prevent potential bugs in the future.
|
/gemini review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces a significant and well-executed refactoring of the agent delegation mechanism. By replacing individual agent tool wrappers with a single DelegateToAgentTool, and leveraging Zod's discriminated unions to dynamically generate a strongly-typed input schema, the changes greatly improve type safety, maintainability, and the model's ability to correctly invoke sub-agents. The simplification of the AgentRegistry and the dynamic updates to the tool's description and system prompt are also excellent improvements. The new tests are comprehensive and cover the new functionality well. Overall, this is a high-quality contribution that enhances the robustness and scalability of the agent framework. I found no issues of high or critical severity.
4cdf81e to
369f359
Compare
|
|
||
| // Instantiate the Subagent Loop | ||
| const subagentInvocation = new SubagentInvocation( | ||
| agentArgs as AgentInputs, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we do any sort of validation of the type instead of casting?
Is there any potential that we might get an unexpected type for agentArgs?
Alternatively, should SubagentInvocation and this.params be updated to use the same type?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So in this case, we technically don't need the cast at all since in both cases the type would be Record<string, unknown>. It was originally kept to act as documentation, but we can remove it if you think that's best?
And then for validation, the zod discriminated union should have already validated by the time the execute is called, so we should be in good shape for that as well.
WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My initial thought is that if it's not needed to compile we should omit it. It could turn a compile time issue into a failure at runtime if the code is changed in the future.
| * Returns a list of all registered agent names. | ||
| */ | ||
| getAllAgentNames(): string[] { | ||
| return Array.from(this.agents.keys()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I noticed that the codebase seems to have a lot of these sorts of getter methods implemented as actual methods instead of using TS's getter and setter constructs.
get allAgentNames(): string[] {
return Array.from(this.agents.keys());
}Is this an intentional team convention?
https://www.typescriptlang.org/docs/handbook/classes.html#accessors
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah that's a good question. I took a quick look and it does seem that a majority of places uses the getter methods. it seems to just be the repo practice but not sure if it was intentional. If there are benefits, maybe in a separate PR/epic we can consolidate this to the constructs you mentioned!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Most of us in the early days of the project were new to typescript and we relied heavily on the models we were using to build the product. So I would say these kinds of patterns are more organic from that process than intentional.
gundermanc
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
| super( | ||
| DELEGATE_TO_AGENT_TOOL_NAME, | ||
| 'Delegate to Agent', | ||
| `Delegates to a specialized sub-agent (available: ${agentList}). Use for deep analysis tasks like bug investigation or refactoring scope.`, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I imagine we'd want to expose the description of the agents in this description as well? Or nay?
Also what's the motivation for the "Use for deep analysis tasks like bug investigation or refactoring scope."? That seems like it may pigeon hole the delegation tech
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 for description being important.
Also thinking...
- Ideally that description should be agent-author provided and indicate what the agent is for.
- Part of the tool description should indicate which scenarios it should be called in, and its input and output and behavior expectations (e.g.: how do you prevent the outer agent from just "asking" the delegate agent a question instead of asking it to do the actual work?). This could be part of the agent description, a separate schema item that gets folded into the tool description, or just some common string.
- Finally, I wonder if it'd be easier for users to reason about if the behavior is instead to always delegate when an agent that purports to be relevant is present. i.e.: if I add a test debugging agent, I'm going to expect that it's always used to debug tests vs. leaving it up to the model's judgement of the complexity of the task.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great point about the agent descriptions. I missed that and agree that we should have agent descriptions in the tools. Currently, we do have this directly in the system prompt where we use getDirectoryContext(). So we technically already have the descriptions of the subagents in the system prompt, but by adding them in the tool description we can have them in both places. Does that sound alright?
That seems like it may pigeon hole the delegation tech
That's a good point. I don't think we need that. i'll remove it. It's hyperspecific to codebase investigator
Ideally that description should be agent-author provided and indicate what the agent is for.
+1. The agent definition already has this description so we can just use that. We're already doing that in the sys prompt and i've added this to the tool description as well.
Part of the tool description should indicate which scenarios it should be called in, and its input and output and behavior expectations (e.g.: how do you prevent the outer agent from just "asking" the delegate agent a question instead of asking it to do the actual work?). This could be part of the agent description, a separate schema item that gets folded into the tool description, or just some common string.
So we're already using the description and then also have the actual input and output schema being provided to the tool descriptions. Do you think we need an additional field on the AgentDefinition for when to use the agent or can we expect that the description should handle that via natural language?
I wonder if it'd be easier for users to reason about if the behavior is instead to always delegate when an agent that purports to be relevant is present. i.e.: if I add a test debugging agent, I'm going to expect that it's always used to debug tests vs. leaving it up to the model's judgement of the complexity of the task.
Good question. Initially for this v1, i would think it may be best to leave this to the judgment of the model and then allow users to explicitly ask or use context files to push for further usage. And then in the future, we can tweak the system prompt and/or the tool description to make this more explicit. Wdyt?
369f359 to
e17dcc9
Compare
abhipatel12
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the review!
|
|
||
| // Instantiate the Subagent Loop | ||
| const subagentInvocation = new SubagentInvocation( | ||
| agentArgs as AgentInputs, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So in this case, we technically don't need the cast at all since in both cases the type would be Record<string, unknown>. It was originally kept to act as documentation, but we can remove it if you think that's best?
And then for validation, the zod discriminated union should have already validated by the time the execute is called, so we should be in good shape for that as well.
WDYT?
| super( | ||
| DELEGATE_TO_AGENT_TOOL_NAME, | ||
| 'Delegate to Agent', | ||
| `Delegates to a specialized sub-agent (available: ${agentList}). Use for deep analysis tasks like bug investigation or refactoring scope.`, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great point about the agent descriptions. I missed that and agree that we should have agent descriptions in the tools. Currently, we do have this directly in the system prompt where we use getDirectoryContext(). So we technically already have the descriptions of the subagents in the system prompt, but by adding them in the tool description we can have them in both places. Does that sound alright?
That seems like it may pigeon hole the delegation tech
That's a good point. I don't think we need that. i'll remove it. It's hyperspecific to codebase investigator
Ideally that description should be agent-author provided and indicate what the agent is for.
+1. The agent definition already has this description so we can just use that. We're already doing that in the sys prompt and i've added this to the tool description as well.
Part of the tool description should indicate which scenarios it should be called in, and its input and output and behavior expectations (e.g.: how do you prevent the outer agent from just "asking" the delegate agent a question instead of asking it to do the actual work?). This could be part of the agent description, a separate schema item that gets folded into the tool description, or just some common string.
So we're already using the description and then also have the actual input and output schema being provided to the tool descriptions. Do you think we need an additional field on the AgentDefinition for when to use the agent or can we expect that the description should handle that via natural language?
I wonder if it'd be easier for users to reason about if the behavior is instead to always delegate when an agent that purports to be relevant is present. i.e.: if I add a test debugging agent, I'm going to expect that it's always used to debug tests vs. leaving it up to the model's judgement of the complexity of the task.
Good question. Initially for this v1, i would think it may be best to leave this to the judgment of the model and then allow users to explicitly ask or use context files to push for further usage. And then in the future, we can tweak the system prompt and/or the tool description to make this more explicit. Wdyt?
| * Returns a list of all registered agent names. | ||
| */ | ||
| getAllAgentNames(): string[] { | ||
| return Array.from(this.agents.keys()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah that's a good question. I took a quick look and it does seem that a majority of places uses the getter methods. it seems to just be the repo practice but not sure if it was intentional. If there are benefits, maybe in a separate PR/epic we can consolidate this to the constructs you mentioned!
| // Fallback if no agents are registered (mostly for testing/safety) | ||
| schema = z.object({ | ||
| agent_name: z.string().describe('No agents are currently available.'), | ||
| }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm okay with this approach, but I think all of the validation logic can be simplified using the ajv validator.
To do this, should be very straightforward. Essentially, we can add a validateSchema method to our existing schema validator here, which can just call the validateSchema method method in the AJV API. Then after validation we can just use the inputDef directly. I think there are 3 major benefits to doing this:
- This will ensure consistent reporting and semantics of validation for schemas between tools and agents.
- Lets us remove the custom validation logic.
- Gives us full JSON schema type semantics "for free."
However, we can always do this in a follow-up.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I took a look and agreed! Since this will require some broader changes to get working with the AJV validation, i'll create an issue to separate out this task and keep this PR focused.
e17dcc9 to
182bdcf
Compare
allenhutchison
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, but you should update the default policy documents with delegate_to_agent and assign a default permission (likely ASK_USER). While this isn't necessary for the tool confirmation system, it's good practice to document our default policies.
…update tests Refactors the delegation tool to use a Zod discriminated union for better type safety and model guidance. Updates AgentRegistry to provide simplified directory context. Updates Config and Prompts to support the new tool and ResourceRegistry. Fixes all related tests.
Adds validation to DelegateToAgentTool to throw an error if a sub-agent defines an input parameter named 'agentName', which would conflict with the discriminated union discriminator. Adds a regression test.
Replaces if/else chain with a switch statement and exhaustive check to ensure all input types are handled at compile time.
182bdcf to
0460ff5
Compare
Yep 100%. Added this as |
…discriminated union (google-gemini#14769)
Summary
Refactors the
delegate_to_agenttool to use a Zod discriminated union for its input schema. This provides explicit, strongly-typed argument schemas for each subagent directly within the tool definition, improving type safety and guiding the model more effectively. It also simplifies theAgentRegistrydirectory context and updates the tool description to be dynamic.Details
DelegateToAgentTool: Now usesz.discriminatedUnionkeyed byagentName. This allows the model to see the specific arguments required for the chosen agent.AgentRegistry:getDirectoryContext()now returns a simplified list of agents with their descriptions and "when to use" guidance, removing the redundant full schema dump.DelegateToAgentTooltests and fixedconfig.test.tsandprompts.test.tsto align with these changes.ResourceRegistrymock toconfig.test.tsand updatedprompts.test.tsmocks.Related Issues
Closes #14316
How to Validate
npm test --workspace=packages/coreto verify all core tests pass.packages/core/src/agents/delegate-to-agent-tool.tsto see the new schema implementation.Pre-Merge Checklist