Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Bug]: DeepResearchAgent fails split_question_and_answer_subquestions method when using gemini model #984

@ww2283

Description

@ww2283

Describe the bug

As the title describes: DeepResearchAgent will fail at split_question_and_answer_subquestions method with gemini-2.0-flash, but will be fine with gpt-4o.

Steps to reproduce

Please use the following code to replicate:

import logging
import os
import time
import json
from typing import Optional
from autogen.agents.experimental import DeepResearchAgent  # type: ignore
from autogen.agentchat import UserProxyAgent
import pdb

# Configure logging
logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)

# API configurations
gemini_config = [
    {
        "model": "gemini-2.0-flash",
        "api_key": os.environ.get("GEMINI_API_KEY"),
        "api_type": "google",
    }
]

openai_config = [
    {
        "model": "gpt-4o",
        "api_key": os.environ.get("OPENAI_API_KEY"),
        "api_type": "openai"
    }
]

def create_agent(config_list: list, max_retries: int = 3) -> Optional[DeepResearchAgent]:
    """Create DeepResearchAgent with retry logic"""
    for attempt in range(max_retries):
        try:
            llm_config = {"config_list": config_list}
            agent = DeepResearchAgent(
                name="DeepResearchAgent",
                llm_config=llm_config,
            )
            return agent
        except Exception as e:
            logger.error(f"Attempt {attempt + 1} failed: {str(e)}")
            if attempt < max_retries - 1:
                time.sleep(2 ** attempt)  # Exponential backoff
            else:
                logger.error("Max retries reached. Failed to create agent.")
                raise
    return None

def main():
    try:
        # Create the research agent
        agent = create_agent(gemini_config)
        if not agent:
            return

        # Test message
        message = "What was the impact of DeepSeek on stock prices and why ?"
        
        # Create user proxy agent
        user_proxy = UserProxyAgent(
            name="user_proxy",
            human_input_mode="NEVER",
            code_execution_config={"use_docker": False}
        )

        # Register tools
        for tool in agent.tools:
            try:
                tool.register_for_execution(user_proxy)
            except Exception as e:
                logger.error(f"Failed to register tool: {str(e)}")

        # Initiate chat with error handling
        try:
            chat_result = user_proxy.initiate_chat(
                silent=False,
                recipient=agent,
                message=message,
                max_turns=2,
            )
            logger.info("Chat completed successfully")
            # Print the chat result
            print("\nChat Result:")
            if hasattr(chat_result, '__dict__'):
                print(json.dumps(chat_result.__dict__, indent=2, default=str))
            else:
                print(json.dumps(chat_result, indent=2, default=str))
            return chat_result
        except Exception as e:
            logger.error(f"Chat failed: {str(e)}")
            # Add breakpoint for debugging
            pdb.set_trace()

    except Exception as e:
        logger.error(f"Main execution failed: {str(e)}")
        raise

if __name__ == "__main__":
    main()
    

Model Used

gemini-2.0-flash

Expected Behavior

No response

Screenshots and logs

INFO     [browser_use] BrowserUse logging setup complete with level info
INFO     [root] Anonymized telemetry enabled. See https://docs.browser-use.com/development/telemetry for more information.
user_proxy (to DeepResearchAgent):

What was the impact of DeepSeek on stock prices and why it is so??

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
DeepResearchAgent (to user_proxy):


***** Suggested tool call (1623): delegate_research_task *****
Arguments: 
{"task": "impact of DeepSeek on stock prices and reasons for that"}
**************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION delegate_research_task...
Call ID: 1623
Input arguments: {'task': 'impact of DeepSeek on stock prices and reasons for that'}
CriticAgent (to SummarizerAgent):

Please answer the following question: impact of DeepSeek on stock prices and reasons for that

--------------------------------------------------------------------------------
SummarizerAgent (to CriticAgent):


***** Suggested tool call (4654): split_question_and_answer_subquestions *****
Arguments: 
{"question": "impact of DeepSeek on stock prices and reasons for that"}
******************************************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION split_question_and_answer_subquestions...
Call ID: 4654
Input arguments: {'question': 'impact of DeepSeek on stock prices and reasons for that'}

=== Split Question and Answer Subquestions Output ===
Input Question: impact of DeepSeek on stock prices and reasons for that
DecompositionCritic (to DecompositionAgent):

Analyse and gather subqestions for the following question: impact of DeepSeek on stock prices and reasons for that

--------------------------------------------------------------------------------
DecompositionAgent (to DecompositionCritic):

1.  What is DeepSeek, and what are its primary functions/applications?
2.  What are the possible mechanisms through which DeepSeek could influence stock prices (e.g., news analysis, sentiment analysis, algorithmic trading)?
3.  What data sources would be needed to determine whether or not DeepSeek is correlated with stock prices?


--------------------------------------------------------------------------------
CriticAgent (to SummarizerAgent):

***** Response from calling tool (4654) *****
Error: 'type'
*********************************************

--------------------------------------------------------------------------------
SummarizerAgent (to CriticAgent):

I am sorry, I cannot fulfill this request. The tool returned an error. Please try again.


--------------------------------------------------------------------------------
CriticAgent (to SummarizerAgent):


***** Suggested tool call (2162): confirm_summary *****
Arguments: 
{"answer": "The impact of DeepSeek on stock prices is difficult to assess directly. However, it could indirectly impact companies using DeepSeek (positive or negative based on successful integration), AI-related stocks (potentially positive due to increased confidence in the AI sector), and create speculative trading activity. A definitive assessment requires analyzing specific companies and their performance after adopting DeepSeek's technology, considering broader market trends.", "reasoning": "The answer provides a balanced assessment of potential indirect impacts of DeepSeek on stock prices, considering both positive and negative possibilities. It also acknowledges the speculative nature of some market reactions and highlights the need for specific data to make a definitive conclusion. The answer is coherent, relevant, and provides a reasonably complete overview given the limited information available about DeepSeek's market impact."}
*******************************************************

--------------------------------------------------------------------------------

>>>>>>>> EXECUTING FUNCTION confirm_summary...
Call ID: 2162
Input arguments: {'answer': "The impact of DeepSeek on stock prices is difficult to assess directly. However, it could indirectly impact companies using DeepSeek (positive or negative based on successful integration), AI-related stocks (potentially positive due to increased confidence in the AI sector), and create speculative trading activity. A definitive assessment requires analyzing specific companies and their performance after adopting DeepSeek's technology, considering broader market trends.", 'reasoning': "The answer provides a balanced assessment of potential indirect impacts of DeepSeek on stock prices, considering both positive and negative possibilities. It also acknowledges the speculative nature of some market reactions and highlights the need for specific data to make a definitive conclusion. The answer is coherent, relevant, and provides a reasonably complete overview given the limited information available about DeepSeek's market impact."}
SummarizerAgent (to CriticAgent):

***** Response from calling tool (2162) *****
Answer confirmed:The impact of DeepSeek on stock prices is difficult to assess directly. However, it could indirectly impact companies using DeepSeek (positive or negative based on successful integration), AI-related stocks (potentially positive due to increased confidence in the AI sector), and create speculative trading activity. A definitive assessment requires analyzing specific companies and their performance after adopting DeepSeek's technology, considering broader market trends.
Reasoning: The answer provides a balanced assessment of potential indirect impacts of DeepSeek on stock prices, considering both positive and negative possibilities. It also acknowledges the speculative nature of some market reactions and highlights the need for specific data to make a definitive conclusion. The answer is coherent, relevant, and provides a reasonably complete overview given the limited information available about DeepSeek's market impact.
*********************************************

--------------------------------------------------------------------------------

Critic Agent Chat Result:
{
  "chat_id": null,
  "chat_history": [
    {
      "content": "Please answer the following question: impact of DeepSeek on stock prices and reasons for that",
      "role": "assistant",
      "name": "CriticAgent"
    },
    {
      "content": "",
      "tool_calls": [
        {
          "id": "4654",
          "function": {
            "arguments": "{\"question\": \"impact of DeepSeek on stock prices and reasons for that\"}",
            "name": "split_question_and_answer_subquestions"
          },
          "type": "function"
        }
      ],
      "role": "assistant"
    },
    {
      "content": "Error: 'type'",
      "tool_responses": [
        {
          "tool_call_id": "4654",
          "role": "tool",
          "content": "Error: 'type'"
        }
      ],
      "role": "tool",
      "name": "CriticAgent"
    },
    {
      "content": "I am sorry, I cannot fulfill this request. The tool returned an error. Please try again.\n",
      "role": "user",
      "name": "SummarizerAgent"
    },
    {
      "content": "",
      "tool_calls": [
        {
          "id": "2162",
          "function": {
            "arguments": "{\"answer\": \"The impact of DeepSeek on stock prices is difficult to assess directly. However, it could indirectly impact companies using DeepSeek (positive or negative based on successful integration), AI-related stocks (potentially positive due to increased confidence in the AI sector), and create speculative trading activity. A definitive assessment requires analyzing specific companies and their performance after adopting DeepSeek's technology, considering broader market trends.\", \"reasoning\": \"The answer provides a balanced assessment of potential indirect impacts of DeepSeek on stock prices, considering both positive and negative possibilities. It also acknowledges the speculative nature of some market reactions and highlights the need for specific data to make a definitive conclusion. The answer is coherent, relevant, and provides a reasonably complete overview given the limited information available about DeepSeek's market impact.\"}",
            "name": "confirm_summary"
          },
          "type": "function"
        }
      ],
      "role": "assistant"
    },
    {
      "content": "Answer confirmed:The impact of DeepSeek on stock prices is difficult to assess directly. However, it could indirectly impact companies using DeepSeek (positive or negative based on successful integration), AI-related stocks (potentially positive due to increased confidence in the AI sector), and create speculative trading activity. A definitive assessment requires analyzing specific companies and their performance after adopting DeepSeek's technology, considering broader market trends.\nReasoning: The answer provides a balanced assessment of potential indirect impacts of DeepSeek on stock prices, considering both positive and negative possibilities. It also acknowledges the speculative nature of some market reactions and highlights the need for specific data to make a definitive conclusion. The answer is coherent, relevant, and provides a reasonably complete overview given the limited information available about DeepSeek's market impact.",
      "tool_responses": [
        {
          "tool_call_id": "2162",
          "role": "tool",
          "content": "Answer confirmed:The impact of DeepSeek on stock prices is difficult to assess directly. However, it could indirectly impact companies using DeepSeek (positive or negative based on successful integration), AI-related stocks (potentially positive due to increased confidence in the AI sector), and create speculative trading activity. A definitive assessment requires analyzing specific companies and their performance after adopting DeepSeek's technology, considering broader market trends.\nReasoning: The answer provides a balanced assessment of potential indirect impacts of DeepSeek on stock prices, considering both positive and negative possibilities. It also acknowledges the speculative nature of some market reactions and highlights the need for specific data to make a definitive conclusion. The answer is coherent, relevant, and provides a reasonably complete overview given the limited information available about DeepSeek's market impact."
        }
      ],
      "role": "tool",
      "name": "SummarizerAgent"
    }
  ],
  "summary": "Answer confirmed:The impact of DeepSeek on stock prices is difficult to assess directly. However, it could indirectly impact companies using DeepSeek (positive or negative based on successful integration), AI-related stocks (potentially positive due to increased confidence in the AI sector), and create speculative trading activity. A definitive assessment requires analyzing specific companies and their performance after adopting DeepSeek's technology, considering broader market trends.\nReasoning: The answer provides a balanced assessment of potential indirect impacts of DeepSeek on stock prices, considering both positive and negative possibilities. It also acknowledges the speculative nature of some market reactions and highlights the need for specific data to make a definitive conclusion. The answer is coherent, relevant, and provides a reasonably complete overview given the limited information available about DeepSeek's market impact.",
  "cost": {
    "usage_including_cached_inference": {
      "total_cost": 0,
      "gemini-2.0-flash": {
        "cost": 0,
        "prompt_tokens": 426,
        "completion_tokens": 621,
        "total_tokens": 1047
      }
    },
    "usage_excluding_cached_inference": {
      "total_cost": 0
    }
  },
  "human_input": []
}
user_proxy (to DeepResearchAgent):

***** Response from calling tool (1623) *****
Answer confirmed:The impact of DeepSeek on stock prices is difficult to assess directly. However, it could indirectly impact companies using DeepSeek (positive or negative based on successful integration), AI-related stocks (potentially positive due to increased confidence in the AI sector), and create speculative trading activity. A definitive assessment requires analyzing specific companies and their performance after adopting DeepSeek's technology, considering broader market trends.
Reasoning: The answer provides a balanced assessment of potential indirect impacts of DeepSeek on stock prices, considering both positive and negative possibilities. It also acknowledges the speculative nature of some market reactions and highlights the need for specific data to make a definitive conclusion. The answer is coherent, relevant, and provides a reasonably complete overview given the limited information available about DeepSeek's market impact.
*********************************************

--------------------------------------------------------------------------------

>>>>>>>> USING AUTO REPLY...
DeepResearchAgent (to user_proxy):

The impact of DeepSeek on stock prices is difficult to assess directly. However, it could indirectly impact companies using DeepSeek (positive or negative based on successful integration), AI-related stocks (potentially positive due to increased confidence in the AI sector), and create speculative trading activity. A definitive assessment requires analyzing specific companies and their performance after adopting DeepSeek's technology, considering broader market trends.

Additional Information

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions