[WIP] Add cloud to integration tests #155
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Pull Request Overview
This WIP PR adds extensive error logging to the ground-to-cloud test file to help diagnose issues during test execution. The changes wrap existing async operations in nested try-except blocks with contextual log messages at different stages of the test workflow.
Key Changes:
- Added nested try-except blocks around session initialization, tool listing, and tool calling operations
- Introduced contextual logging tags ([Initialize], [Tools], [Add Tool], [Client Session], [Streamable HTTP Client]) to identify error sources
- Minor formatting adjustments to multi-line lists
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
testcases/ground-to-cloud/test.py
Outdated
Comment on lines
66
to
72
| except Exception as e: | ||
| print(f"[Add Tool] Error calling add tool: {e}") | ||
| raise e | ||
|
|
||
| if actual_result != expected_result: | ||
| raise AssertionError(f"Expected {expected_result}, got {actual_result}") | ||
|
|
There was a problem hiding this comment.
The variable actual_result is used outside the try-except block (line 70) but is only defined inside it (line 65). If an exception occurs in the try block before actual_result is assigned, the code will fail with a NameError when trying to access this undefined variable in the comparison on line 70.
Suggested change
| except Exception as e: | |
| print(f"[Add Tool] Error calling add tool: {e}") | |
| raise e | |
| if actual_result != expected_result: | |
| raise AssertionError(f"Expected {expected_result}, got {actual_result}") | |
| if actual_result != expected_result: | |
| raise AssertionError(f"Expected {expected_result}, got {actual_result}") | |
| except Exception as e: | |
| print(f"[Add Tool] Error calling add tool: {e}") | |
| raise e |
testcases/ground-to-cloud/test.py
Outdated
Comment on lines
36
to
68
| try: | ||
| await session.initialize() | ||
| except Exception as e: | ||
| print(f"[Initialize] Error initializing session: {e}") | ||
| raise e | ||
|
|
||
| # List available tools | ||
| try: | ||
| tools_result = await session.list_tools() | ||
| available_tools = [tool.name for tool in tools_result.tools] | ||
| expected_available_tools = [ | ||
| "add", "subtract", "multiply", "divide", "power", "square_root", "nth_root", | ||
| "sin", "cos", "tan", "log10", "natural_log", "log_base", "mean", "median", "standard_deviation", | ||
| "complex_add", "complex_multiply", "convert_temperature", "solve_quadratic", "get_constants" | ||
| ] | ||
|
|
||
| print (f"Available tools: {available_tools}") | ||
| except Exception as e: | ||
| print(f"[Tools] Error listing tools: {e}") | ||
| raise e | ||
|
|
||
| if set(available_tools) != set(expected_available_tools): | ||
| raise AssertionError(f"Tool sets don't match. Expected: {set(expected_available_tools)}, Got: {set(available_tools)}") | ||
|
|
||
| # Call the add tool directly | ||
| try: | ||
| call_tool_result = await session.call_tool(name="add", arguments={"a": 7, "b": 5}) | ||
|
|
||
| expected_result = "12.0" | ||
| actual_result = call_tool_result.content[0].text if call_tool_result.content else None | ||
| except Exception as e: | ||
| print(f"[Add Tool] Error calling add tool: {e}") | ||
| raise e |
There was a problem hiding this comment.
These catch-and-re-raise blocks (lines 36-40, 53-55, 66-68) add no value since they simply log and re-raise the same exception without any transformation or recovery logic. The exceptions will propagate naturally and can be logged at a higher level. This pattern creates unnecessary code duplication and increases the indentation depth without providing additional functionality.
Suggested change
| try: | |
| await session.initialize() | |
| except Exception as e: | |
| print(f"[Initialize] Error initializing session: {e}") | |
| raise e | |
| # List available tools | |
| try: | |
| tools_result = await session.list_tools() | |
| available_tools = [tool.name for tool in tools_result.tools] | |
| expected_available_tools = [ | |
| "add", "subtract", "multiply", "divide", "power", "square_root", "nth_root", | |
| "sin", "cos", "tan", "log10", "natural_log", "log_base", "mean", "median", "standard_deviation", | |
| "complex_add", "complex_multiply", "convert_temperature", "solve_quadratic", "get_constants" | |
| ] | |
| print (f"Available tools: {available_tools}") | |
| except Exception as e: | |
| print(f"[Tools] Error listing tools: {e}") | |
| raise e | |
| if set(available_tools) != set(expected_available_tools): | |
| raise AssertionError(f"Tool sets don't match. Expected: {set(expected_available_tools)}, Got: {set(available_tools)}") | |
| # Call the add tool directly | |
| try: | |
| call_tool_result = await session.call_tool(name="add", arguments={"a": 7, "b": 5}) | |
| expected_result = "12.0" | |
| actual_result = call_tool_result.content[0].text if call_tool_result.content else None | |
| except Exception as e: | |
| print(f"[Add Tool] Error calling add tool: {e}") | |
| raise e | |
| await session.initialize() | |
| # List available tools | |
| tools_result = await session.list_tools() | |
| available_tools = [tool.name for tool in tools_result.tools] | |
| expected_available_tools = [ | |
| "add", "subtract", "multiply", "divide", "power", "square_root", "nth_root", | |
| "sin", "cos", "tan", "log10", "natural_log", "log_base", "mean", "median", "standard_deviation", | |
| "complex_add", "complex_multiply", "convert_temperature", "solve_quadratic", "get_constants" | |
| ] | |
| print (f"Available tools: {available_tools}") | |
| if set(available_tools) != set(expected_available_tools): | |
| raise AssertionError(f"Tool sets don't match. Expected: {set(expected_available_tools)}, Got: {set(available_tools)}") | |
| # Call the add tool directly | |
| call_tool_result = await session.call_tool(name="add", arguments={"a": 7, "b": 5}) | |
| expected_result = "12.0" | |
| actual_result = call_tool_result.content[0].text if call_tool_result.content else None |
testcases/ground-to-cloud/test.py
Outdated
| "complex_add", "complex_multiply", "convert_temperature", "solve_quadratic", "get_constants" | ||
| ] | ||
|
|
||
| print (f"Available tools: {available_tools}") |
There was a problem hiding this comment.
There's an extra space between print and the opening parenthesis. This is inconsistent with Python style conventions (PEP 8) which recommend no space between a function name and its opening parenthesis.
Suggested change
| print (f"Available tools: {available_tools}") | |
| print(f"Available tools: {available_tools}") |
testcases/ground-to-cloud/test.py
Outdated
Comment on lines
31
to
82
| try: | ||
| async with streamablehttp_client(mcp_url, headers={ 'Authorization': f'Bearer {token}' }) as (read_stream, write_stream, _): | ||
| try: | ||
| async with ClientSession(read_stream, write_stream) as session: | ||
| # Initialize the session | ||
| try: | ||
| await session.initialize() | ||
| except Exception as e: | ||
| print(f"[Initialize] Error initializing session: {e}") | ||
| raise e | ||
|
|
||
| # List available tools | ||
| try: | ||
| tools_result = await session.list_tools() | ||
| available_tools = [tool.name for tool in tools_result.tools] | ||
| expected_available_tools = [ | ||
| "add", "subtract", "multiply", "divide", "power", "square_root", "nth_root", | ||
| "sin", "cos", "tan", "log10", "natural_log", "log_base", "mean", "median", "standard_deviation", | ||
| "complex_add", "complex_multiply", "convert_temperature", "solve_quadratic", "get_constants" | ||
| ] | ||
|
|
||
| print (f"Available tools: {available_tools}") | ||
| except Exception as e: | ||
| print(f"[Tools] Error listing tools: {e}") | ||
| raise e | ||
|
|
||
| if set(available_tools) != set(expected_available_tools): | ||
| raise AssertionError(f"Tool sets don't match. Expected: {set(expected_available_tools)}, Got: {set(available_tools)}") | ||
|
|
||
| # Call the add tool directly | ||
| try: | ||
| call_tool_result = await session.call_tool(name="add", arguments={"a": 7, "b": 5}) | ||
|
|
||
| expected_result = "12.0" | ||
| actual_result = call_tool_result.content[0].text if call_tool_result.content else None | ||
| except Exception as e: | ||
| print(f"[Add Tool] Error calling add tool: {e}") | ||
| raise e | ||
|
|
||
| if actual_result != expected_result: | ||
| raise AssertionError(f"Expected {expected_result}, got {actual_result}") | ||
|
|
||
| print("Test completed successfully") | ||
| except Exception as e: | ||
| print(f"[Client Session] Error during MCP session: {e}") | ||
| raise e | ||
| except Exception as e: | ||
| print(f"[Streamable HTTP Client] Error connecting to MCP server: {e}") | ||
| raise e | ||
| except Exception as e: | ||
| print(f"Unexpected error connecting to MCP server: {e}") | ||
| raise AssertionError(f"Connection error, {e}") from e |
There was a problem hiding this comment.
These nested try-except blocks create unnecessary complexity and redundancy. Each block catches exceptions, logs them with a contextual tag, and re-raises them. However, this creates multiple layers of exception handling where:
- Inner exceptions are caught, logged, and re-raised
- Outer exceptions catch the same exception, log it again with a different tag, and re-raise
- The outermost handler (line 80-82) catches everything and logs it a third time
This results in duplicate error logging and makes the code harder to maintain. Consider using a single try-except block around the main operations with appropriate logging at that level, or use a logging decorator if you need contextual information at different stages.
Suggested change
| try: | |
| async with streamablehttp_client(mcp_url, headers={ 'Authorization': f'Bearer {token}' }) as (read_stream, write_stream, _): | |
| try: | |
| async with ClientSession(read_stream, write_stream) as session: | |
| # Initialize the session | |
| try: | |
| await session.initialize() | |
| except Exception as e: | |
| print(f"[Initialize] Error initializing session: {e}") | |
| raise e | |
| # List available tools | |
| try: | |
| tools_result = await session.list_tools() | |
| available_tools = [tool.name for tool in tools_result.tools] | |
| expected_available_tools = [ | |
| "add", "subtract", "multiply", "divide", "power", "square_root", "nth_root", | |
| "sin", "cos", "tan", "log10", "natural_log", "log_base", "mean", "median", "standard_deviation", | |
| "complex_add", "complex_multiply", "convert_temperature", "solve_quadratic", "get_constants" | |
| ] | |
| print (f"Available tools: {available_tools}") | |
| except Exception as e: | |
| print(f"[Tools] Error listing tools: {e}") | |
| raise e | |
| if set(available_tools) != set(expected_available_tools): | |
| raise AssertionError(f"Tool sets don't match. Expected: {set(expected_available_tools)}, Got: {set(available_tools)}") | |
| # Call the add tool directly | |
| try: | |
| call_tool_result = await session.call_tool(name="add", arguments={"a": 7, "b": 5}) | |
| expected_result = "12.0" | |
| actual_result = call_tool_result.content[0].text if call_tool_result.content else None | |
| except Exception as e: | |
| print(f"[Add Tool] Error calling add tool: {e}") | |
| raise e | |
| if actual_result != expected_result: | |
| raise AssertionError(f"Expected {expected_result}, got {actual_result}") | |
| print("Test completed successfully") | |
| except Exception as e: | |
| print(f"[Client Session] Error during MCP session: {e}") | |
| raise e | |
| except Exception as e: | |
| print(f"[Streamable HTTP Client] Error connecting to MCP server: {e}") | |
| raise e | |
| except Exception as e: | |
| print(f"Unexpected error connecting to MCP server: {e}") | |
| raise AssertionError(f"Connection error, {e}") from e | |
| async with streamablehttp_client(mcp_url, headers={ 'Authorization': f'Bearer {token}' }) as (read_stream, write_stream, _): | |
| async with ClientSession(read_stream, write_stream) as session: | |
| # Initialize the session | |
| await session.initialize() | |
| # List available tools | |
| tools_result = await session.list_tools() | |
| available_tools = [tool.name for tool in tools_result.tools] | |
| expected_available_tools = [ | |
| "add", "subtract", "multiply", "divide", "power", "square_root", "nth_root", | |
| "sin", "cos", "tan", "log10", "natural_log", "log_base", "mean", "median", "standard_deviation", | |
| "complex_add", "complex_multiply", "convert_temperature", "solve_quadratic", "get_constants" | |
| ] | |
| print (f"Available tools: {available_tools}") | |
| if set(available_tools) != set(expected_available_tools): | |
| raise AssertionError(f"Tool sets don't match. Expected: {set(expected_available_tools)}, Got: {set(available_tools)}") | |
| # Call the add tool directly | |
| call_tool_result = await session.call_tool(name="add", arguments={"a": 7, "b": 5}) | |
| expected_result = "12.0" | |
| actual_result = call_tool_result.content[0].text if call_tool_result.content else None | |
| if actual_result != expected_result: | |
| raise AssertionError(f"Expected {expected_result}, got {actual_result}") | |
| print("Test completed successfully") | |
| except Exception as e: | |
| print(f"Error during MCP server test: {e}") | |
| raise AssertionError(f"Test failed: {e}") from e |
testcases/ground-to-cloud/test.py
Outdated
Comment on lines
53
to
59
| except Exception as e: | ||
| print(f"[Tools] Error listing tools: {e}") | ||
| raise e | ||
|
|
||
| if set(available_tools) != set(expected_available_tools): | ||
| raise AssertionError(f"Tool sets don't match. Expected: {set(expected_available_tools)}, Got: {set(available_tools)}") | ||
|
|
There was a problem hiding this comment.
The variable available_tools is used outside the try-except block (line 57) but is only defined inside it (line 45). If an exception occurs in the try block before available_tools is assigned, the code will fail with a NameError when trying to access this undefined variable in the comparison on line 57.
Suggested change
| except Exception as e: | |
| print(f"[Tools] Error listing tools: {e}") | |
| raise e | |
| if set(available_tools) != set(expected_available_tools): | |
| raise AssertionError(f"Tool sets don't match. Expected: {set(expected_available_tools)}, Got: {set(available_tools)}") | |
| if set(available_tools) != set(expected_available_tools): | |
| raise AssertionError(f"Tool sets don't match. Expected: {set(expected_available_tools)}, Got: {set(available_tools)}") | |
| except Exception as e: | |
| print(f"[Tools] Error listing tools: {e}") | |
| raise e |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.