Python: fix: avoid empty final response after function limits#6137
Python: fix: avoid empty final response after function limits#6137he-yufeng wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR ensures a user-visible fallback assistant message is returned/emitted when function-invocation limits are reached but the model produces an empty “final” assistant turn.
Changes:
- Add a shared fallback text and helpers to detect/patch missing final assistant content when limits force
tool_choice="none". - Emit the fallback both in non-streaming responses and in streaming mode when the final response is empty.
- Add tests covering max-iterations and max-function-calls scenarios (including streaming) where the final assistant response is empty.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| python/packages/core/agent_framework/_tools.py | Adds fallback message logic and streaming/non-streaming handling when limits are hit and the final assistant content is empty. |
| python/packages/core/tests/core/test_function_invocation_logic.py | Adds regression tests for empty-final-response behavior when limits trigger tool disabling (non-streaming + streaming). |
| original = chat_client_base._get_non_streaming_response # type: ignore[attr-defined] | ||
|
|
||
| async def empty_final_response(*, messages, options, **kwargs): # type: ignore[no-untyped-def] | ||
| if options.get("tool_choice") == "none": | ||
| return ChatResponse(messages=Message(role="assistant", contents=[])) | ||
| return await original(messages=messages, options=options, **kwargs) | ||
|
|
||
| chat_client_base._get_non_streaming_response = empty_final_response # type: ignore[attr-defined,method-assign] | ||
| chat_client_base.run_responses = [ |
| original = chat_client_base._get_streaming_response # type: ignore[attr-defined] | ||
|
|
||
| def empty_final_stream(*, messages, options, **kwargs): # type: ignore[no-untyped-def] | ||
| if options.get("tool_choice") == "none": | ||
|
|
||
| async def stream_empty(): # type: ignore[no-untyped-def] | ||
| for update in (): | ||
| yield update | ||
|
|
||
| return ResponseStream(stream_empty(), finalizer=ChatResponse.from_updates) | ||
| return original(messages=messages, options=options, **kwargs) | ||
|
|
||
| chat_client_base._get_streaming_response = empty_final_stream # type: ignore[attr-defined,method-assign] | ||
| chat_client_base.streaming_responses = [ |
| async def test_max_iterations_empty_final_response_uses_limit_fallback(chat_client_base: SupportsChatGetResponse): | ||
| @tool(name="test_function", approval_mode="never_require") | ||
| def ai_func(arg1: str) -> str: | ||
| return f"Processed {arg1}" |
|
|
||
| async def test_max_function_calls_empty_final_response_uses_limit_fallback(chat_client_base: SupportsChatGetResponse): | ||
| @tool(name="test_function", approval_mode="never_require") | ||
| def ai_func(arg1: str) -> str: | ||
| return f"Processed {arg1}" |
| if approval_result.get("action") == "stop": | ||
| mutable_options["tool_choice"] = "none" | ||
| limit_fallback_requested = True | ||
| return |
74ae684 to
4e01443
Compare
|
|
||
|
|
||
| _FUNCTION_INVOCATION_LIMIT_FALLBACK_TEXT = ( | ||
| "Function invocation stopped because the configured limit was reached before the model returned a final answer." |
There was a problem hiding this comment.
Stepping back from line nits: is fabricating an assistant message the right approach here?
- We synthesize assistant-role content the model never produced. No precedent in core; the only framework strings in
_tools.pyare tool-rolefunction_resultpayloads fed back to the model, never user-facing assistant text. And it's hardcoded English (no i18n) that lands in persisted history -> on replay it reads as if the model said it. - .NET's
FunctionInvokingChatClient(what we mirror) handles this same limit by stripping tools and returning the model's actual final response. No synthetic message, no fallback text, no signal. Pre-PR Python already does the same (strip tools -> final real call -> INFO log). - The limit is a boolean set at ~6 sites across
_get_responseand_stream; one (approval-stop) is already missed -> the condition wants modeling once, not sprinkling. Plus we mutateresponse.messages[-1].contentsin place, against the build-new convention for ChatResponse content.
Could we signal the limit instead of faking content? finish_reason is an open NewType(str) and we already carry control-flow via additional_properties (he internal conversation id) -> surfacing the limit there lets the caller decide on filler, keeps history honest, and matches .NET. If we do want user-facing text, that feels like an agent/caller-layer concern and should at least be marked framework-generated, not a hardcoded literal.
I understand based on the issue this is trying to solve that some clients will produce a final empty output. Perhaps the fix is to have it belong at that layer?
|
Closing this for now. The design concern is fair: synthesizing an assistant message that the model never produced is not a small behavioral fix, and I don't want to keep a questionable fallback PR in the queue. If the maintainers want this solved, I can reopen with a different approach after agreeing on the desired API/UX behavior first. |
Summary
Fixes #5769.
When the function invocation loop hits
max_iterations,max_function_calls, or the error-stop fallback, the framework makes one last model call with tools disabled so the user gets a plain final answer. Some providers can return an empty assistant message for that call. In that case the SDK currently returns a blank final response even though the loop stopped because of a framework limit.This adds a framework fallback only for those forced limit/stop paths. Normal empty model responses outside that fallback path are left alone.
Verification
PYTHONPATH=python/packages/core python -m pytest python/packages/core/tests/core/test_function_invocation_logic.py -q -k "empty_final_response_uses_limit_fallback"PYTHONPATH=python/packages/core python -m pytest python/packages/core/tests/core/test_function_invocation_logic.py -q -k "max_iterations or max_function_calls"PYTHONPATH=python/packages/core python -m pytest python/packages/core/tests/core/test_function_invocation_logic.py -qpython -m py_compile python/packages/core/agent_framework/_tools.py python/packages/core/tests/core/test_function_invocation_logic.pypython -m ruff check python/packages/core/agent_framework/_tools.py python/packages/core/tests/core/test_function_invocation_logic.pygit diff --checkNote: pytest still emits the existing local Windows
.pytest_cachepermission warning in this checkout; tests pass.