Problem:

An error occurs when using langfuse-python with OpenRouter as the base_url and deepseek/deepseek-r1-0528 as the model in a function calling scenario.

Error:

Root Cause:

1. At Time A, the first chunk received from the streaming API contains tool_calls.function.arguments with a value of None.
   Chunk: {'id': 'gen-1757583928-Bk8WIfqQDQprOXBvlnP4', 'choices': [Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=0, id='chatcmpl-tool-005b7a71760046f3aa3205fc354342f9', function=ChoiceDeltaToolCallFunction(arguments=None, name='builtin_search_web'), type='function')]), finish_reason=None, index=0, logprobs=None, native_finish_reason=None)], 'created': 1757583931, 'model': 'deepseek/deepseek-r1-0528', 'object': 'chat.completion.chunk', 'service_tier': None, 'system_fingerprint': '', 'usage': None}

2. The _extract_streamed_openai_response function processes this chunk, and the curr variable is set to:
   curr: [{'name': 'builtin_search_web', 'arguments': None}]

3. At Time B, a subsequent chunk is received.
   Chunk: {'id': 'gen-1757583928-Bk8WIfqQDQprOXBvlnP4', 'choices': [Choice(delta=ChoiceDelta(content=None, function_call=None, refusal=None, role='assistant', tool_calls=[ChoiceDeltaToolCall(index=0, id=None, function=ChoiceDeltaToolCallFunction(arguments='{"', name=None), type='function')]), finish_reason=None, index=0, logprobs=None, native_finish_reason=None)], 'created': 1757583931, 'model': 'deepseek/deepseek-r1-0528', 'object': 'chat.completion.chunk', 'service_tier': None, 'system_fingerprint': '', 'usage': None}

4. The _extract_streamed_openai_response function attempts to append the new arguments chunk to curr[-1]["arguments"]. Since curr[-1]["arguments"] is None, a type error occurs when trying to concatenate a string ('{"') to None.

Fix:
Before performing the string concatenation, check if the current value of curr[-1]["arguments"] is None. If it is, initialize it as an empty string ("") to prevent the error.

Disclaimer: Experimental PR review

Greptile Summary

Updated On: 2025-09-16 03:03:22 UTC

This PR fixes a bug in the OpenAI integration's streaming tool calls functionality. The issue occurs when using OpenAI-compatible providers like OpenRouter with certain models (specifically deepseek/deepseek-r1-0528) that return None for tool call arguments in initial streaming chunks, rather than empty strings.

The fix is implemented in langfuse/openai.py within the _extract_streamed_openai_response function at lines 645-650. The solution adds a null check before attempting string concatenation:

if curr[-1]["arguments"] is None:
    curr[-1]["arguments"] = ""

curr[-1]["arguments"] += getattr(
    tool_call_chunk, "arguments", None
)

This change ensures compatibility across different OpenAI-compatible providers by normalizing None values to empty strings before concatenation. The streaming tool calls functionality processes responses incrementally, building up the complete arguments string from multiple chunks. When the first chunk contains None instead of an empty string, the existing code would fail with a TypeError when trying to concatenate subsequent string chunks to None.

The fix maintains the same final result while preventing runtime errors through defensive programming. It specifically addresses the edge case where different providers handle initial streaming responses differently, without changing the core functionality of tool call argument processing.

Confidence score: 4/5

• This PR is safe to merge with low risk as it's a targeted bug fix for a specific edge case
• Score reflects simple defensive programming change that handles provider differences without affecting core logic
• The fix only touches the OpenAI streaming functionality and includes proper null checking