remove structured LLM output from voice pipeline#6345
Conversation
Backs out the structured-output feature that shipped inside the expressive mode PR (#6116), leaving expressive mode itself intact: - Agent.llm_output_format class attribute and its __init_subclass__ validation, plus the derived _response_field_name - llm.Response / llm.ResponseField annotations (llm/response_field.py) - ChatMessage.llm_output - streaming partial-JSON parse and final strict parse in generation._llm_inference_task, and _extract_spoken_text - BaseModel plumbing through text_ch / tts_node / TTSNode / perform_tts_inference / transcription and the segment channels
There was a problem hiding this comment.
🔍 Breaking public API removal: tts_node signature, ChatMessage.llm_output, and Agent.llm_output_format
This PR removes three public API surfaces that downstream users may depend on:
Agent.llm_output_formatclass attribute and the__init_subclass__validation — any subclass setting this will silently lose structured output support.ChatMessage.llm_outputfield (chat_context.py:293-294in the base) — any code readingmsg.llm_outputwill get anAttributeErrorat runtime.Agent.tts_nodeandAgent.default.tts_nodesignatures changed fromAsyncIterable[str | BaseModel]toAsyncIterable[str]— any user override with the old signature will still work (it's contravariant), but code that was sendingBaseModelinstances into the stream will break.llm.Responseandllm.ResponseFieldare no longer exported fromlivekit.agents.llm.
The removal itself is internally consistent (no dangling references found), but this is a significant breaking change that should be documented in a changelog or migration guide.
Was this helpful? React with 👍 or 👎 to provide feedback.
| @@ -117,8 +116,6 @@ | |||
| "RealtimeSessionRestoredEvent", | |||
There was a problem hiding this comment.
🔍 Pre-existing: RealtimeSessionRestoredEvent listed in all but never imported
RealtimeSessionRestoredEvent appears in __all__ at line 116 but is not imported from any submodule. This means from livekit.agents.llm import RealtimeSessionRestoredEvent will raise an ImportError. This is a pre-existing issue (present in the base commit 77d1d399) and not introduced by this PR, but it's in the same __all__ block that was modified (lines 119-121 were removed).
Was this helpful? React with 👍 or 👎 to provide feedback.
Backs out the structured-output feature that shipped inside the expressive mode PR (#6116), leaving expressive mode itself intact: