fix(tts): restart ChunkedStream retries under a fresh request_id#6346
Conversation
A transient mid-synthesis failure could leave frames from the failed attempt and its retry interleaved under mixed request_ids (flaky test_tts_synthesize failures). Now ChunkedStream settles the emitter before retrying so no stale frames are delivered after the retry starts, skips retries for non-retryable errors, and the retry's new request_id signals downstream that partial audio from the failed attempt is stale. Tests accept a request_id restart and validate the final attempt's audio.
There was a problem hiding this comment.
🔍 Metrics accumulate audio duration from all attempts including failed ones
The _metrics_monitor_task at livekit-agents/livekit/agents/tts/tts.py:356-396 iterates over the shared _event_ch tee and accumulates audio_duration from every frame it sees. With this PR, partial audio from a failed attempt now flows through _event_ch before the emitter is closed, so the metrics will report an inflated audio_duration (sum of all attempts) and a ttfb measured from the very first attempt's start time. The request_id in the metrics will be from the last event (the successful attempt), which is correct, but the duration figures will be inaccurate after a retry with partial audio. This is a pre-existing design limitation that becomes more visible now that partial frames are intentionally delivered. The new test test_tts_synthesize_retry_after_partial_audio checks metrics_collected_events.count == 1 but does not assert on the audio_duration value.
(Refers to lines 356-396)
Was this helpful? React with 👍 or 👎 to provide feedback.
Problem
test_tts_synthesize[deepgram]failed on main (run): a transient connection error mid-response causedChunkedStreamto retry after partial audio had already been emitted, so the consumer received frames from two attempts with mixedrequest_ids — and frames from the failed attempt could still trickle out after the retry started, interleaving the two attempts.Changes
ChunkedStream._main_task: settle the emitter (aclose) before retrying, so no stale frames from the failed attempt are delivered once the retry begins. The retry restarts synthesis under a freshrequest_id, signaling downstream that any partial audio from the failed attempt is stale. Also skip retries for non-retryable errors (matchingSynthesizeStream).tests/test_tts.py:_do_synthesisnow accepts a request_id restart — it asserts request_ids don't interleave between attempts and validates completeness/audio on the final attempt's frames. New testtest_tts_synthesize_retry_after_partial_audiocovers fail-once-then-succeed: exactly one retry, recoverable error event, two non-interleaved request_ids, full audio from the retry, one metrics event.tests/fake_tts.py:fake_exception_countoption to fail the first N attempts, and yield control between pushes so partial audio reaches the consumer like a real provider streaming over the network.Notes
SynthesizeStreamstill refuses to retry after partial audio (#5242); making it restart under a fresh request_id like this would be a follow-up.