Open
Description
Describe the bug
During a Twilio call, When a handoff is triggered from a RealtimeAgent
to another agent using handoffs: [billingAgent, refundAgent]
, the new agent's voice becomes broken — similar to static noise or a corrupted audio stream. After this, the agent also stops responding entirely.
The issue occurs even though:
- The
agent_handoff
event fires with the correct arguments. - The response from the new agent does appear correctly (clean audio) in the Traces dashboard.
This suggests the issue may be related to an audio format or transport inconsistency between Twilio and the OpenAI SDK after a handoff.
I was able to replicate this issue in both:
- My original project setup
- A minimal setup using the
examples/realtime-twilio
code from this repo, with agent handoff added
Debug information
- Agents SDK version:
v0.0.8
- Node.js version:
v22.13.0
Repro steps
My sample code:
import Fastify from 'fastify';
import dotenv from 'dotenv';
import fastifyFormBody from '@fastify/formbody';
import fastifyWs from '@fastify/websocket';
import { RealtimeAgent, RealtimeSession } from '@openai/agents/realtime';
import { TwilioRealtimeTransportLayer } from '@openai/agents-extensions';
import { RECOMMENDED_PROMPT_PREFIX } from '@openai/agents-core/extensions';
dotenv.config();
const { OPENAI_API_KEY } = process.env;
const fastify = Fastify();
fastify.register(fastifyFormBody);
fastify.register(fastifyWs);
const billingAgent = new RealtimeAgent({ name: 'Billing agent' });
const refundAgent = new RealtimeAgent({
name: 'Refund agent',
instructions: `${RECOMMENDED_PROMPT_PREFIX}\nYou are Sami a helpful assistant that helps users with refunds.`,
});
const agent = new RealtimeAgent({
name: 'Triage Agent',
instructions: 'You are a helpful and friendly triage assistant. Begin each conversation with a creative and engaging greeting. Based on the user\'s request, determine whether their issue is related to billing or refunds. If it is, use the appropriate tool to hand off the conversation to the Billing Agent or the Refund Agent so they can assist further.',
handoffs: [billingAgent, refundAgent]
});
fastify.all('/twiml', async (request, reply) => {
const twiml = `
<Response>
<Say>O.K. you can start talking!</Say>
<Connect>
<Stream url="wss://${request.headers.host}/media-stream" />
</Connect>
</Response>`.trim();
reply.type('text/xml').send(twiml);
});
fastify.register(async (fastify) => {
fastify.get('/media-stream', { websocket: true }, async (connection) => {
const twilioTransportLayer = new TwilioRealtimeTransportLayer({
twilioWebSocket: connection,
});
const session = new RealtimeSession(agent, {
transport: twilioTransportLayer,
});
await session.connect({ apiKey: OPENAI_API_KEY });
session.on('agent_handoff', (_, currentAgent, newAgent) => {
console.log(`Handoff from ${currentAgent.name} to ${newAgent.name}`);
});
});
});
fastify.listen({ port: 5050 });
Expected behavior
- When a handoff occurs (e.g., from Triage Agent to Refund Agent), the new agent should begin speaking clearly.
- The new agent should continue responding as expected.