Latency Test Checklist

Outcome

Response delay is measured per stage and brought to conversational levels, with a clear record of what each change bought.

Main risk

Optimizing a single number hides a stage (like turn detection) that still makes the call feel laggy or cuts callers off.

Prevention

Always measure the full per-stage breakdown, include endpointing/barge-in timing, and test under realistic network conditions.

Fallback

If a stage cannot hit target on the current provider, switch that component (or the provider) and re-run the checklist.

Architecture overview

Latency Test Checklist uses a bounded agent handoff layer for AI Agents. A structured measurement pass that breaks a voice agent's response delay into its parts (STT, LLM, TTS, network, turn detection) and checks each ag... The architecture connects instrument timestamps at, vapi, retell, and agent handoff with an explicit control path.

• Conversation layer: Instrument timestamps at each hop: speech end, STT final, LLM first token, TTS first audio, playback
• Reasoning layer: Run representative calls and record the per-stage breakdown plus end-to-end response time
• Tools layer: Vapi runs the bounded conversation step for Latency Test Checklist while keeping tool use, transcripts, and escalation outcomes explicit.
• Records layer: Retell connects calls, messages, calendar work, or CRM writes while always measure the full per-stage breakdown, include endpointing/barge-in timing, and test under realistic network conditions.
• Escalation layer: Response delay is measured per stage and brought to conversational levels, with a clear record of what each change bought.

Data flow

1. Instrument timestamps at each hop: speech end, STT final, LLM first token, TTS first audio, playback
2. Run representative calls and record the per-stage breakdown plus end-to-end response time
3. Compare each stage against targets (sub-second end-to-end as the bar) and flag the worst offender
4. Tune the bottleneck (model choice, streaming, endpointing, region) and re-measure to confirm

Controls and fallbacks

• Optimizing a single number hides a stage (like turn detection) that still makes the call feel laggy or cuts callers off.
• Always measure the full per-stage breakdown, include endpointing/barge-in timing, and test under realistic network conditions.
• If a stage cannot hit target on the current provider, switch that component (or the provider) and re-run the checklist.

Tools

• Vapi
• Retell
• Deepgram
• ElevenLabs
• OpenAI
• Twilio