Skip to main content
Most of what makes a voice agent right is decided in the skill, and you can test that over chat. What chat cannot show you is how the agent sounds and how fast it answers. This page covers testing those during development and watching them in production.

​
Talk to the agent in the Inspector

The Inspector has a voice mode, so you can test with your own voice before connecting a phone line. Configure it with the same speech providers as your production line, as in Getting Started. See Speech providers.
  1. Set your speech provider’s API key in your environment or .env (DEEPGRAM_API_KEY for Deepgram).
  2. Run rasa inspect.
  3. Leave the message box empty and select Start voice conversation (the waveform button). Allow microphone access when the browser asks.
  4. Select End conversation to hang up.
During the call, the Inspector shows the same skill, memory, and tool events as a chat conversation, alongside voice events:

​
What to listen for

Test each skill out loud at least once. Things that read fine in chat often sound wrong on a call:
  • Long replies. More than a few sentences is hard to follow by ear.
  • Lists. Several items read in a row are hard to remember.
  • Numbers and identifiers. Check that order numbers, card digits, dates, and amounts are read the way a caller expects.
  • Dead air. A long silence before a reply usually means a slow tool or model call. Check the turn’s latency.
  • Interruptions. Talk over the agent mid-sentence and check it stops, and that any tool with side effects did not run twice.
  • The opening and the close. Check what the caller hears first, and that the agent says goodbye once.
Designing for voice covers the fix for each.

​
Test the speech path separately

Simulations help you check skill behavior using text. They do not establish that a caller’s audio was recognized correctly, that a reference sounded right, or that interruptions worked on your phone line. Keep a small set of repeatable spoken cases alongside your text scenarios: Start with the same utterances and expected tool outcomes for each configuration. Record the Rasa version, speech provider, model, voice, language, channel, and test date. Change one setting at a time. Include the names and pronunciations your callers use; a good overall transcription score can hide errors in the few fields that drive a lookup. A provider change also needs a test on the target phone line, since browser and telephony audio paths differ. If a short answer produces no user input, inspect recognition and suppression before changing the skill prompt. The community short-reply investigation shows this diagnostic on specific Rasa versions. It is a versioned example, not a claim about every recognizer or release. For pauses that split speech or delay a turn, start with Deepgram Flux settings when using Flux; other recognizers have their own turn-completion settings.

​
Monitor calls in production

Voice calls emit OpenTelemetry traces and metrics. Configure an exporter in integrations.yml:
integrations.yml

​
Traces

Each call produces one trace that follows the provider connections, each transcription, each agent turn, speech synthesis, and audio playback. Use traces to answer “what happened on this call”: why a turn was slow, why an interruption was accepted or ignored, or where a provider failed. Transcripts and agent text are not captured by default. See Content capture before turning it on. The full span catalog is on Observability.

​
Metrics

Metrics answer “how are calls going overall”. Start with how long callers wait before they hear each reply (time to first audio) and how long each turn takes. Then watch how often callers interrupt, how often they go quiet, which often points to a confusing question, and the failure counts for speech recognition, speech synthesis, and playback.

​
Separate first audio from the answer

An acknowledgement such as “Let me check” gives the caller feedback, but is not the result they asked for. Track the wait to that first sound separately from the wait to the useful answer, such as the order status or confirmation:
  • rasa.voice.tts.time_to_first_audio measures one message’s synthesis from a provider request or cache lookup. It excludes speech recognition, the model, and tool work before that message.
  • rasa.turn.time_to_first_output starts at accepted input. For speech, that is the final transcript, not when the caller stopped speaking. It ends at the first TTS audio chunk, which may be an acknowledgement.
  • To measure the caller’s wait to the answer, instrument your test client from the end of the caller’s audio to the playback of the response that supplies the answer. This is an additional measurement, not a built-in answer-latency metric.
Keep cache hits and misses separate, and retain failures and turns with no audio rather than treating a missing metric as a zero. The community guide How to measure voice agent latency to the answer walks through recorded examples and a test harness. The companion voice driver shows the client-side audio and playback instrumentation used by those examples. Read the example’s setup before running it: live calls use provider credentials and can incur charges. Its recorded timings are not a latency guarantee for your deployment. Every metric, with its dimensions and precise timing boundaries, is on Metrics.

​
Next steps

Designing for voice

Fix what you heard: lists, numbers, confirmations, and endings.

Connect a phone line

Put the agent behind a real phone number.