Speech providers

Every voice channel turns the caller’s audio into text with a speech recognition (ASR) provider, and turns the agent’s replies into audio with a speech synthesis (TTS) provider. You choose both per channel in integrations.yml, under the channel’s asr: and tts: keys. Recognition and synthesis are chosen separately, so you can mix providers. To use a provider that is not built in, see Custom speech providers.

Choose a provider

Set name on asr: and tts:, and choose models and voices per language in language_map:

integrations.yml

channels:
  inspector:
    enabled: true
    asr:
      name: deepgram
      language_map:
        en:
          language: en
          model: flux-general-en
    tts:
      name: cartesia
      language_map:
        en:
          language: en
          voice: f786b574-daa5-4673-aa0c-cbe3e8534c02

Provider	`asr.name`	`tts.name`	API key environment variable
Deepgram	`deepgram`	`deepgram`	`DEEPGRAM_API_KEY`
Deepgram Flux	`deepgram` with a `flux-` model	`deepgram_flux`	`DEEPGRAM_API_KEY`
Azure AI Speech	`azure`	`azure`	`AZURE_SPEECH_API_KEY`
Cartesia	—	`cartesia`	`CARTESIA_API_KEY`
Rime	—	`rime`	`RIME_API_KEY`
ElevenLabs	`elevenlabs`	`elevenlabs`	`ELEVENLABS_API_KEY`

Set the API key in your environment or .env. If a key is missing, Mantle logs the missing variable and the voice conversation can’t start. In the Inspector, you see a “Voice isn’t set up yet” warning.

Defaults

If the Inspector has no asr: or tts: keys, it still works with only DEEPGRAM_API_KEY, using Deepgram’s earlier models:

	Provider	Model
Speech recognition	`deepgram`	`nova-3`, English
Speech synthesis	`deepgram`	`aura-2-thalia-en`

With no asr: or tts: keys, the Inspector uses English Deepgram models, even if agent.language is another language. For another language, set language_map. Telephony channels have no defaults, so you must set both asr: and tts:.

Speech recognition settings

Settings other than name and language_map go next to them, under asr:.

Provider	Settings
`deepgram`	`endpoint` (host name, default `api.deepgram.com`), `endpointing` (ms of silence before a transcript is final, default `400`), `smart_format` (default `true`), `utterance_end_ms` (default `1000`)
`deepgram` with a `flux-` model	`endpoint`, `eot_threshold` (0.5–0.9, default `0.7`), `eot_timeout_ms` (500–10000, default `5000`). See Deepgram Flux for speech recognition.
`azure`	One of `speech_region`, `speech_host`, or `speech_endpoint` (required)
`elevenlabs`	`endpoint` (full WebSocket URL), `commit_strategy` (`vad` or `manual`, default `vad`), `include_timestamps` (default `false`), `include_language_detection` (default `false`)

Speech synthesis settings

Settings other than name and language_map go next to them, under tts:.

Provider	Settings
`deepgram`	`endpoint` (full URL, default `wss://api.deepgram.com/v1/speak`)
`deepgram_flux`	`endpoint` (default `wss://api.deepgram.com/v2/speak`), `speed` (0.5–1.5 in 0.05 steps), `expressivity` (-2 to 2, default `0`). Every `language_map` entry needs a `flux-` model; the default is `flux-alexis-en`.
`azure`	`speech_region` (default `eastus`), `endpoint`, `ws_endpoint`. Default voice `en-US-JennyNeural`.
`cartesia`	`model_id` (default `sonic-3.6`), `version`, `endpoint`
`rime`	`model_id` (default `coda`), `endpoint`, `speed_alpha` (default `1.0`), `segment` (default `immediate`), `no_text_normalization` (default `false`). Default voice `astra`.
`elevenlabs`	`endpoint` (base WebSocket URL, default `wss://api.elevenlabs.io/v1/text-to-speech`), `model_id` (default `eleven_multilingual_v2`), `stability`, `similarity_boost`. Each `language_map` entry must include an ElevenLabs voice ID the API key can access; streaming synthesis includes word-level timestamps.

All speech synthesis engines also accept timeout (seconds, default 30). Synthesized replies are cached in memory, so a repeated line, such as a greeting, plays without a new provider request.

Regional endpoints

To keep audio in a region, point endpoint at the provider’s regional host. For Deepgram, the speech recognition endpoint is a host name and the speech synthesis endpoint is a full URL:

asr:
  name: deepgram
  endpoint: api.eu.deepgram.com
tts:
  name: deepgram_flux
  endpoint: wss://api.eu.deepgram.com/v2/speak

Check your provider’s documentation for the hosts available to your account.

Troubleshoot setup

Symptom	Check first
The Inspector says “Voice isn’t set up yet”	Usually missing ASR or TTS keys: set them in the environment of the process running `rasa inspect`, or in the project’s `.env`, and restart. Any other voice startup error shows the same warning, so check the server log for the cause.
Chat works, but the caller is not heard	Check microphone permission and ASR connection errors, then whether a final transcript reaches the agent. Chat does not test the speech provider.
The agent produces text but no speech	Check the TTS connection, model or voice identifier, and provider access for your account.
A language uses the wrong voice or recognizer	Match `language_map` keys to the agent’s language codes and set the provider’s `language`, `model`, and `voice` values explicitly where required. The Inspector’s implicit defaults are English.

Keep keys out of integrations.yml, shared logs, and source control. When switching providers, preserve the rest of the file, including model_groups. Speech settings belong under the selected channel, not at the file root. Channel settings are read from the live project; restart to pick them up. Changing skills, or which model group agent.yml names, needs rasa train. See integrations.yml for configuration lifecycles. Use Test and monitor voice to distinguish recognition, turn handling, tool, and playback failures before tuning a provider.

Reference

Choose a provider

Defaults

Speech recognition settings

Speech synthesis settings

Regional endpoints

Troubleshoot setup

See also

​Choose a provider

​Defaults

​Speech recognition settings

​Speech synthesis settings

​Regional endpoints

​Troubleshoot setup

​See also

Choose a provider

Defaults

Speech recognition settings

Speech synthesis settings

Regional endpoints

Troubleshoot setup

See also