> ## Documentation Index
> Fetch the complete documentation index at: https://rasa.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Voice agents

> Build voice agents that hold natural, real-time conversations, with streaming speech, fast turn-taking and interruptions handled for you.

A voice agent in Mantle is the same project as a chat agent, with a voice channel
switched on. Your skills, memory, tools, responses, and control levers do not
change. Mantle detects that a turn arrived on a voice channel and adapts how
the agent speaks, listens, and waits.

That means you build and test one agent, and it serves both channels. Voice
just adds a few things worth tuning.

## How a voice turn works

```mermaid theme={null}
flowchart LR
    A[Caller speaks] --> B[Speech recognition]
    B -->|transcript| C[Mantle turn]
    C -->|text, streamed| D[Speech synthesis]
    D --> E[Caller hears the reply]
```

1. **Speech recognition** turns the caller's audio into text and decides when
   they have finished speaking.
2. **Mantle runs the turn** exactly as it does for chat: it picks the skill,
   runs tools, enforces your constraints, and writes memory.
3. **Speech synthesis** turns the agent's reply into audio. Providers that
   support streaming text input can start speaking before the whole response
   is written. Custom engines must implement the
   [streaming input interface](https://rasa.com/docs/mantle/reference/custom-speech-providers.md#custom-speech-synthesis)
   to do this.

The speech recognition and synthesis providers are configured on the channel in
`integrations.yml`. Recognition and synthesis are chosen separately, so you can mix
providers, for example Deepgram for recognition and Rime for synthesis. Your
skills do not change when you switch provider. See
[Speech providers](https://rasa.com/docs/mantle/reference/speech-providers.md).

## What Mantle handles for you

| On a voice call | What happens | Where to change it |
| - | - | - |
| **Spoken style** | The model is told to speak in short sentences, ask one question at a time, avoid formatting, and say dates, amounts, URLs, and email addresses aloud | [`orchestrator.voice_rules`](https://rasa.com/docs/mantle/reference/agent-yml#orchestrator) |
| **Pauses while tools run** | The model says a short acknowledgement before calling a tool | [`fillers.enabled` and `fillers.examples`](https://rasa.com/docs/mantle/reference/agent-yml#fillers) |
| **Interruptions** | When the caller talks over the agent, playback stops and the turn ends, including any running tool. On by default in the Inspector; turn it on for telephony channels. | [Interruptions](https://rasa.com/docs/mantle/reference/integrations-yml.md#interruptions) |
| **Backchannels** | With interruptions on, short speech that is classified as an acknowledgement does not interrupt the agent | [Interruptions](https://rasa.com/docs/mantle/reference/integrations-yml.md#interruptions) |
| **Silence** | After the silence timeout, the agent checks in. After three check-ins, it says goodbye and hangs up | [Silence timeout](https://rasa.com/docs/mantle/reference/default-skills.md#silence-timeout) |
| **Buttons** | Response buttons are read aloud as a list of their titles | [Responses](https://rasa.com/docs/mantle/concepts/responses.md#static-buttons) |
| **Call start** | The channel sends `/session_start` when the call connects, so a greeting skill can speak first | [Start a conversation](https://rasa.com/docs/mantle/orchestration/runtime-loop.md#start-a-conversation) |

None of this needs changes to your skills. The same skill keeps its text
behavior on chat channels.

## What you control

These are the decisions worth making for a voice agent. Each links to how.

* **Who speaks first.** Replace the silent `default_session_start` with a
  greeting, and add any recording notice there. See
  [Designing for voice](https://rasa.com/docs/mantle/voice/designing-for-voice.md#speak-first-when-the-caller-expects-it).
* **How much the agent says at once.** Lists and long explanations need
  breaking up for a listener. See
  [Read lists in small pieces](https://rasa.com/docs/mantle/voice/designing-for-voice.md#read-lists-in-small-pieces).
* **Exact wording for numbers.** Order numbers and references belong in
  [responses](https://rasa.com/docs/mantle/concepts/responses.md), not in model-generated text.
* **Confirmation before side effects.** Use
  [`requires_confirmation`](https://rasa.com/docs/mantle/build-guide/tool-constraints#explicit-confirmations)
  so the caller agrees out loud before anything irreversible.
* **Keypad input.** A collect step can accept digits from the phone keypad. See
  [DTMF collect steps](https://rasa.com/docs/mantle/reference/ordered-block-steps.md#dtmf).
* **Ending the call.** An ordered-block `action: hangup` ends the call after an
  authored goodbye. See [Step kinds](https://rasa.com/docs/mantle/reference/ordered-block-steps.md#step-kinds).
* **Voice and recognition.** Choose the speech providers, models, and voice per
  language. See [Speech providers](https://rasa.com/docs/mantle/reference/speech-providers.md).

## Try it

The Inspector has a voice mode built in. With an ASR and TTS model enabled, run `rasa inspect` and select **Start voice conversation**. See
[Getting Started](https://rasa.com/docs/mantle/getting-started.md#7-talk-to-your-agent).

## Learn from a complete community project

The [Rasa Community voice tutorial](https://rasa.community/library/tutorials/voice-ai-agent/)
builds a travel agent with references, tools, confirmations, and ordered steps.
Use it when you want to see several skills working together after completing
the [card replacement tutorial](https://rasa.com/docs/mantle/voice/tutorial.md).
Check the tutorial's tested Rasa version and compatibility notes before adapting
its examples; the configuration reference in these docs describes this Mantle
version.

> **[Rasa Community voice tutorial](https://rasa.community/library/tutorials/voice-ai-agent/)**
>
> Follow the travel-agent walkthrough on rasa.community.

> **[Runnable community agent](https://github.com/RasaHQ/rasa-community-resources/tree/4aa0c4419dc193fef7a969c12d59edcf720f2606/tutorials/rasa-voice-agent-tutorial)**
>
> Open the companion project's setup instructions, skills, and dependency pins.

## Next steps

> **[Build a voice agent](https://rasa.com/docs/mantle/voice/tutorial.md)**
>
> A step-by-step tutorial, from first draft to phone-ready.

> **[Designing for voice](https://rasa.com/docs/mantle/voice/designing-for-voice.md)**
>
> Patterns for skills and responses that work well out loud.

> **[Connect a phone line](https://rasa.com/docs/mantle/voice/telephony.md)**
>
> Put the agent behind a real phone number.

> **[Test and monitor voice](https://rasa.com/docs/mantle/voice/testing.md)**
>
> Listen in the Inspector, then watch latency and interruptions in production.


## Related topics

- [Build a voice agent](https://rasa.com/docs/mantle/voice/tutorial.md)
- [Designing for voice](https://rasa.com/docs/mantle/voice/designing-for-voice.md)
- [Test and monitor voice](https://rasa.com/docs/mantle/voice/testing.md)
- [Introduction](https://rasa.com/docs/index.md)
- [Changelog](https://rasa.com/docs/mantle/changelog.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.