> ## Documentation Index
> Fetch the complete documentation index at: https://rasa.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# WebSocket Voice Channel

> Stream conversation audio from your own mobile or web app over a plain WebSocket.

<Info>
  **New in 3.19**

  From Rasa Pro 3.19, you can stream conversation audio to your Rasa
  assistant from your own mobile or web app over a plain WebSocket.
</Info>

This channel is a voice-stream connector that accepts audio from any client
that can open a WebSocket and exchange base64-encoded audio, such as an iOS
or Android app, a browser, or a desktop app. Unlike the telephony stream
channels ([Jambonz Stream](/docs/reference/channels/jambonz-stream),
[AudioCodes Voice Stream](/docs/reference/channels/audiocodes-stream)), it is not tied to a
carrier or phone number. Your app captures microphone audio, sends it to
Rasa, and plays back the synthesized response.

## Configure Rasa Assistant

Use the built-in channel `websockets_voice` to configure your Rasa assistant.
Create or edit the `credentials.yml` file at the root of your assistant
directory to add the `websockets_voice` channel. Here's an example:

```yaml title="credentials.yml" theme={null}
websockets_voice:
  server_url: "<your-domain>"
  asr:
    name: deepgram
  tts:
    name: cartesia

  # Optional configurations
  interruptions:
    enabled: true
    min_words: 3
  silence_timeout: 3.0
```

The channel configuration accepts the following properties:

* `server_url` (required): The domain at which the Rasa server is available. Do not include the protocol (`ws://` or `wss://`). For example, if your server is deployed on `https://example.ngrok.app`, `server_url` should be `example.ngrok.app`.

* `asr` (required): Configuration for Automatic Speech Recognition. See [Speech Integrations](/docs/reference/integrations/speech-integrations#automatic-speech-recognition-asr) for the list of ASR engines Rasa integrates with.

* `tts` (required): Configuration for Text-To-Speech. See [Speech Integrations](/docs/reference/integrations/speech-integrations#text-to-speech-tts) for the list of TTS engines Rasa integrates with.

* `interruptions` (optional): Configuration for interruption handling. This lets the assistant detect when a user speaks over it and respond more naturally. See [Interruption Handling](/docs/reference/primitives/patterns#interruption-handling) for more information.

* `silence_timeout` (optional): Number of seconds of silence after which the current user turn is closed. Set this when you want the assistant to react to a pause in speech.

Run the assistant with `rasa run`. Your app needs a URL that can reach the Rasa
server. For development, you can expose your local server with tools like
[ngrok](https://ngrok.com/) or
[Cloudflare Tunnel](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/).

<Info>
  **Bot URLs for development**

  Visit this [section](/docs/reference/channels/messaging-and-voice-channels#testing-channels-on-your-local-machine) to learn how to generate
  the required bot URL when testing the channel on your local machine.
</Info>

## Connecting Your App

Your app connects to the WebSocket endpoint exposed by the channel:

```text theme={null}
wss://<your-domain>/webhooks/websockets_voice/websocket
```

A `GET` request to `/webhooks/websockets_voice/` returns `{"status": "ok"}`,
which you can use as a health check.

### Audio Format

Audio is exchanged in both directions as Linear PCM (L16), 24 kHz, mono,
16-bit. Rasa supports 24 kHz natively, so no server-side transcoding is
required. Your app must capture and play back audio at this sample rate.

## WebSocket Protocol

The client and server exchange JSON text frames. Audio payloads are
base64-encoded PCM data.

### Client to Server

Send captured audio:

```json theme={null}
{ "audio": "<base64-encoded-pcm-data>" }
```

Send a text message instead of audio (the assistant treats it as a user turn):

```json theme={null}
{ "text": "I want to book a flight" }
```

Acknowledge a playback marker (see [Playback markers](#playback-markers)):

```json theme={null}
{ "marker": "<marker-id>" }
```

### Server to Client

Synthesized assistant audio to play back:

```json theme={null}
{ "audio": "<base64-encoded-pcm-data>" }
```

A playback marker to acknowledge once the corresponding audio has finished
playing:

```json theme={null}
{ "marker": "<marker-id>" }
```

A signal to stop playback immediately, sent when the assistant detects an
interruption:

```json theme={null}
{ "interruptPlayback": true }
```

A custom payload from a [custom action](/docs/pro/build/custom-actions)'s
`custom` response. The channel sends it as data rather than speaking it, so
your app can act on it silently (for example, to refresh an expired token):

```json theme={null}
{ "custom": { "your": "payload" } }
```

## Playback Markers

The server tags bot audio with markers so it can track what the client has
actually played. When your app finishes playing a chunk of audio, send the
matching `marker` frame back. Rasa uses these acknowledgements to know when
the assistant has stopped speaking and, for example, when it is safe to end
the call after a goodbye message. An app that ignores markers still plays
audio, but the assistant cannot reliably tell when playback has finished.

## Sending and Receiving Messages

When a user speaks, your app streams the audio frames to Rasa. The configured
ASR engine converts speech to text, and Rasa interprets the message like any
other channel. The assistant responds with text, the configured TTS engine
converts it to speech, and the audio streams back to your app.

```yaml title="domain.yml" theme={null}
utter_greet:
  - text: "Hello! How can I help you today?"
```

<Note>
  Only text and audio are supported. Images, attachments, and buttons cannot be
  used with voice stream channels. Custom (`custom`) responses are delivered to
  the client as data frames instead of being spoken.
</Note>

## Passing Client Data to Actions

Your app can attach per-user data when it opens the WebSocket, and that data
is carried into every user turn. There are two ways to send it:

* The `Authorization` header on the WebSocket upgrade request, available under the `authorization` key.
* Any query argument on the connection URL, except `language`.

```text theme={null}
wss://<your-domain>/webhooks/websockets_voice/websocket?token=abc123&user_id=456
```

The channel collects these values into the `extra` field of the call metadata.
A [custom action](/docs/pro/build/custom-actions) can read them from the
latest message or from the `session_started_metadata` slot:

```python title="actions.py" theme={null}
def run(self, dispatcher, tracker, domain):
    metadata = tracker.latest_message["metadata"]["extra"]
    token = metadata.get("token")      # "abc123"
    user_id = metadata.get("user_id")  # "456"
    # Use for authentication, personalization, or session context.
```

The `extra` field only forwards these values to your assistant. It does not
authenticate the WebSocket connection, and returning a token here does not
restrict who can connect. Enforce access control in your custom actions or in
front of the Rasa server. Client-supplied values are confined to `extra`, so
they cannot overwrite call parameters such as `call_id`.

The `language` query argument is reserved: it sets the language of the call
rather than being forwarded in `extra`.

```text theme={null}
wss://<your-domain>/webhooks/websockets_voice/websocket?language=en
```

## Call Metadata

Metadata about the call can be accessed through the `session_started_metadata`
slot at the beginning of the conversation. The following fields are available:

| Field Name  | Description                                                               |
| ----------- | ------------------------------------------------------------------------- |
| `call_id`   | A unique identifier generated for the connection, prefixed with `app-`.   |
| `stream_id` | The unique stream identifier (same as `call_id`).                         |
| `language`  | The call language, taken from the `language` query argument if provided.  |
| `extra`     | Client-supplied data from the `Authorization` header and query arguments. |

A [custom `action_session_start`](/docs/reference/primitives/default-actions) can be used to store this information to a slot.


## Related topics

- [CHIRP](/docs/reference/channels/chirp.md)
- [Jambonz Voice Stream Channel](/docs/reference/channels/jambonz-stream.md)
- [Custom Connectors](/docs/reference/channels/custom-connectors.md)
- [Connecting to Messaging and Voice Channels](/docs/reference/channels/messaging-and-voice-channels.md)
- [Genesys Cloud](/docs/reference/channels/genesys-cloud-voice.md)