Customizing LLM-based Components

Rasa Labs

Rasa Labs access - New in 3.7.0b1

Rasa Labs features are experimental. We introduce experimental features to co-create with our customers. To find out more about how to participate in our Labs program visit our Rasa Labs page. We are continuously improving Rasa Labs features based on customer feedback. To benefit from the latest bug fixes and feature improvements, please install the latest pre-release using:

pip install 'rasa-pro>3.8' --pre --upgrade

The LLM components can be extended and modified with custom versions. This allows you to customize the behavior of the LLM components to your needs and experiment with different algorithms.

Customizing a component

The LLM components are implemented as a set of classes that can be extended and modified. The following example shows how to extend the ContextualResponseRephraser component to add a custom behavior. For example, we can change the rephrasing logic to use some more simple rephrasing strategy instead of calling an LLM in specific cases (e.g., when a certain keyword is present in the response). This can be done by extending the ContextualResponseRephraser class and overriding the rephrase method:

from rasa.core import ContextualResponseRephraser

class CustomContextualResponseRephraser(ContextualResponseRephraser):
    async def rephrase(self, response: Dict[str, Any], tracker: DialogueStateTracker) -> Dict[str, Any]:
        original_text = response.get("text", "")
        if "example_keyword" in original_text:
            response["text"] = original_text.replace("hello", "bonjour")
            return response
        else:
            # Fallback to the default rephrase method
            return await super().rephrase(response, tracker)

The custom component can then be used in the Rasa configuration file:

config.yml

pipeline:
  - name: CustomContextualResponseRephraser
    # ...

To reference a component in the Rasa configuration file, you need to use the full name of the component class. The full name of the component class is <module>.<class>. All components are well documented in their source code. The code can be found in your local installation of the Rasa Pro python package.

Common functions to be overridden

Below is a list of functions that could be overwritten to customize the LLM components:

ContextualResponseRephraser

rephrase

Rephrases the response generated by the LLM. The default implementation rephrases the response by prompting an LLM to generate a response based on the incoming message and the generated response. The generated response is then replaced with the generated response.

    def rephrase(
        self,
        response: Dict[str, Any],
        tracker: DialogueStateTracker,
    ) -> Dict[str, Any]:
        """Predicts a variation of the response.

        Args:
            response: The response to rephrase.
            tracker: The tracker to use for the prediction.
            model_name: The name of the model to use for the prediction.

        Returns:
            The response with the rephrased text.
        """

IntentlessPolicy

select_response_examples

Samples responses that fit the current conversation. The default implementation samples responses from the domain that fit the current conversation. The selection is based on the conversation history, the history will be embedded and the most similar responses will be selected.

    def select_response_examples(
        self,
        history: str,
        number_of_samples: int,
        max_number_of_tokens: int,
    ) -> List[str]:
        """Samples responses that fit the current conversation.

        Args:
            history: The conversation history.
            policy_model: The policy model.
            number_of_samples: The number of samples to return.
            max_number_of_tokens: Maximum number of tokens for responses.

        Returns:
            The sampled conversation in order of score decrease.
        """

select_few_shot_conversations

Samples conversations from the training data. The default implementation samples conversations from the training data that fit the current conversation. The selection is based on the conversation history, the history will be embedded and the most similar conversations will be selected.

    def select_few_shot_conversations(
        self,
        history: str,
        number_of_samples: int,
        max_number_of_tokens: int,
    ) -> List[str]:
        """Samples conversations from the given conversation samples.

        Excludes conversations without AI replies

        Args:
            history: The conversation history.
            number_of_samples: The number of samples to return.
            max_number_of_tokens: Maximum number of tokens for conversations.

        Returns:
            The sampled conversation ordered by similarity decrease.
        """

Rasa Labs access - New in 3.7.0b1

​Customizing a component

​Common functions to be overridden

​ContextualResponseRephraser

​rephrase

​IntentlessPolicy

​select_response_examples

​select_few_shot_conversations