> ## Documentation Index > Fetch the complete documentation index at: https://rasa.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Telemetry Event Reference > Telemetry events are only reported if telemetry is enabled. A detailed explanation Telemetry events are only reported if telemetry is enabled. A detailed explanation on the reasoning behind collecting optional telemetry events can be found in our [telemetry documentation](/docs/reference/telemetry.md). ## CALM ### Response Rephrased backend Triggered when a response is rephrased. Event properties: * `rephrase_all` *(boolean)*: True if the rephraser is setup to rephrase all responses. * `custom_prompt_template` *(string)*: A custom prompt template, if specified. * `llm_type` *(string)*: The type of LLM. * `llm_model` *(string)*: The model name of the LLM. ### Intentless Policy Training Started backend Triggered when a user trains the IntentlessPolicy. Event properties: ### Intentless Policy Training Completed backend Triggered when the training of IntentlessPolicy completed. Event properties: * `embeddings_type` *(string)*: The type of embeddings. * `embeddings_model` *(string)*: The model name for embeddings. * `llm_type` *(string)*: The type of LLM. * `llm_model` *(string)*: The model name of the LLM. ### Intentless Policy Predicted backend Triggered when the IntentlessPolicy makes a prediction. Event properties: * `embeddings_type` *(string)*: The type of embeddings. * `embeddings_model` *(string)*: The model name for embeddings. * `llm_type` *(string)*: The type of LLM. * `llm_model` *(string)*: The model name of the LLM. * `score` *(number)*: The prediction score. ### Enterprise Search Policy Training Started backend Triggered when a user trains the EnterpriseSearchPolicy. Event properties: ### Enterprise Search Policy Training Completed backend Triggered when the training of EnterpriseSearchPolicy completed. Event properties: * `vector_store` *(string)*: The vector store. * `embeddings_type` *(string)*: The type of embeddings. * `embeddings_model` *(string)*: The model name for embeddings. * `llm_type` *(string)*: The type of LLM. * `llm_model` *(string)*: The model name of the LLM. * `citation_enabled` *(boolean)*: Whether source\_citation is enabled or not. ### Enterprise Search Policy Predicted backend Triggered when the EnterpriseSearchPolicy makes a prediction. Event properties: * `vector_store` *(string)*: The vector store. * `embeddings_type` *(string)*: The type of embeddings. * `embeddings_model` *(string)*: The model name for embeddings. * `llm_type` *(string)*: The type of LLM. * `llm_model` *(string)*: The model name of the LLM. * `citation_enabled` *(boolean)*: Whether source\_citation is enabled or not. ### PII Management in CALM Enabled backend Triggered when PII management in CALM is enabled. Event properties: * `num_total_rules` *(integer)*: Total number of anonymization rules defined. * `redact_count` *(integer)*: Number of redact rules defined. * `mask_count` *(integer)*: Number of mask rules defined. * `stream_pii` *(boolean)*: Whether streaming PII in un-anonymized events to event brokers is enabled or not. * `tracker_store_anonymization_enabled` *(boolean)*: Whether anonymization of eligible trackers in the tracker store is enabled or not. * `tracker_store_deletion_enabled` *(boolean)*: Whether deletion of eligible trackers in the tracker store is enabled or not. * `anonymization_cron_trigger` *(string)*: Cron trigger for anonymization of eligible tracker sessions in the tracker store. * `deletion_cron_trigger` *(string)*: Cron trigger for deletion of eligible tracker sessions in the tracker store. ## End-to-End Testing ### E2E Test Run Started backend Triggered when end-to-end testing has been started. Event properties: * `number_of_test_cases` *(integer)*: Number of test cases to be run. * `number_of_fixtures` *(integer)*: Number of fixtures defined globally. * `uses_fixtures` *(boolean)*: Indicates if any fixtures have been defined globally. * `uses_metadata` *(boolean)*: Indicates if any metadata has been defined globally. * `number_of_metadata` *(integer)*: Number of metadata defined globally. * `uses_assertions` *(boolean)*: Indicates if any assertions have been defined in test cases. * `flow_started_count` *(integer)*: Number of flow\_started assertion type used in the test run. * `flow_completed_count` *(integer)*: Number of flow\_completed assertion type used in the test run. * `flow_cancelled_count` *(integer)*: Number of flow\_cancelled assertion type used in the test run. * `pattern_clarification_count` *(integer)*: Number of pattern\_clarification assertion type used in the test run. * `action_executed_count` *(integer)*: Number of action\_executed assertion type used in the test run. * `slot_was_set_count` *(integer)*: Number of slot\_was\_set assertion type used in the test run. * `slot_was_not_set_count` *(integer)*: Number of slot\_was\_not\_set assertion type used in the test run. * `bot_uttered_count` *(integer)*: Number of bot\_uttered assertion type used in the test run. * `generative_response_is_relevant_count` *(integer)*: Number of generative\_response\_is\_relevant assertion type used in the test run. * `generative_response_is_grounded_count` *(integer)*: Number of generative\_response\_is\_grounded assertion type used in the test run. ## Model Training ### Training Started backend A training of a Rasa machine learning model got started. The event provides information on aggregated training data statistics. Event properties: * `language` *(string)*: Language model is trained with, e.g. 'en'. * `training_id` *(string)*: Generated unique identifier for this training. * `type` *(string)*: Type of model trained, either 'nlu', 'core' or 'rasa'. * `pipeline` *(undefined)*: List of the pipeline configurations used for training. * `policies` *(undefined)*: List of the policy configurations used for training. * `train_schema` *(undefined)*: Training graph schema for graph recipe * `predict_schema` *(undefined)*: Predict graph schema for graph recipe * `num_intent_examples` *(integer)*: Number of NLU examples. * `num_entity_examples` *(integer)*: Number of entity examples. * `num_actions` *(integer)*: Number of actions defined in the domain. * `num_templates` *(integer)*: Number of templates or responses defined in the domain. * `num_conditional_response_variations` *(integer)*: Number of conditional response variations defined in the domain. * `num_slot_mappings` *(integer)*: Number of total slot mappings defined in the domain. * `num_custom_slot_mappings` *(integer)*: Number of custom slot mappings defined in the domain. * `num_conditional_slot_mappings` *(integer)*: Number of slot mappings with conditions attached defined in the domain. * `num_slots` *(integer)*: Number of slots defined in the domain. * `num_forms` *(integer)*: Number of forms defined in the domain. * `num_intents` *(integer)*: Number of intents defined in the domain. * `num_entities` *(integer)*: Number of entities defined in the domain. * `num_story_steps` *(integer)*: Number of story steps available. * `num_lookup_tables` *(integer)*: Number of different lookup tables. * `num_synonyms` *(integer)*: Total number of entity synonyms defined. * `num_regexes` *(integer)*: Total number of regexes defined. * `is_finetuning` *(boolean)*: True if a model is trained by finetuning an existing model. * `recipe` *(string)*: Recipe used in training the model, either 'default.v1' or 'graph.v1'. * `num_flows` *(integer)*: Number of flows. * `num_flows_with_nlu_trigger` *(integer)*: Number of flows that have an NLU trigger defined. * `num_flows_with_flow_guards` *(integer)*: Number of flows that have a flow guard condition. * `num_flows_with_not_startable_flow_guards` *(integer)*: Number of flows with the flow guard condition 'if: False'. * `num_collect_steps` *(integer)*: Number of collect steps in flows. * `num_collect_steps_with_separate_utter` *(integer)*: Number of collect steps which have a different utterance defined. * `num_collect_steps_with_rejections` *(integer)*: Number of collect steps with rejections included. * `num_collect_steps_with_not_reset_after_flow_ends` *(integer)*: Number of collect steps with 'reset\_after\_flow\_ends' set to 'False'. * `num_set_slot_steps` *(integer)*: Number of set slot steps in flows. * `num_link_steps` *(integer)*: Number of link steps in flows. * `num_call_steps` *(integer)*: Number of call steps in flows. * `max_depth_of_if_construct` *(integer)*: Maximum depth of an if construct. * `num_shared_slots_between_flows` *(integer)*: Number of slots being shared across flows. * `llm_command_generator_model_name` *(string)*: The name of the model used in the 'LLMCommandGenerator'. * `llm_command_generator_custom_prompt_used` *(boolean)*: True, if a custom prompt was configured for the 'LLMCommandGenerator', False otherwise. * `multi_step_llm_command_generator_custom_handle_flows_prompt_used` *(boolean)*: True, if a custom prompt was configured for handling flows in the 'MultiStepLLMCommandGenerator', False otherwise. * `multi_step_llm_command_generator_custom_fill_slots_prompt_used` *(boolean)*: True, if a custom prompt was configured for filling slots in the 'MultiStepLLMCommandGenerator', False otherwise. * `flow_retrieval_enabled` *(boolean)*: True, if flow retrieval is configured for the 'LLMCommandGenerator', False otherwise. * `flow_retrieval` *(string)*: The name of the embedding model used by flow retrieval within 'LLMCommandGenerator'. * `agents` *(object)*: Agent configuration and usage information including MCP servers, agents, and their usage in flows. ### Training Completed backend The training of a Rasa machine learning model finished. The event provides information about the resulting model. Event properties: * `training_id` *(string)*: Generated unique identifier for this training. Can be used to join with 'Training Started'. * `type` *(string)*: Type of model trained, either 'nlu', 'core' or 'rasa'. * `runtime` *(integer)*: The time in seconds it took to train the model. ## Model Testing ### Model Core Tested backend Triggered when a Core model is getting tested. Event properties: * `project` *(string,null)*: Fingerprint of the project the tested model got trained in. * `num_story_steps` *(integer)*: Number of story steps used for testing * `end_to_end` *(boolean)*: Indicates if tests are running in end-to-end mode, testing message handling and dialogue handling at the same time ### Model NLU Tested backend Triggered when an NLU model is getting tested. Event properties: * `num_intent_examples` *(integer)*: Number of NLU examples. * `num_entity_examples` *(integer)*: Number of entity examples. * `num_lookup_tables` *(integer)*: Number of different lookup tables. * `num_synonyms` *(integer)*: Total number of entity synonyms defined. * `num_regexes` *(integer)*: Total number of regexes defined. ## Model Serving ### Interactive Learning Started backend Triggered when an interactive learning session got started. Event properties: * `skip_visualization` *(boolean)*: Whether the visualization of stories should be shown during the interactive learning session * `save_in_e2e` *(boolean)*: Whether the data should be stored in end-to-end format ### Server Started backend Triggered when a Rasa server gets started. Event properties: * `input_channels` *(array)*: Names of the used input channels * `api_enabled` *(boolean)*: Indicator if the API is enabled or if only the input channel is running * `number_of_workers` *(integer)*: Amount of Sanic workers started as part of the server * `endpoints_nlg` *(string,null)*: Type of the used NLG endpoint * `endpoints_nlu` *(string,null)*: Type of the used NLU endpoint * `endpoints_action_server` *(string,null)*: Type of the used action server * `endpoints_model_server` *(string,null)*: Type of the used model server * `endpoints_tracker_store` *(string,null)*: Type of the used tracker store * `endpoints_lock_store` *(string,null)*: Type of the used lock store * `endpoints_event_broker` *(string,null)*: Type of the used event broker * `project` *(string,null)*: Hash of the deployed model the server is started with ### Shell Started backend Triggered when a shell session is started to talk to a trained bot. Event properties: * `type` *(string)*: Type of the model, either 'nlu', 'core' or 'rasa'. ## Markers Extraction ### Markers Extraction Initiated backend Triggered when marker extraction has been initiated. Event properties: * `strategy` *(string)*: Strategy to use when selecting trackers to extract from. * `only_extract` *(boolean)*: Indicates if path to write out statistics hasn't been specified. * `seed` *(boolean)*: The seed to initialise the random number generator for use with the 'sample' strategy. * `count` *(integer,null)*: Number of trackers to extract from (for any strategy except 'all'). ### Markers Extracted backend Triggered when markers have been extracted. Event properties: * `trackers_count` *(integer)*: Number of processed trackers. ### Markers Parsed backend Triggered when markers have been successfully parsed. Event properties: * `marker_count` *(integer)*: Number of parsed markers. * `max_depth` *(integer)*: Maximum depth of the parsed markers. * `branching_factor` *(integer)*: Maximum number of children of any of the parsed markers. ### Markers Statistics Computed backend Triggered when marker statistics have been computed. Event properties: * `trackers_count` *(integer)*: Number of processed trackers. ## Data Handling ### Training Data Split backend Triggered when training data gets split. Event properties: * `fraction` *(number)*: Percentage of the data which goes into training data (the rest goes into the test set). * `type` *(string)*: Type of data, either 'nlu', 'core' or 'rasa'. ### Training Data Validated backend Triggered when training data gets validated. Event properties: * `validation_success` *(boolean)*: whether the validation was successful ### Training Data Converted backend Triggered when training data gets converted. Event properties: * `output_format` *(string)*: target format of the converter * `type` *(string)*: Type of data, either 'nlu', 'core', 'config' or 'nlg'. ### Tracker Exported backend Triggered when conversations get exported from a tracker store through an event broker. Event properties: * `event_broker` *(string)*: Name of the used event broker * `tracker_store` *(string)*: Name of the used tracker store * `number_of_exported_events` *(integer)*: Number of events exported through the event broker ### Story Visualization Started backend Triggered when stories are getting visualized. ## Rasa Pro Services ### Analytics Started backend Triggered when the Analytics pipeline is started. Event properties: * `consumers` *(number)*: The number of Kafka consumers. ### Assistant Session Started backend Triggered when the Analytics pipeline detects a new session start. Event properties: ## Miscellaneous ### Telemetry Disabled backend Triggered when telemetry reporting gets disabled. Last event sent before disabling telemetry. This event is not sent, if the user never enabled telemetry reporting before deactivating it. ### Project Created backend Triggered when a project is created using rasa init. Event properties: * `init_directory` *(string)*: Hash of the directory path the project is created in ## Related topics - [Rasa Telemetry](/docs/reference/telemetry.md) - [Rasa Pro Change Log](/docs/reference/changelogs/rasa-pro-changelog.md) - [Events](/docs/reference/integrations/action-server/sdk-events.md) - [Rasa Pro Services Change Log](/docs/reference/changelogs/rasa-pro-services-changelog.md) This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.