How-to: RASA chatbot¶

Overview¶

chatbot_rasa is the default chatbot engine used on the robot. It uses the RASA chatbot engine for natural language processing, an open-source chatbot framework based on machine learning. While it may be limited in terms of flexibility and ease of use compared to an LLM-based chatbot, it has the advantage of running fully on the robot since it requires much less resources.

ROS interface¶

chatbot_rasa integrates with communication_hub through the /chatbot_rasa/start_dialogue and /chatbot_rasa/dialogue_interaction interfaces, as described in Chatbot interaction.

Clearly chatbot_rasa is a localized node, meaning its current language selection is controlled by the i18n_manager. Refer to [‼️ROS 1] Internationalisation and language support for more information on language availability and selection.

Note

The loading time of a new model is quite long (up to 1 minute). This happens whenever a new locale is selected for the first time since startup, since models are locale specific. To avoid long loading times, only the model specified by the parameter default_locale is loaded at startup. The other models are only loaded when the associated locale is selected for the first time since startup.

Dialogues support¶

The current implementation has limited support for dialogues:

Only one role.name can be supported at a time, which is the one of the model family selected in the model parameter. The model family is the collection of models stemming from similar training data, differing only in the language used for training.
The role.configuration is always ignored.
Dialogues never return any result, they can only be aborted in case of an error.

Only one model family is currently available by default, called, in a leap of creativity effort, default.

Web interface¶

The Web User Interface provides the status of the chatbot_rasa under Diagnostics > Communication > Chatbot > chatbot_rasa.

There you can check, among other things:

Locales: the locales available for the model family;
State for <locale>: the current state of the locale-specific model of the model family;
Last input: the last input receive by the chatbot;
Last recognised RASA intent: the last rasa internal intent recognized by the chatbot;
Last output: the last response output by the chatbot;

RASA basic concepts¶

There are 4 key concepts to know when talking about chatbot design with RASA:

Intent: What is the user intending to ask about?

Caution

Chatbot’s intents should not be confused with the system-wide robot Intents. The chatbot intents are solely extracted from input text. They are internal to the chatbot, and are not visible outside of it.

The detected ‘system’ intents are included in the /chatbot_rasa/dialogue_interaction response, and are multi/modal (for instance, if a person tries to engage with the robot by getting closer, an ENGAGE_WITH intent will be published on the /intents topic).

Note however that a ‘system’ intent can be triggered from a ‘chatbot’ intent: for instance, if a user tells the robot “I want you go in the kitchen”, the chatbot might recognise a go_to intent in the sentence that will trigger the publication of a MOVE_TO intent on the /intents topic.

Entity and Slots: What are the important pieces of information in the user’s query?
Stories or Rules: What is the possible way the conversation can go?
Action: What action should the robot take upon a specific request?

RASA has two main modules:

RASA NLU for understanding user messages. It detects Intents and Entities in your messages and may have different components to recognize them such as Spacy for Pretrained Embeddings or Tensorflow for Supervised Embeddings.
RASA Core for holding conversations and deciding what to do next. It can predict dialogue as a reply and can trigger actions as well.

Training a new chatbot¶

The typical training data structure for a RASA chatbot is as follows:

my_chatbot/
├── data/
│   ├── nlu.yml
│   ├── rules.yml
│   └── stories.yml
├── config.yml
└── domain.yml

config.yml configuration of your NLU and Core models. In-case you are dealing with Tensorflow or Spacy, you need to define such pipeline here. To handle this file, you show know about Machine Learning and Deep Learning. You can see additional information in the Assistant Configuration section of the RASA documentation.
domain.yml, nlu.yml, rules.yml, stories.yml contain the training data for the NLU and the configuration of the Core model. They contain the list of intents, entities, rules and actions. You can see additional information in the Rasa Primitives section of the RASA documentation.

Note

chatbot_rasa uses two hard-coded slots:

speaker_id: the person_id of the input is filled in this slot; it can be used by the chatbot model in formulating the response
intents: this slot is cleared on every input, and is expected to be filled by the chatbot model when a system intent is detected. You can see for example the action_present_content.

To train the chatbot:

make sure you have installed the RASA backend in a PAL OS docker container,
run the RASA training command in the training data directory (e.g., my_chatbot/).

$ sudo apt install rasa
$ rasa train --fixed-model-name <model_name>

The training process should output a models/<model_name>.tar.gz file, which is the trained model.

Training example¶

To have an idea of what the chatbot model looks like, let’s consider the following example. We will create it for American English (en_US), but the same applies to any other language.

Note

It is recommended that all the keys in the following yaml are kept in English, regardless of the language of the training data, since they are used by the RASA engine and should not be translated.

We have:

domain.yml: the chatbot’s domain. It combines different intents, which are declared in the beginning, that the chatbot can detect, and a list of replies. If you have created custom actions, they should be declared here (e.g., action_get_weather).
```
intents:
- greet

responses:
  utter_greet:
  - text: Hi!
  - text: Hey, I am your robotic companion
  - text: what's up

actions:
  - action_get_weather
```
The action action_get_weather should be a custom action that is available at runtime. This one in particular is already implemented in the chatbot_rasa_default_models_common package (it requires internet connection to work properly). See Custom actions for more information on how to create your own custom actions.

nlu.yml: the NLU training data, where you define intents, as well as related sentences the robot should recognize to match the intent. For instance:

nlu:
- intent: greet
  examples: |
    - hey
    - hello
    - hi
    - hello there
    - good morning
    - good evening
    - hey there
    - let's go
    - hey dude
    - goodmorning
    - goodevening
    - good afternoon

- intent: get_weather
  examples: |
    - what is the weather in [Barcelona](location)
    - weather of [Madrid](location)
    - Today's weather in [Paris](location)
    - [Milan](location) weather
    - Tell me weather of [London](location)
    - Can you tell me weather of [New York](location)
    - Tell me weather of [Tokyo](location)
    - weather of [Sydney](location)

Notice the use of [word](entity) to define an entity. The entity is extracted from the sentence and can be used in the response, in this case in the script of action_get_weather.

rules.yml: optionally, describe short pieces of conversations that should always follow the same path. In this example, we want the robot to greet the user, trigger the utter_greet response defined in the domain, everytime it hears a Hello or greet intent. It is important that the response name must be prefixed with utter_, to implicitly instruct RASA to pick a random sentence of utter_greet as a response.

Likewise, if the user asks for the weather, we want to trigger the action_get_weather action. In this case we want to trigger a custom action, so the action name must be prefixed with action_.
```
rules:
- rule: Greeting Rule
  steps:
  - intent: greet
  - action: utter_greet

- rule: get weather
  steps:
  - intent: get_weather
  - action: action_get_weather
```
stories.yml: optionally, the chatbot may contain stories, that defines the flow of the conversation. The default chatbots of the robot for instance does not contain such a file. See Stories for more information.

Note

A chatbot should always have at least a rules.yml or stories.yml file in order to train.

Installing a trained chatbot¶

In order for the chatbot to be discoverable by chatbot_rasa, the trained model must be installed as a ROS 2 CMake package, together with some supporting files.

The package should be structured as follows (the file names are examples):

my_pkg/
├── actions_locales.yml
├── CMakeLists.txt
├── model_family_name.tar.gz
├── model_manifest.yml
└── package.xml

model_family_name.tar.gz: the trained model
actions_locales.yml: the localization of the responses of custom actions. Those are all in the form of "sentence in code, usually in English": "translated sentence", on a separate line for each sentence. The sentences may be formatted strings, in which case anything between curly brackets {} should be left unchanged.

For example, the localization of the action action_get_weather in Spanish could look like this: .. code-block:: yaml

“It is currently {condition} in {city}.”: “Actualmente esta {condition} en {city}.” “Sorry, I can not connect to the weather server”: “Lo siento, no puedo conectarme al servior del tiempo.”

If you are adding a new language to an existing model family, you can copy the existing localization file of any member of the family and change the translated sentences. Otherwise, you must check all the localized sentences of all the custom actions used (all the strings contained in _("sentence to translate") function calls, as in gettext) and add them to the localization file.

model_manifest.yml: the model manifest file, containing (the values are examples):

name: model_family_name  # the name of the model family is part of
locale: en_US  # the locale of the training data used to train the model
path: model_family_name.tar.gz  # the path of the installed model relative to the package share
chatbots: []
catalogues: actions_locales.yml  # the path of the installed action translations relative to the package share

CMakeLists.txt: the CMake instructs to install the above files in the package share folder and install an ament index marker file for the model manifest into the chatbot.rasa.model resources

install(
  FILES
    model_family_name.tar.gz
    actions_locales.yml
    model_manifest.yml
  DESTINATION
    share/${PROJECT_NAME}
)

ament_index_register_resource("chatbot.rasa.model" CONTENT "model_manifest.yml")

If this package is installed and sourced, the model family model_family_name will be selectable by the chatbot_rasa node using the model parameter. The supported locales for this model family would be only en_US, until additional models are trained using translated training data and installed.

Custom actions¶

An essential part of the chatbot is the actions definition. Indeed, in RASA is possible to define RASA custom actions as responses to specific intents (as listed at the bottom of the chatbot’s domain.yml file).

Note

We do not use RASA’s default action server, but chatbot_rasa itself hosts an action server. The implementation of custom action also differs from the one in the official documentation.

Custom actions are Python functions that you can define to add custom behavior and logic to your chatbot. Custom actions allow you to go beyond simple predefined responses and enable your chatbot to perform actions, like executing some ROS code, connecting to the Web to fetch some information, or interact with databases or external APIs.

We provide a set of custom actions, installed with the package chatbot_rasa_default_models_common. Each action is a short Python script, automatically discovered and loaded by chatbot_rasa (which acts also as action server for the RASA backend).

You can see the list of actions available by default in this folder:

$ tree $(ros2 pkg prefix chatbot_rasa_default_models_common)/share/chatbot_rasa_default_models_common/actions

Creating a new custom action¶

For an action to be usable by chatbot_rasa, it must first be an implementation of the Python abstract class Action of the module chatbot_rasa.types:

import abc
from rclpy.node import Node

class Action(abc.ABC):
    @abc.abstractmethod
    def __init__(self, node: Node):
        raise NotImplementedError

    @staticmethod
    @abc.abstractmethod
    def name() -> str:
        raise NotImplementedError

    @abc.abstractmethod
    def run(self, data: dict) -> dict:
        raise NotImplementedError

In the init() method, the node argument is a handle to the chatbot_rasa node, which can be used to interface with the ROS 2 system, for instance creating a publisher or log some information.
The name() method must return the name of the action as defined in the domain.yml.
The run() method is the one that will be called by the RASA engine when the action is triggered. It takes as input a dictionary containing the custom action request data, which can be used to extract the info like the sender id, the entities, etc.

For example, all the entities can be extracted like this:
```
entities = {
    e["entity"]: e["value"] for e in data["tracker"]["latest_message"]["entities"]
}
```
The method must return a dictionary containing the response data. The most notably: * to send a response to be uttered back to the user:
return {"responses": [{"text": _("<response>")}]}
Note

Notice how the response is localized using the _() function. See Installing a trained chatbot for more information about it.
- to add a value to a slot (like a detected system intent):
```
return{
  "events": [
    {
      "event": "slot",
      "name": "<slot name>",
      "value": "<slot value>"
    }
  ]
}
```
  Specifically for the intents slot, the expected value is the raw serialized /intents message type:
```
from hri_actions_msgs.msg import Intent
from rclpy.serialization import serialize_message

intent_msg = Intent()
# set the intent_msg fields
intents = [serialize_message(intent_msg).hex()]

return {
  "events": [
    {
      "event": "slot",
      "name": "intents",
      "value": intents
    }
  ]
}
```

Installing new custom actions¶

In order for the chatbot to be discoverable by chatbot_rasa, the new custom action must be installed as a ROS 2 Python package, together with some supporting files.

The package should be structured as follows:

my_pkg/
├── actions/
│   ├── my_new_action.py
│   └── my_other_action.py
├── resource.chatbot.rasa.action/
│   └── my_pkg
├── resource/
│   └── my_pkg
├── package.xml
├── setup.cfg
└── setup.py

actions/: the folder containing the custom actions.
resource.chatbot.rasa.action/my_pkg: the ament index marker file that tells chatbot_rasa where to find the custom actions. Its content should be the path to the folder containing the custom actions, relative to the package share folder. In this case, it should be actions. It must be installed in the chatbot.rasa.action resource index.

setup.py: the setup script to install the package as a ROS 2 Python package. The setup.py file must contain the following lines:

setup(
    ...
    data_files=[
        ('share/my_pkg/actions', [
            'actions/my_new_action.py',
            'actions/my_other_action.py'
        ]),
        ('share/ament_index/resource_index/chatbot.rasa.action', ['resource.chatbot.rasa.action/my_pkg']),
        ('share/ament_index/resource_index/packages', ['resource/my_pkg']),
        ('share/my_pkg', ['package.xml'])
    ],
    ...
)

Default chit-chat/smalltalk capabilities¶

The robot comes with default ‘chit-chat’ dialogue capabilities.

This page gives an overview of which commands the robot understands (for each entry, small language variations are permissible).

Note

As of PAL OS edge, we provide ‘chit-chat’ capabilities for the robot in the following languages:

en_US
en_GB
es_ES
ca_ES

English (en_US, en_GB)

Questions	Pre-requisites
What can you help me with?/What can you do?
What is your name?
How tall are you?
How much do you weight?
How old are you?
When is your birthday?
Are you a boy or a girl?
What is your tablet for?
Can I trust you?
Are you intelligent?
Where do you live?
How is the weather today?	Internet access
I love you/I hate you
Are you a robot?
Are you a human?
How are you?
What did I say?
Who created you?
Do you know the laws of robotics?
Do you have friends?
Can we be friends?
Do you detect emotions?
Do you eat?
Do you think robots will replace humans?
What is the time?
How is the weather in [city]	Internet access
How is the weather today?	Internet access
Are you charging?	Running on the robot
What is the battery level?	Running on the robot
Are you connected to the internet?	Running on the robot
What is you IP adress?	Running on the robot with Internet access
What languages can you speak?	Running on the robot

Spanish (es_ES)

Questions	Pre-requisites
Necesito que me ayudes
Como te llamas?
Cuánto mides?
Cuánto pesas?
Cuántos años tienes?
Eres chico o chica?
Para que es tu pantalla?
Puedo fiarme de ti?
Eres inteligente?
Dónde vives?
Que tiempo hace?	Internet access
Te quiero/Te odio
Eres un robot?
Eres humano?
Que tal estas?
Puedes repetir?
Quién te ha creado?
Sabes las leyes de la robótica?
Tienes amigos?
Podemos ser amigos?
Puedes detectar emociones?
Te gusta comer?
Crees que los robots remplazarán a los humanos?
Que hora es?
Qué tiempo hace en [ciudad]?	Internet access
Estás cargando?	Running on the robot
Cuanta batería tienes?	Running on the robot
Estas conectado a internet?	Running on the robot
Cuál es tu dirección de red?	Running on the robot and Internet access
Que idiomas puedes hablar?	Running on the robot

Catalan (ca_ES)

Questions	Pre-requisites
Necessito que m’ajudis
Com et dius?
Quant mesures?
Quant peses?
Quants anys tens?
Ets noi o noia?
Per què és la teva pantalla?
Puc fiar-me de tu?
Ets intel·ligent?
On vius?
Quin temps fa?	Internet access
T’estimo/T’odio
Ets un robot?
Ets humà?
Com estàs?
Pots repetir?
Qui t’ha creat?
Saps les lleis de la robòtica?
Tens amics?
Podem ser amics?
Pots detectar emocions?
T’agrada menjar?
Creus que los robots reemplaçaran als humans?
Quina hora és?
Quin temps fa a [ciudad]?	Internet access
Estàs carregant?	Running on the robot
Quanta bateria tens?	Running on the robot
Estàs connectat a internet?	Running on the robot
Quina és la teva adreça de xarxa?	Running on the robot with Internet access
Quins idiomes pots parlar?	Running on the robot

How-to: RASA chatbot¶

Overview¶

ROS interface¶

Dialogues support¶

Web interface¶

RASA basic concepts¶

Training a new chatbot¶

Training example¶

Installing a trained chatbot¶

Custom actions¶

Creating a new custom action¶

Installing new custom actions¶

Default chit-chat/smalltalk capabilities¶

See also¶