💬 Communication

Your robot can recognize speech, handle dialogues, and synthesize voice in several languages. It is fully compliant with the ROS4HRI REP-155 ROS standard.

In its default configuration, the entire speech pipeline runs on-board; no cloud-based services are used (and consequently, no Internet connection is required).

The following figure provides complete picture of the communication pipeline on the robot, and how it interacts with components of different subsystems (clicking on the boxes redirects to the relevant page).

The main components are:

• the robot’s microphone;

• the Vosk based Automatic Speech Recognition;

• the monitor of incoming speech for wake-up/sleep keywords;

• the communication hub, a central component handling the robot’s communication capabilities, providing the high-level communication skills API;

• a chatbot, currently one of the following:

  • the RASA powered chatbot (enabled by default);

  • the LLM chatbot, using the Ollama backend;

• the semantic state aggregator, which grounds the chatbot dialogues to the the robot’s state and knowledge;

• the Acapela based Text To Speech synthesizer.

The communication subsystem is also closely integrated with the internationalization manager, with many of the components above being language-dependent.

How-to

• How-to: Automatic Speech Recognition (ASR)
• How-to: Wake-up word detector
• How-to: Dialogue management
• How-to: RASA chatbot
• How-to: LLM chatbot
• 🚧 How-to: Using the knowledge base with the chatbot
• How-to: Speech synthesis (TTS)

Tutorials

• Tutorial: Parrot the human
• 🚧 Tutorial: Programming an interactive a story

FAQ

• How to change the robot’s language?

• What languages are supported by the robot?

• How to change the ‘wake-up’ (or ‘sleep’) keywords?

• What is the default wake-up word?

References

• Communication skills