How-to: Wake-up word detector

Section

💬 Communication - how-to/FAQ

Overview

PAL robots feature a wake-up/sleep keyword monitor (also known as keyword spotter), that can be used to enable or disable language processing.

ROS interface

This wake-up word monitor is a soft monitor. Speech recognition is always performed in the background; the wake-up word detector monitors the /humans/voices/*/speech topics for configurable keywords, and publish a wakeup or suspend intent on /intents whenever, respectively, a wake-up or sleep pattern is detected.

Important

Note that these intents do not do anything by themselves: to actually stop the speech processing and/or put the robot to ‘sleep’, your mission controller must handle these intents and perform the corresponding actions.

You can check how this is done for instance in the pre-installed 🚧 Default application or in the rpk-generated mission controller templates.

The wake-up and sleep patterns are defined as regular expressions, one set per supported language, respectively in the wakeup_patterns.<locale> and sleep_patterns.<locale> parameters.

Default wake-up/sleep words

By default, the ‘wakeup’ pattern for en_US are: * ^(hey|hello|wake up) (robots?|robert)$ * ^(robots?|robert)( please)? listen$

For instance “robot listen” or “hello robot” (the speech recognition algorithm often confuses robot and robert, so both are accepted).

The default ‘sleep’ pattern is ^(robots?|robert)( please)? (stop listening|sleep)$, for instance one of “robot please stop listening”, “robot stop listening”, “robot please sleep”, “robot sleep”.

soft_wakeup_word is a localized node, meaning its current language selection is controlled by the i18n_manager. Refer to [‼️ROS 1] Internationalisation and language support for more information on language availability and selection.

Web interface

The Web User Interface provides the status of the ASR node under Diagnostics > Communication > Wakeup > soft_wakeup_word.

There you can check, among other things: * Current wake-up patterns: the wake-up patterns for all supported locales; * Current sleep patterns: the sleep patterns for all supported locales.

How to change the wake-up/sleep keywords?

You can easily set a custom regular expression as wake-up/sleep patterns by setting the related parameters. Let’s consider only the en_US locale for the sake of the example.

For instance, for the robot to start listening when you say: “hello robot”, you might want to call:

ros2 param set /soft_wakeup_word wakeup_patterns.en_US  ['^hello robot$'']

Another example, for the robot to stop listening when you say “sleep well” or “sleep well little robot”:

ros2 param set /soft_wakeup_word wakeup_patterns.en_US  ['^^sleep well( little robot|)$'']

Your pattern must follow the syntax of the ECMAscript regular expressions (with minor C++-only modifications).