../_images/tiagopro-icon.png ../_images/tiago-head-icon.png ../_images/tiago-icon.png ../_images/ari-icon.png

Tutorial: Parrot the human

This is the first of a series of tutorials on how to use the communication subsystem of the robot. We will have code examples of increasing complexity, progressively leveraging additional features of the communication subsystem.

This tutorial is about how to program a simple parrot-like behavior.

🏁 Goal of this tutorial

By the end of this tutorial, you will know how to:

  • create a simple ROS 2 package interacting with the communication subsystem

  • use the ASR system to recognize the human speech

  • use the How-to: Speech synthesis (TTS) system to reply back to the human

Pre-requisites

You should first have read how to:

How it works

The parrot_tutorial example node below shows how to subscribe to the audio transcription, provided by ASR, and reply back the same sentence. To utter a sentence, we use its How-to: Speech synthesis (TTS) capability, calling the /say.

 1#!/usr/bin/env python3
 2# -*- coding: utf-8 -*-
 3from communication_skills.action import Say
 4from hri_msgs.msg import LiveSpeech
 5import rclpy
 6from rclpy.action import ActionClient
 7from rclpy.callback_groups import MutuallyExclusiveCallbackGroup
 8from rclpy.executors import ExternalShutdownException, MultiThreadedExecutor
 9from rclpy.node import Node
10from rclpy.task import Future
11
12# The following demo subscribes to speech-to-text output and triggers TTS
13# based on response
14
15class ASRDemo(Node):
16    def __init__(self):
17        super().__init__("asr_tutorial")
18
19        self.asr_sub = self.create_subscription(
20            LiveSpeech,
21            '/humans/voices/anonymous_speaker/speech',
22            self.asr_result,
23            1)
24
25        self.say_client = ActionClient(
26            self,
27            Say,
28            "/say",
29            callback_group=MutuallyExclusiveCallbackGroup())
30        self.say_client.wait_for_server()
31
32        self.tts_goal_future_handle = Future()
33
34        self.get_logger().info("ASR demo ready")
35
36    def asr_result(self, msg: LiveSpeech):
37
38        # the LiveSpeech message has two main field: incremental and final.
39        # 'incremental' is updated has soon as a word is recognized, and
40        # will change while the sentence recognition progresses.
41        # 'final' is only set at the end, when a full sentence is
42        # recognized.
43        sentence = msg.final
44
45        self.get_logger().info("Understood sentence: " + sentence)
46
47        goal = Say.Goal()
48        goal.meta.priority = 1
49        if (sentence == "hello"):
50            goal.input = "Hello!"
51        elif (sentence == "how are you"):
52            goal.input = "I am feeling great"
53        elif (sentence == "goodbye"):
54            goal.input = "See you!"
55
56        self.say_client.send_goal(goal)
57
58
59
60if __name__ == "__main__":
61    rclpy.init()
62    node = ASRDemo()
63    executor = MultiThreadedExecutor()
64    executor.add_node(node)
65    try:
66        executor.spin()
67    except (KeyboardInterrupt, ExternalShutdownException):
68        node.destroy_node()

To run this ROS 2 node, you can start a PAL OS Docker container and create a ROS 2 Python package for it as described in Build and run a new ROS package.

You can notice how in the list of included libraries, we have included the rclpy library (which is the ROS 2 Python library), and the PAL libraries communication_skills and hri_msgs. Make sure to include them all in your package.xml:

<?xml version="1.0"?>
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
<package format="3">
  ...
  <depend>communication_skills</depend>
  <depend>hri_msgs</depend>
  <depend>rclpy</depend>
  ...
</package>

Then, you can either:

Next steps

The next tutorial is yet to be written, check back later! .. TODO: Add a link to the next tutorial