📝 Developing robot apps

Section

📜 Developing applications - high-level documentation

Robot applications (or robot apps) are a great way to develop and distribute new behaviours for PAL’s robots.

PAL’s robots are built on top of ROS (specifically, as of PAL OS 25.01, ROS 2 Humble), and a robot application is itself a package of several ROS nodes, combined together to implement a specific behaviour.

This page:

• introduces the main concepts and terminology you need to understand robot applications and start developing your own;

• explain how to quickly create your first application with the rpk tool;

• explain the typical workflow for development and testing on the robot;

• link to tutorials to learn how to build more complex applications.

Structure of a robot application

First, a quick run through the terminology:

• the mission controller implements the main role or behaviour of the robot. It reacts to incoming intents and starts tasks to perform actual actions. At a given time, only one mission can be active.

• intents represent the user’s desires or commands. The mission controller monitors incoming intents, and reacts accordingly. Intents are recognised (or extracted) by dedicated nodes, the intent recognition nodes. Your robot provides a set of standard intents recognition nodes (including, for instance, a chatbot), but you can also create your own.

• a task is a (finite) set of actions performed by the robot to achieve a specific goal (i.e. “Doing an inventory”, “Getting a glass of water”, “Going to the kitchen”). It is time-bound (i.e. it has a finite duration), and can usually be paused and resumed. Tasks rely on the robot individual skills to perform to actions.

• skills are smaller, self-contained operations performed by the robot: saying something using TTS, navigating to a position, performing a gesture are all examples of skills. Some skills are provided out-of-the-box on the robot, we call them system skills, some other are custom for a particular application, we call them user skills.

• Finally, an application is a package made of usually one mission controller, several tasks implementations, and, based on the needs, custom intent recognition nodes, skills, and generic resources (eg images, sounds).

Note

This terminology is based on the RobMoSys conceptual model.

We discuss each of these concepts in greater detail below.

How to create a new application

The tutorial: Create an application with rpk will guide you through the steps to create a new application:

1. Create a new application package using the rpk template generator. This will create a new directory with the basic structure of a robot application.

2. Customize the mission controller to implement the desired behaviour. If needed, create as well new tasks, skills, and intent recognition nodes. Templates are provided to help you get started with each of those.

3. Build the application using the standard ROS 2 build tools (colcon build).

4. Deploy the application on your robot: Deploying ROS 2 packages on your robot.

Note

The rpk tool is a command-line tool that helps you create new applications and manage your robot packages. It is available on the robot’s command line.

As of PAL OS 25.01, the rpk tool is the recommended way to create new applications.

In details: the components of a robot application

Application

An application is a bundle of one mission controller (see below) and all its required resources (for instance data, text, images, models, additional skills…)

Mission controllers

Mission controllers implement the main role or behaviour of the robot. Its main role it to react to incoming intents and start tasks to perform actual actions. In that sense, the mission controller is the orchestrator of the robot’s behaviour.

The robot’s default application (that you experience eg the first time you turn on the robot) comes with a simple controller that does not do much. For instance, on the ARI robot, it simply triggers the different tasks demoed on the landing page.

Soon, you will want to create more interesting autonomous behaviours for the robot, suited to your task, by implementing your own mission controller.

PAL OS does not enforce any specific structure or technique to implement your controller: you are free to use any control paradigm you wish, from simple imperative-style control scripts, to behaviour trees (BT), finite-state machines (FSM) or symbolic task planner (eg, HTN).

In the section Next steps, we show how a new mission controller can be quickly created with the help of the rpk tool, and then customized to your application.

Examples of mission controller that you could implement yourself include:

• a ‘robot receptionist’ mission, which welcomes visitors when they enter a building;

• an interactive information robot, which answers questions about a location or an organisation;

• a controller implementing robot tele-operation for an scientific study;

Alternative terminology

Mission controllers are sometimes also referred to as supervisors or high-level behaviours in the literature.

Intents and intent recognition

An intent is an abstract description of an operation to be performed by the robot.

Intents are primarily designed to capture user-initiated desires or commands. For instance, a button click on a touchscreen, the result of a chatbot-based verbal interaction, a command started by a remote user interface.

Intents are represented as ROS messages, and they are all published on the same channel (the ROS /intents topic). By monitoring incoming messages on this channel, the active mission can react to user requests.

Note

Intents are also an effective way to test/debug your application: you can manually publish a new Intent message on the /intents topic to simulate any user interaction, and check the controller’s reaction.

In order to facilitate re-usability, intents are standardised. Examples of intents include ENGAGE_WITH, GUIDE, PRESENT_CONTENT, etc.

Learn more about the structure of intents and the list of available ones.

Your application must provide a way to extract intents from the user’s input. This is done by creating an intent recognition node, which listens to the user’s input (eg, a chatbot) and publishes the corresponding intent on the /intents topic.

The robot comes with a set of standard intent recognition nodes, including a chatbot and a node for automatic engagement detection.

🚧 Tutorial: Creating a interactive user interface explains how to create your own intent recognition node using a graphical user interface.

Tasks

A task represents a finite set of actions performed by the robot to achieve some goal (i.e. “Doing an Inventory”, “Getting a glass of water”, “Going to the kitchen”). It is time bound (i.e. it has a finite duration), and can usually be paused and resumed.

The robot’s mission controller is responsible to start (and possibly stop/pause/resume) the required tasks to achieve the desired behaviour. It does so by calling the task’s control API (usually, a ROS action called /<task_name>/control, but this may depend on each particular task).

Tasks themselves then rely on calling skills to perform specific actions.

Note

When developing simpler applications, your mission controller might decide to directly call skills, without creating a task. This is fine, but not recommended for complex applications, as it makes it harder to monitor and control the robot’s behaviour.

Tasks are regular ROS nodes and users usually create their own tasks, as required by their application.

Alternative terminology

Tasks are sometimes also referred to as activities or behaviours in the literature.

Skills

Skills are ‘unit’ operations on the robot: navigating to a location, looking at a specific target, grasping an object, etc.

PAL OS offers a set of off-the-shelf skills, called system skills. They are organised in broad categories:

• navigation skills

• gesture-related skills

• social skills

• expressive skills

• communication-related skills

• reasoning and knowledge-related skills

• or direct hardware control

You can as well create your own custom skills as ROS nodes, and bundle them with your application.

Alternative terminology

Skills are also sometimes referred to as Actions, Atomic actions or Atomic task.

Skills should not be confused with ROS actions. ROS actions are a generic mechanism to perform asynchronous remote procedure calls (RPC), while PAL OS’ concept of skills refers to the implementation of a robot’s operation.

That said, in practice, ROS actions are often used to implement the robot’s skills, especially for asynchronous control.

Next steps

The next step is to create your first application. The tutorial Create an application with rpk will guide you through the steps to create a new application.