Introduction to robot app programming

Section

Developing applications - high-level documentation

Robot applications (or robot apps) are a great way to develop and distribute new behaviours for PAL’s robots.

PAL’s robots 100% build on ROS, and a robot application is itself a ROS node that follows a few simple conventions to ease modularity, packaging and capability auto-discovery.

This page:

• introduces the main concepts and terminology you need to understand robot applications and start developing your own;

• explain how to quickly create your first application with the pal_app tool;

• explain the typical workflow for development and testing on the robot;

• link to tutorials to learn how to build more complex applications.

Structure of a robot application

First, a quick run through the PAL’s robots terminology:

• the application controller implements the main role or activity of the robot. It reacts to incoming intents and performs actions via capabilities. At a given time, only one application controller (the main controller) can be active.

• intents represent the user’s desires or commands. The main application controller monitors incoming intents, and reacts accordingly.

• capabilities are smaller, self-contained operations performed by the robot: saying something using TTS, navigating to a position, performing a gesture are all examples of capabilities. Some capabilities are always available, we call them system capabilities, some are specific to a particular application, we call them application capabilities.

• an application is a package made of exactly one application controller, and can include additional resources (eg images, sounds) and non-standard capabilities. Applications also have a manifest.

• Finally, PAL’s robots also has a high-level application manager that controls which application is currently active. For instance it could start a ‘receptionist’ application during the day, and a ‘patrolling’ application during the night.

We discuss each of these concepts in greater detail below.

Application

An application is a bundle of one application controller (see below) and all its required resources (for instance data, text, images, models, additional capabilities…)

An application must also contain a manifest, with meta data like the name of the application, etc.

Application controllers

Application controllers implement the main role or behaviour of the robot.

The robot’s default application (that you experience eg the first time you turn on the robot) comes with a simple controller that does not do much. It simply triggers the different activities demoed on the landing page.

Soon, you will want to create more interesting autonomous behaviours for the robot, suited to your task, by implementing your own application controller.

The PAL’s robots SDK does not enforce any specific structure or technique to implement your controller: you are free to use any control paradigm you wish, from simple imperative-style control scripts, to behaviour trees (BT), finite-state machines (FSM) or symbolic task planner (eg, HTN).

Note

Future releases of the PAL’s robots SDK might include ‘no-code’ tools (eg, visual programming) to quickly create application controllers. See Creating no-code demos and presentations.

In the section Development workflow: how to create new applications?, we show how a new application controller can be quickly created with the help of the pal_app tool, and then customized to your application.

Examples of application controller that you could implement yourself include:

• a ‘robot receptionist’ application, which welcomes visitors when they enter a building;

• an interactive information robot, which answers questions about a location or an organisation;

• a controller implementing robot tele-operation for an scientific study;

Alternative terminology

Application controllers are sometimes also referred as tasks, supervisors or behaviours in the literature.

Intents

An intent is an abstract description of an operation to be performed by the robot.

Intents are primarily designed to capture user-initiated desires or commands. For instance, a button click on a touchscreen, the result of a chatbot-based verbal interaction, a command started by a remote user interface.

Intents are represented as ROS messages, and they are all published on the same channel (the ROS /intents topic). By monitoring incoming messages on this channel, the active application can react to user requests.

Note

Intents are also an effective way to test/debug your application: you can manually publish a new Intent message on the /intents topic to simulate any user interaction, and check the controller’s reaction.

In order to facilitate re-usability, intents are standardised. Examples of intents include ENGAGE_WITH, GUIDE, PRESENT_CONTENT, etc.

Learn more about the structure of intents and the list of available ones.

Capabilities

Capabilities are ‘unit’ operations on the robot: navigating to a location, looking at a specific target, grasping an object, etc.

The robot’s SDK offers a set of off-the-shelf capabilities, called system capabilities. They are organised in broad categories:

• navigation capabilities

• gesture-related capabilities

• social capabilities

• expressive capabilities

• speech-related capabilities

• reasoning and knowledge-related capabilities

• or direct hardware control

You can as well create your own custom capabilities as ROS node, and bundle them with your application.

Alternative terminology

Capabilities are also sometimes referred to as Actions, Atomic actions or Atomic task.

Capabilities should not be confused with ROS actions. ROS actions are a generic mechanism to perform asynchronous remote procedure calls (RPC), while the SDK’s concept of capability refers to the implementation of a robot’s operation.

In practise, ROS actions are often used to implement the robot’s capabilities, especially for asynchronous control.

Application manager

The robot’s application manager is the top-level controller of the robot.

The application manager does not directly execute any capability or react to intents. Instead, it starts, manages, and stops applications based on high-level rules (time of the day, particular context of interaction, etc.).

In pal-sdk-23.12, you can only access the application manager via the The WebCommander tool interface, to manually start and stop applications, and configure a default application that will automatically be launched at startup.

In future revision of the SDK, we might offer additional options to schedule more complex scenarios.

Manifest

An application manifest is a text file that provides additional meta-data about the application, like its name.

As applications are primarily ROS nodes, we reuse and extend the existing ROS package.xml manifest with additional informations.

As of pal-sdk-23.12, only one additional tag is required in a ROS package manifest to declare an application: the <role> tag with the value application. This tag must appear as a child of the package.xml <export> tag.

An example of a minimal application manifest could be:

<?xml version="1.0"?>
<package format="3">
    <name>app_name</name>
    <version>1.0.0</version>
    <description>Short description of your app</description>

  <!-- One maintainer tag required, multiple allowed, one person per tag -->
  <maintainer email="todo@todo.todo">TODO</maintainer>

  <!-- One license tag required, multiple allowed, one license per tag -->
  <license>Proprietary</license>

  <!-- Dependencies can be catkin packages or system dependencies -->
  <depend>...</depend>

  <export>
      <!-- This specifies that this package should be treated as an -->
      <!-- application. -->
      <role>application</role>
  </export>
</package>

Robot package

Applications can be bundled as robot application packages. Those packages are zip archives of the application and its resources (including the manifest), and can be easily installed on the robot by drag-dropping them onto the robot’s WebCommander interface.

Robot packages must have the .rpk extension. They can be created using the pal_app tool (see below).

Caution

Robot packages are not yet fully supported in pal-sdk-23.12. You can instead use the pal_deploy tool. See Tutorial: Deploying ROS packages on the robot.

Development workflow: how to create new applications?

The pal_app tool simplifies the creation of new applications for the robot.

Discover more: Create an application with pal_app.

You can also directly go to our tutorials to learn how to create applications:

• Tutorial: ARI’s “Hello world”

• Tutorial: Creating a simple multi-modal interaction