|
|
🖥️ User interfaces¶
Your robot might come with an additional screen, such as a tablet or a touchscreen. This screen can be used to display information, interact with the robot, or to create a more immersive experience for the user.
General documentation¶
Graphics system¶
Your robot’s display is a regular monitor, directly plugged into the robot’s main computer. It runs a stripped-down version of the Gnome desktop manager on top of the X11 Linux display server.
As such, any Linux-compatible graphical application can be run on the robot’s display, including web browsers, games, and other applications.
To start a graphical application, connect to the robot via SSH,
type export DISPLAY=:0 and run the application from the command-line.
Building a touchscreen interface via ROS and QML¶
PAL provides an easy way to build a touchscreen interface using the QML declarative language.
QML is a user interface markup language that allows you to create user interfaces in a declarative way. It is part of the Qt framework, which is widely used for building graphical user interfaces in C++ and Python.
Note
You can read more about QML and look at some code samples in the Qt documentation.
PAL has extended it to add support for ROS: you can easily create a user interface that sends messages to your main mission controller when eg a user presses a button.
The following QML code sample does just that:
import QtQuick 2.0
import QtQuick.Controls 2.0
import Ros 2.0
Item {
anchors.fill: parent
StringTopic {
id: commandTopic
topic: "/ui/command"
}
Image {
// sets a background image
// (you could use here any image, including URLs)
source: "file:///home/pal/Pictures/pal_wallpaper.jpg"
anchors.fill: parent
}
Button {
anchors.bottom: parent.bottom;
anchors.bottomMargin: 20;
anchors.horizontalCenter: parent.horizontalCenter;
text: "Lets start!";
onClicked: {
commandTopic.value = "start_app";
}
}
}
If you save this code in a file called example.qml, you can run it on the
robot by calling the ROS service /ui/set_fragment:
ros2 service call /ui/set_fragment ui_msgs/srv/SetUiFragment "qml_fragment: '`cat example.qml`'"
This will display a window with a background image and a button. When the user
clicks the button, the robot will send a message to the topic
/ui/command with the value start_app:
You can then subscribe to this topic in your Python code to receive the message and perform an action.
Check the tutorials section below for more examples of how to use QML to create a user interface, including how to update the QML content dynamically.
Architecture¶
PAL robots come with a ui_server node that manages the QML display, and let you easily set, change, update the QML content to be displayed: