../_images/tiago-icon.png ../_images/tiagopro-icon.png ../_images/ari-icon.png

Social perception with ROS4HRI#

ROS for Human-Robot Interaction (or ROS4HRI [ros4hri]) is the main API that PAL’s robots implements to represent information about the human surrounding and interacting with the robot.

ROS4HRI actually refers to a set of conventions and tools that help developing Human-Robot Interaction capabilities. The specification (originally developed by PAL Robotics) is available online as the ROS REP-155.

The various IDs of ROS4HRI

The ROS4HRI API defines several types of identifiers (IDs)#

On PAL’s robots, we have implemented the following main parts of the specification:

  1. we follow the ROS4HRI human model representation, as a combination of a permanent identity (person) and transient parts that are intermittently detected (e.g. face, skeleton, voice);

  2. in pal-sdk-23.12, we specifically support:

    • face detection and recognition (including extraction of facial landmarks);

    • single body detection, and 2D and 3D skeleton tracking;

    • speech recognition (without support for voice separation or voice identification)

    • probabilistic fusion of faces, bodies and voices

  3. we follow the ROS4HRI topic naming conventions, with all human-related messages published under the /humans/ topic namespace;

  4. we follow the ROS4HRI kinematic model of the human and 3D tf frame conventions (naming, orientation), as specified here.

In addition, PAL’s robots also provide:

  • gaze estimation

  • automatic engagement detection

How to use ROS4HRI?#

The ROS4HRI topics are documented on Social perception topics.

To ease access to these (many!) topics, you can use pyhri (Python) or libhri (C++). These two open-source libraries are developed by PAL Robotics, and are documented on the ROS Wiki:

Next steps#

More tutorials and ‘how-to’ on the Social perception main page.

See also#

  • The REP-155 aka ROS4HRI specification, on ROS website

  • The ROS wiki contains useful resources about ROS4HRI.

References#

[ros4hri]

ROS for Human-Robot Interaction Y. Mohamed; S. Lemaignan, IROS 2021, doi: 10.1109/IROS51168.2021.9636816