🧩 Target Detection Plugins#

In this section, you’ll learn how to create a new Target Detection Plugin and how to use it.

Target Detection Plugins can be dynamically loaded by the target_detector_server which is a Task Server that implements the /detect_target action server.

The detection pipeline is independent of the detector plugin and is composed by the following steps:

Initialization: On startup, the server loads the specified detection plugins and subscribes to necessary topics.
Detection Request: Upon receiving a detection request, the server invokes the appropriate plugin to process the image data.
Pose Calculation: For each detected target, the plugin calculates its pose relative to the camera.
Result Publication: Detection results, including pose information, are published on the configured topic for use by other system components.

The target pose must follow the convention defined in REP-103 (x: forward; y: left; z: up). Therefore, plugins are responsible for complying with this requirement.

How to create your Target Detection Plugin#

In this tutorial, you are going to learn how to create a new Target Detection Plugin that detects a fake target 2 meters in front of the robot. For this demonstration, we are going to use a ROS 2 package called target_detection_tutorial in which we are going to create a new C++ source file called dummy_target_detector.cpp.

#include <memory>
#include <string>
#include "rclcpp/rclcpp.hpp"
#include "rclcpp/clock.hpp"
#include "rclcpp_lifecycle/lifecycle_node.hpp"
#include "tf2_ros/buffer.h"
#include "pal_nav2_core/target_detector.hpp"

namespace target_detection_tutorial
{

class DummyTargetDetector : public pal_nav2_core::TargetDetector
{
public:
  DummyTargetDetector() = default;
  ~DummyTargetDetector() = default;

  void configure(
    const rclcpp_lifecycle::LifecycleNode::WeakPtr & parent, std::string name,
    std::shared_ptr<tf2_ros::Buffer> ) override
  {
    parent_ = parent;
    auto node = parent_.lock();
    if (!node) {
      throw std::runtime_error{"Failed to lock node in simple target detector plugin!"};
    }
    plugin_name_ = name;
    logger_ = node->get_logger();
    clock_ = node->get_clock();
  }

  void cleanup() override
  {
    RCLCPP_INFO(logger_, "Cleaning up %s target detector", plugin_name_.c_str());
  }

  void activate() override
  {
    RCLCPP_INFO(logger_, "Activating %s target detector", plugin_name_.c_str());
  }

  void deactivate() override
  {
    RCLCPP_INFO(logger_, "Deactivating  %s target detector", plugin_name_.c_str());
  }

  bool detectTarget(
    int id, geometry_msgs::msg::TransformStamped & transform,
    double & accuracy) override
  {
    if (id != 0) {
      RCLCPP_WARN(logger_, "%s: Target not detected", plugin_name_.c_str());
      return false;
    }
    accuracy = 0.8;
    transform.header.frame_id = "base_footprint";
    transform.child_frame_id = "target";
    transform.header.stamp = clock_->now();
    transform.transform.translation.x = 2.0;

    return true;
  }

  bool isValidTime(const rclcpp::Duration &) override
  {
    return true;
  }

private:
  rclcpp::Logger logger_{rclcpp::get_logger("dummy_target_detector")};
  rclcpp_lifecycle::LifecycleNode::WeakPtr parent_;
  rclcpp::Clock::SharedPtr clock_;
  std::string plugin_name_;
};
}  // namespace target_detection_tutorial
#include "pluginlib/class_list_macros.hpp"
PLUGINLIB_EXPORT_CLASS(
  target_detection_tutorial::DummyTargetDetector,
  pal_nav2_core::TargetDetector)

Examine the code#

The example plugin inherits from the pal_nav2_core::TargetDetector class, which is the base class for all the Target Detection Plugins.

class SimpleTargetDetector : public pal_nav2_core::TargetDetector

This class provides the following virtual methods that need to be implemented by the plugin class.

Virtual method	Description	Requires override?
`confiugure()`	Method that is called when the Target Detection plugin is initialized. This method takes as input a `const rclcpp_lifecycle::LifecycleNode::WeakPtr & parent` to define the parent node, a `std::string name` to define the plugin name, and a `std::shared_ptr<tf2_ros::Buffer> tf` to define a pointer to the tf buffer.	Yes
`cleanup()`	Method that is used to clean up all the defined resources.	Yes
`activate()`	Method that is used to activate the target detector and any threads involved in execution. It sets up the plugin with all its publishers, subscribers, or any ROS services that will be used.	Yes
`deactivate()`	Method used to deactivate the target detector and any threads involved in execution.	Yes
`detectTarget()`	Method that is called to detect and localize a target given the id. It takes as input an `int id` to define the target id, a `geometry_msgs::msg::TransformStamped & transform` to define the transform between the robot and the target, and a `double & accuracy` to define the detection accuracy. The method returns true if the target has been found and localized, and false otherwise.	Yes
`reset()`	Method that is called to reset the state of the `TargetDetector` if necessary after the task is exited.	No

In the configure() function you can setup the ROS 2 Interface of your plugin (e.g Subscribers, Publishers, Services and so on) using the parent_ variable which is an instance of the rclcpp_lifecycle::LifecycleNode::WeakPtr class and which allows you to access the base ROS 2 Node.

rclcpp_lifecycle::LifecycleNode::WeakPtr parent_;
auto node = parent_.lock();
if (!node) {
  throw std::runtime_error{"Failed to lock node in simple target detector plugin!"};
}

The detectTarget() funcion function is where the real logic of the plugin is implemented. It should generate a geometry_msgs::msg::TransformStamped message with the pose of the detected target and the accuracy of the detection. In this tutorial, this pose is simply hardcoded to be 2 meters in front of the robot.

transform.header.frame_id = "base_footprint";
transform.child_frame_id = "target";
transform.header.stamp = clock_->now();
transform.transform.translation.x = 2.0;

Compile and install#

To compile and install the plugin, the file CMakeLists.txt should declare the following (minimum) dependencies:

find_package(ament_cmake REQUIRED)
find_package(rclcpp REQUIRED)
find_package(rclcpp_lifecycle REQUIRED)
find_package(pal_nav2_core REQUIRED)
find_package(pluginlib REQUIRED)
find_package(tf2_ros REQUIRED)

set(dependencies
  rclcpp
  rclcpp_lifecycle
  pal_nav2_core
  pluginlib
  tf2_ros
)

To compile and install the plugin, you need to add and install a new library

add_library(dummy_target_detector SHARED src/dummy_target_detector.cpp)
ament_target_dependencies(dummy_target_detector ${dependencies})

install(TARGETS dummy_target_detector
  DESTINATION lib/${PROJECT_NAME})

Also, it should be created a new XML file called plugins.xml within the target_detection_tutorial package with the declaration of the plugin.

<class_libraries>
    <library path="dummy_target_detector">
        <class type="target_detection_tutorial::DummyTargetDetector" base_class_type="pal_nav2_core::TargetDetector">
            <description>Detects a fake target 2mts in front of the robot</description>
        </class>
    </library>
</class_libraries>

Finally, the package.xml file should include the required dependencies:

<depend>rclcpp</depend>
<depend>rclcpp_lifecycle</depend>
<depend>pal_nav2_core</depend>
<depend>pluginlib</depend>
<depend>tf2_ros</depend>

And should export the plugin:

<export>
  <build_type>ament_cmake</build_type>
  <target_detection_tutorial plugin="${prefix}/plugins.xml" />
</export>

Usage#

To start using your new plugin, you can create a new target_detector_server configuration file. Start by creating a new folder in the .pal directory of your robot.

mkdir -p ~/.pal/config

And then, within this folder, you can create a new target_detector_server configuration file.

touch ~/.pal/config/99_my_target_detector_server_config.yaml

Attention

The name of the file should start with a number to ensure that it is loaded last and overrides the default PAL configurations.

Within the newly created 99_my_target_detector_server_config.yaml file, you can insert your custom target_detector_server parameters. When creating a new Navigation configuration file, you need to specify the node name it refers to, in this case, target_detector_server.

/target_detector_server:
   ros__parameters:
     # Your target_detector_server parameters

     detector_plugins: [... , "dummy_target_detector"]

     #Other target_detector_server plugin configurations
     dummy_target_detector:
         plugin: "target_detection_tutorial::DummyTargetDetector"

Once created, to start using your custom target_detector_server configuration, you need to restart the advanced_navigation module with the command:

pal module restart advanced_navigation

Note

This change is is persistent and will be loaded every time you start the advanced_navigation module. If you want to revert to the default PAL configuration, you can simply delete the custom configuration file you created in the ~/.pal/config folder.

This will restart the target_detector_server with your new plugin that you can use to detect a new target with the command:

ros2 action send_goal /detect_target pal_nav2_msgs/action/DetectTarget "{id: 0, detector: 'dummy_target_detector'}"

If you want to visualize the results, you can add the /updated_goal to the RViz configuration and check the /tf.

You should see something like this: