|
|
Advanced Grasping API#
Attention
This page documents a Premium robot feature. It may or may not be available on your robot.
Important
If you are new to the Advanced Grasping pipeline, you might first want to check the overview of the Overview of the Advanced Grasping pipeline.
ROS API#
Action server interface#
Perception
/advanced_grasping/perception/goal(pal_bt_grasping_msgs/PerceptionAction)Action interface that looks for an object on a surface in front of the robot. The action will create a representation of the environment perceived with the camera. This representation is stored as an octomap in the MoveIt! planning scene. The detected object is stored as well in the planning scene with a unique object id.
string
object_color: Specify the color of the object. New colors can be used by adding a color template figure (Object detection). If this argument is empty no color selection is done.string
object_type: Select the type of object to grasp. Available options are cylinder, cube or sphere. If this argument is empty, the algorithm will look for any of these objects.
rostopic pub /advanced_grasping/perception/goal pal_bt_grasping_msgs/PerceptionActionGoal "header:
seq: 0
stamp:
secs: 0
nsecs: 0
frame_id: ''
goal_id:
stamp:
secs: 0
nsecs: 0
id: ''
goal:
object_color: ''
object_type: ''"
The action returns a result containing the following information:
bool
success: Returns true if an object was correctly detected.string
object_id: Ifsuccessis true, this will contain the name of the detected object.string
error_msg: Ifsuccessis false this will contain an error message with more information.
Grasp
/advanced_grasping/grasp/goal(pal_bt_grasping_msgs/GraspObjectAction)Action interface that tries to grasp the object specified in the goal. In the MoveIt! planning scene the object will now be attached to the end-effector.
string
object_id: Specify the name of the object that is grasped. Note that this object has to be present in the Moveit planning scene and is given by the Perception action.string
grasping_side: Choose the arm used to grasp the object. Can be empty for TIAGo. For TIAGo++, the options are left or right.
rostopic pub /advanced_grasping/grasp/goal pal_bt_grasping_msgs/GraspObjectActionGoal "header:
seq: 0
stamp:
secs: 0
nsecs: 0
frame_id: ''
goal_id:
stamp:
secs: 0
nsecs: 0
id: ''
goal:
object_id: ''
grasping_side: ''"
The action returns a result containing the following information:
bool
success: Returns true if an object was correctly grasped.string
error_msg: Ifsuccessis false this will contain an error message with more information.
Place
/advanced_grasping/place/goal(pal_bt_grasping_msgs/PlaceObjectAction)Action interface that tries to place the object in a position specified in the goal. Once successful the MoveIt! planning scene will be cleared.
string
grasping_side: Choose the arm used to grasp the object. Can be empty for TIAGo. For TIAGo++, the options are left or right.bool
auto_determine_height: Choose if the placement height is automatically calculated by the software or given by the user.geometry_msgs/PoseStamped
pose: The placement position where the robot should place the object. If no frame id is set in the header, the default frame, base_footprint, will be used.
rostopic pub /advanced_grasping/place/goal pal_bt_grasping_msgs/PlaceObjectActionGoal "header:
seq: 0
stamp:
secs: 0
nsecs: 0
frame_id: ''
goal_id:
stamp:
secs: 0
nsecs: 0
id: ''
goal:
grasping_side: ''
auto_determine_height: false
pose:
header:
seq: 0
stamp:
secs: 0
nsecs: 0
frame_id: ''
pose:
position:
x: 0.0
y: 0.0
z: 0.0
orientation:
x: 0.0
y: 0.0
z: 0.0
w: 0.0"
The action returns a result containing the following information:
bool
success: Returns true if an object was correctly grasped.string
error_msg: Ifsuccessis false this will contain an error message with more information.
Service interface#
/advanced_grasping/clear_planning_scene(pal_bt_grasping_msgs/ClearScene)This service will clean the MoveIt planning scene. Some objects can be excluded from the planning scene.
string []
excluded_objects: A list of object ids that will not be cleared during the service call. To specifically exclude the octomap the following id has to be used: <octomap>
Behavior trees#
Each action server of the AG package will start a behavior tree.
An in depth explanation and code implementation of behavior trees can be found in the documentation of BehaviortreeCPP.V3.
Note that Advanced Grasping currently only is compatible with BehaviorTreeCPP version 3.8.
The behavior trees are created using XML files. These files can be edited directly through a text editor. There also exists a graphical editor, Groot, that provides a GUI to create new behavior trees. For more information on Groot, please follow their official documentation.
By default the behavior trees are located in the folder
~/.pal/advanced_grasping/bt/. Three standard trees are provided by the
Advanced Grasping package, perception_pipeline.xml,
grasping_pipeline.xml and placing_pipeline.xml. Each behavior tree is
linked to an action in a configuration file that specifies the name of the
action, the action type and the behavior tree file. The file can be found at
pal_bt_grasping_tiago/config/tiago_config.yaml:
behaviortree_config:
move_group_capabilities:
- move_group/ExecuteTaskSolutionCapability
actions:
- grasp:
behaviortree_file: grasp_pipeline.xml
action_plugin: advanced_grasping/GraspServer
- place:
behaviortree_file: place_pipeline.xml
action_plugin: advanced_grasping/PlaceServer
- perception:
behaviortree_file: perception_pipeline.xml
action_plugin: advanced_grasping/PerceptionServer
Configuration#
The Advanced Grasping package provides many files that contain configurable parameters. These parameters can be modified to get the desired behavior. The customization can be divided into 3 main groups:
General configuration (behavior trees and actions)
Object detection
Generation of grasp candidates
General Configuration#
As explained in previous sections, the AG package use a plugin based framework for the ROS actions and behavior trees.
This allows for easy customization. Besides creating new behavior trees, new action servers can be created and added to the framework.
This can be done by updating the file tiago_config.yaml introduced in
Section Behavior trees. A more in depth explanation of this will be
given in Section Create an action server plugin.
Object detection#
There are three configuration files that provide object detection/segmentation customization. Object detection is done using pointclouds. Filters and transformations are applied to these pointclouds to detect the shape and size of the possible objects. With the help of template matching it is also possible to perform color-based object detection. The three files are:
object_detection_params.yamlobject_histogram_params.yamlplane_segmentation_params.yaml
The object_detection_params.yaml provides parameters regarding the detection
of primitive shapes.
TabletopDetectionParams:
rate: 3.0
detector_type: 2 # not used
box_neighbors: 10
box_distance_threshold: 0.001
box_points_threshold: 20
sphere_neighbors: 10
sphere_distance_threshold: 0.001
sphere_points_threshold: 150
sphere_minRadius: 0.015
sphere_maxRadius: 0.05
cylinder_neighbors: 10
cylinder_distance_threshold: 0.01
cylinder_points_threshold: 50
cylinder_minRadius: 0.015
cylinder_maxRadius: 0.05
closer_cluster_points_threshold: 100
closer_cluster_minHeight_threshold: 0.05
closer_cluster_maxHeight_threshold: 0.25
The object_histogram_params.yaml provides parameters regarding template
matching for color detection.
TabletopHistogramParams:
threshold: 254
dilate_iterations: 9
dilate_size: 7
erode_iterations: 0
erode_size: 3
dark_pixels_threshold: 15
These parameters become relevant when using color detection. To include more available colors for color detection, they can be added to the following path, named with the desired color identifier:
~/.pal/advanced_grasping/templates/
Finally, the plane_segmentation_params.yaml file provides some preprocessing
filters to be applied to the pointcloud. These filters can be used to remove
points in zones that are not of interest for the grasping application.
PlanarSegmentationParams:
rate: 2.0
processing_frame: "base_footprint"
passthrough_zmin: 0.2
passthrough_zmax: 1.1
passthrough_xmin: 0.2
passthrough_xmax: 1.1
distance_threshold: 0.015
downsampling_size: 0.001
use_horizontal_filter: true
horizontal_filter_threshold: 10.0
horizontal_filter_max_attempts: 3
Generation of grasp candidates#
To ensure successful grasps, different gripper positions are generated around the object. Each of this possible position is called Grasp Candidate and coincides with the grasping frame of the gripper. This frame is located between the fingers of the gripper.
These candidates could be defined differently depending on the gripper axis convention that you are using. Pal Robotics convention defines the X axis pointing towards the object, hence, pointing outside the gripper (Figure Gripper convention). To control the movements of the arm MoveIt! is used. MoveIt! uses a different convention for the grasping frame definition. It defines the Z axis to be pointed toward the object.
Figure: (Left) Pal Robotics grasping frame convention. (Right) MoveIt! grasping frame convention.#
More grasping frames, apart from PAL standard frame, are provided to get compatibility with other conventions:
gripper_grasping_frame: X axis pointing towards the object, PAL conventiongripper_grasping_frame_Z: Z axis pointing towards the object, MoveIt! conventiongripper_grasping_frame_Y: Y axis pointing towards the object
The MoveIt Grasps
package is a grasp generator for objects such as blocks or cylinders. This
package generates the candidates for primitive shapes. It can be configured by
some parameters that are available in the following path
pal_bt_grasping_tiago/config.
All the customizable files for the Advanced Grasping package can be found running:
roscd /pal_bt_grasping_tiago/config
ll
There is a copy of these files with default values in moveit_grasps/config
or in PAL wrapper pal_bt_grasp_generators/config, in case it needs to be
used in standalone.
For a better understanding of what each file is for, the following list is provided:
moveit_grasps_config.yaml: Provides parameters to configure the internal behavior of the generator and the filters. Basically, they are visualization and debugging parameters.moveit_grasps_generator_config.yaml: Provides the ideal grasp orientation and can tune the scores to filter the candidates that are far away from the ideal orientation.moveit_grasps_data_pal-gripper.yaml: Contains the geometric information of the PAL gripper, including the type of gripper (finger).moveit_grasps_data_robotiq-2f-140.yaml: Contains the geometric information of the Robotiq-2f-140 gripper, including the type of gripper (finger).moveit_grasps_data_robotiq-2f-85.yaml: Contains the geometric information of the Robotiq-2f-85 gripper, including the type of gripper (finger).
For more details about the parameters inside these files, please check the official MoveIt Grasps documentation.
API customization#
This section will go more in depth on how to adapt the Advanced Grasping package to custom applications.
First, it is shown how an individual node in a behavior tree can be changed. Next, how to change the behavior tree linked to a specific action. Then, how to create a new custom behavior tree node and finally, how to create a new action server plugin.
Change a behavior tree node#
The behavior trees are written in XML format. A more extensive documentation on this layout can be found on the website BehaviotreeCPP_V3.
To change behavior tree node DoSomething to DoSomethingElse, it suffices
to change the line of the existing node with the new node. A more detailed
example can be found in the Advanced Grasping tutorials.
<root main_tree_to_execute = "MainTree" >
<BehaviorTree ID="MainTree">
<Action ID="DoSomething" input_port_name="example_string"/>
</BehaviorTree>
</root>
<root main_tree_to_execute = "MainTree" >
<BehaviorTree ID="MainTree">
<Action ID="DoSomethingElse" different_input_port_name="example_string_2"/>
</BehaviorTree>
</root>
If compatibility with Groot is required, ensure that the TreeNodeModel in the XML file is updated as well:
<TreeNodeModel>
<Action ID="DoSomethingElse">
<input_port name="different_input_port_name"/>
</Action>
</TreeNodeModel>
Change a behavior tree#
Each action server executes a specific behavior tree. The linked tree is set in the configuration file from Section Behavior trees, listed above. Changing the parameter behaviortree_file for an action to the desired tree will change the executed tree. A more detailed example can be found in the Advanced Grasping tutorials.
Important to take into account is that a tree might require certain blackboard variables to be set within the action server plugin. If these are not set by the action server it is necessary to create a new plugin, which is handled in Section Create an action server plugin
Create a behavior tree node#
To create a new behavior tree node it is recommended to start with the documentation of BehaviorTreeCpp. A brief overview can bo found on how to create general behavior tree nodes in this documentation. An in depth tutorial for creating nodes for the Advanced Grasping package can be found in the Advanced Grasping Tutorials. There are three important steps to take:
Create a C++ class and its header file. Keep in mind that this class has to be inherited from a behavior tree node class.
Register the new class as a behavior tree node.
Export the plugin in the CMakeLists.txt and the package.xml of the package.
Create an action server plugin#
As mentioned before it is possible to create new action server plugins for the Advanced Grasping package. An in depth tutorial can be found in the Advanced Grasping Tutorials. Similar to creating a new behavior tree node there are four important steps to take:
Create a C++ class and its header file. Keep in mind that this class has to be inherited from the AdvancedGraspingServer class.
Register the plugin and add it to the plugin_description.xml.
Export the plugin in the CMakeLists.txt and the package.xml of the package.
Add the new plugin and corresponding behavior tree to the main configuration file
pal_bt_grasping_tiago/config/tiago_config.yaml.
See also#
Overview of the Overview of the Advanced Grasping pipeline