Skip to content

roi_pointcloud_fusion#

Purpose#

The node roi_pointcloud_fusion is to cluster the pointcloud based on Region Of Interests (ROIs) detected by a 2D object detector, specific for unknown labeled ROI.

Inner-workings / Algorithms#

  • The pointclouds are projected onto image planes and extracted as cluster if they are inside the unknown labeled ROIs.
  • Since the cluster might contain unrelated points from background, then a refinement step is added to filter the background pointcloud by distance to camera.

roi_pointcloud_fusion_image

Inputs / Outputs#

Input#

Name Type Description
input sensor_msgs::msg::PointCloud2 input pointcloud
input/camera_info[0-7] sensor_msgs::msg::CameraInfo camera information to project 3d points onto image planes
input/rois[0-7] tier4_perception_msgs::msg::DetectedObjectsWithFeature ROIs from each image
input/image_raw[0-7] sensor_msgs::msg::Image images for visualization

Output#

Name Type Description
output sensor_msgs::msg::PointCloud2 output pointcloud as default of interface
output_clusters tier4_perception_msgs::msg::DetectedObjectsWithFeature output clusters
debug/clusters sensor_msgs/msg/PointCloud2 colored cluster pointcloud for visualization

Parameters#

Core Parameters#

Name Type Description
min_cluster_size int the minimum number of points that a cluster needs to contain in order to be considered valid
max_cluster_size int the maximum number of points that a cluster needs to contain in order to be considered valid
cluster_2d_tolerance double cluster tolerance measured in radial direction
rois_number int the number of input rois

Assumptions / Known limits#

  • Currently, the refinement is only based on distance to camera, the roi based clustering is supposed to work well with small object ROIs. Others criteria for refinement might needed in the future.

(Optional) Error detection and handling#

(Optional) Performance characterization#

(Optional) Future extensions / Unimplemented parts#