Skip to content

roi_pointcloud_fusion#

Purpose#

The node roi_pointcloud_fusion is to cluster the pointcloud based on Region Of Interests (ROIs) detected by a 2D object detector, specific for unknown labeled ROI.

Inner-workings / Algorithms#

  • The pointclouds are projected onto image planes and extracted as cluster if they are inside the unknown labeled ROIs.
  • Since the cluster might contain unrelated points from background, then a refinement step is added to filter the background pointcloud by distance to camera.

roi_pointcloud_fusion_image

Inputs / Outputs#

Input#

Name Type Description
input sensor_msgs::msg::PointCloud2 input pointcloud
input/camera_info[0-7] sensor_msgs::msg::CameraInfo camera information to project 3d points onto image planes
input/rois[0-7] tier4_perception_msgs::msg::DetectedObjectsWithFeature ROIs from each image
input/image_raw[0-7] sensor_msgs::msg::Image images for visualization

Output#

Name Type Description
output tier4_perception_msgs::msg::DetectedObjectsWithFeature output clusters
debug/clusters sensor_msgs/msg/PointCloud2 colored cluster pointcloud for visualization

Parameters#

Core Parameters#

Name Type Description Default Range
fuse_unknown_only boolean Whether to fuse only UNKNOWN clusters. 1 N/A
override_class_with_unknown boolean Whether to override class of output cluster with UNKNOWN. It is effective only when fuse_unknown_only is false. 0 N/A
min_cluster_size integer The minimum number of points that a cluster must contain to be considered as valid. 2 ≥2
max_cluster_size integer The maximum number of points that a cluster must contain to be considered as valid. 20 ≥2
cluster_2d_tolerance float A cluster tolerance measured in radial direction [m] 0.5 >0.0
roi_scale_factor float A scale factor for resizing RoI while checking if points are inside the RoI. 1 >0.0
max_object_size float The maximum size of the object to be considered as valid. 2 >0.0

Assumptions / Known limits#

  • Currently, the refinement is only based on distance to camera, the roi based clustering is supposed to work well with small object ROIs. Others criteria for refinement might needed in the future.

(Optional) Error detection and handling#

(Optional) Performance characterization#

(Optional) Future extensions / Unimplemented parts#