Skip to content

roi_pointcloud_fusion#

Purpose#

The node roi_pointcloud_fusion is to cluster the pointcloud based on Region Of Interests (ROIs) detected by a 2D object detector, specific for unknown labeled ROI.

Inner-workings / Algorithms#

  • The pointclouds are projected onto image planes and extracted as cluster if they are inside the unknown labeled ROIs.
  • Since the cluster might contain unrelated points from background, then a refinement step is added to filter the background pointcloud by distance to camera.

roi_pointcloud_fusion_image

Inputs / Outputs#

Input#

Name Type Description
input sensor_msgs::msg::PointCloud2 input pointcloud
input/camera_info[0-7] sensor_msgs::msg::CameraInfo camera information to project 3d points onto image planes
input/rois[0-7] tier4_perception_msgs::msg::DetectedObjectsWithFeature ROIs from each image
input/image_raw[0-7] sensor_msgs::msg::Image images for visualization

Output#

Name Type Description
output tier4_perception_msgs::msg::DetectedObjectsWithFeature output clusters
debug/clusters sensor_msgs/msg/PointCloud2 colored cluster pointcloud for visualization

Parameters#

Core Parameters#

Name Type Description Default Range
fusion_enabled_classes.UNKNOWN boolean Whether to enable fusion for objects with UNKNOWN class. 1 N/A
fusion_enabled_classes.CAR boolean Whether to enable fusion for objects with CAR class. 0 N/A
fusion_enabled_classes.TRUCK boolean Whether to enable fusion for objects with TRUCK class. 0 N/A
fusion_enabled_classes.BUS boolean Whether to enable fusion for objects with BUS class. 0 N/A
fusion_enabled_classes.TRAILER boolean Whether to enable fusion for objects with TRAILER class. 0 N/A
fusion_enabled_classes.MOTORCYCLE boolean Whether to enable fusion for objects with MOTORCYCLE class. 0 N/A
fusion_enabled_classes.BICYCLE boolean Whether to enable fusion for objects with BICYCLE class. 1 N/A
fusion_enabled_classes.PEDESTRIAN boolean Whether to enable fusion for objects with PEDESTRIAN class. 1 N/A
fusion_enabled_classes.ANIMAL boolean Whether to enable fusion for objects with ANIMAL class. 1 N/A
fusion_enabled_classes.HAZARD boolean Whether to enable fusion for objects with HAZARD class. 1 N/A
override_class_with_unknown boolean Whether to override class of fused output cluster with UNKNOWN. 0 N/A
min_cluster_size integer The minimum number of points that a cluster must contain to be considered as valid. 2 ≥2
max_cluster_size integer The maximum number of points that a cluster must contain to be considered as valid. 20 ≥2
cluster_2d_tolerance float A cluster tolerance measured in radial direction [m] 0.5 >0.0
roi_scale_factor float A scale factor for resizing RoI while checking if points are inside the RoI. 1 >0.0
max_object_size float The maximum size of the object to be considered as valid. 2 >0.0

Assumptions / Known limits#

  • Currently, the refinement is only based on distance to camera, the roi based clustering is supposed to work well with small object ROIs. Others criteria for refinement might needed in the future.

(Optional) Error detection and handling#

(Optional) Performance characterization#

(Optional) Future extensions / Unimplemented parts#