multi_object_tracker#

Purpose#

The results of the detection are processed by a time series. The main purpose is to give ID and estimate velocity.

Inner-workings / Algorithms#

This multi object tracker consists of data association and EKF.

multi_object_tracker_overview

Data association#

The data association performs maximum score matching, called min cost max flow problem. In this package, mussp[1] is used as solver. In addition, when associating observations to tracers, data association have gates such as the area of the object from the BEV, Mahalanobis distance, and maximum distance, depending on the class label.

EKF Tracker#

Models for pedestrians, bicycles (motorcycles), cars and unknown are available. The pedestrian or bicycle tracker is running at the same time as the respective EKF model in order to enable the transition between pedestrian and bicycle tracking. For big vehicles such as trucks and buses, we have separate models for passenger cars and large vehicles because they are difficult to distinguish from passenger cars and are not stable. Therefore, separate models are prepared for passenger cars and big vehicles, and these models are run at the same time as the respective EKF models to ensure stability.

Inputs / Outputs#

The interfaces below are generated from the node design file design/MultiObjectTracker.node.yaml.

Subscribers#

Name	Type	Description	Topic (global)
`detection01/objects`	`autoware_perception_msgs/msg/DetectedObjects`	the detected objects input for channel 1
`detection02/objects`	`autoware_perception_msgs/msg/DetectedObjects`	the detected objects input for channel 2
`detection03/objects`	`autoware_perception_msgs/msg/DetectedObjects`	the detected objects input for channel 3
`detection04/objects`	`autoware_perception_msgs/msg/DetectedObjects`	the detected objects input for channel 4
`detection05/objects`	`autoware_perception_msgs/msg/DetectedObjects`	the detected objects input for channel 5
`detection06/objects`	`autoware_perception_msgs/msg/DetectedObjects`	the detected objects input for channel 6
`detection07/objects`	`autoware_perception_msgs/msg/DetectedObjects`	the detected objects input for channel 7
`detection08/objects`	`autoware_perception_msgs/msg/DetectedObjects`	the detected objects input for channel 8
`detection09/objects`	`autoware_perception_msgs/msg/DetectedObjects`	the detected objects input for channel 9
`detection10/objects`	`autoware_perception_msgs/msg/DetectedObjects`	the detected objects input for channel 10
`detection11/objects`	`autoware_perception_msgs/msg/DetectedObjects`	the detected objects input for channel 11
`detection12/objects`	`autoware_perception_msgs/msg/DetectedObjects`	the detected objects input for channel 12
`input/odometry`	`nav_msgs/msg/Odometry`	the odometry input for ego vehicle state estimation
`ros_transform`	`tf2_msgs/msg/TFMessage`	the ros transform input for ego vehicle state estimation	`/tf`

Publishers#

Name	Type	Description	QoS
`objects`	`autoware_perception_msgs/msg/TrackedObjects`	the tracked objects output from the multi-object tracker	reliability: reliable durability: transient_local
`merged_objects`	`autoware_perception_msgs/msg/DetectedObjects`	the merged detected objects output from the multi-object tracker	reliability: reliable durability: transient_local

Input channel configuration#

Multiple inputs are pre-defined in the input channel parameters (described below) and the inputs can be configured.

rule of the channel configuration

'none' or empty : Indicates that this detection input channel is not used/disabled
Any other string : Specifies a custom channel name to be used for the detection input, configured in schema/input_channels.schema.json

Up to 12 detection inputs can be configured (detection01 through detection12). Each input consists of an objects topic and its corresponding channel configuration.

Example configurations:

Single detection input:

input/detection01/objects: /perception/object_recognition/detection/objects
input/detection01/channel: detected_objects # general input channel type
input/detection02/objects: input/objects02
input/detection02/channel: none # Disabled

Multiple detection inputs:

# lidar centerpoint
input/detection01/objects: /perception/object_recognition/detection/lidar_centerpoint/objects
input/detection01/channel: lidar_centerpoint

# lidar short_range centerpoint
input/detection02/channel: /perception/object_recognition/detection/centerpoint_short_range/objects
input/detection02/objects: lidar_centerpoint_short_range

# camera lidar fusion
input/detection03/objects: /perception/object_recognition/detection/clustering/camera_lidar_fusion/objects
input/detection03/channel: camera_lidar_fusion

# camera lidar fusion based irregular object detection
input/detection04/objects: /perception/object_recognition/detection/irregular_object/objects
input/detection04/channel: camera_lidar_fusion_irregular

# detection by tracker
input/detection05/objects: /perception/object_recognition/detection/detection_by_tracker/objects
input/detection05/channel: detection_by_tracker

# radar
input/detection06/objects: /perception/object_recognition/detection/radar/objects
input/detection06/channel: radar

# disable
input/detection07/objects: input/objects07
input/detection07/channel: none # Disabled

Up to 12 detection inputs can be configured (detection01 through detection12). Each input consists of an objects topic and its corresponding channel configuration.

Parameters#

Node parameters#

The node launch parameters below are generated from the node design file design/MultiObjectTracker.node.yaml.

Name	Type	Default	Description
`input/detection01/channel`	string	`none`	the channel name for channel 1
`input/detection02/channel`	string	`none`	the channel name for channel 2
`input/detection03/channel`	string	`none`	the channel name for channel 3
`input/detection04/channel`	string	`none`	the channel name for channel 4
`input/detection05/channel`	string	`none`	the channel name for channel 5
`input/detection06/channel`	string	`none`	the channel name for channel 6
`input/detection07/channel`	string	`none`	the channel name for channel 7
`input/detection08/channel`	string	`none`	the channel name for channel 8
`input/detection09/channel`	string	`none`	the channel name for channel 9
`input/detection10/channel`	string	`none`	the channel name for channel 10
`input/detection11/channel`	string	`none`	the channel name for channel 11
`input/detection12/channel`	string	`none`	the channel name for channel 12
`publish_merged_objects`	boolean	`false`	switch to publish the merged detected objects output
`ego_source`	string	`tf`	the source of ego vehicle state estimation. "tf" or "odometry" are supported. If "tf" is selected, the ego vehicle state is estimated from the ros transform input. If "odometry" is selected, the ego vehicle state is estimated from the odometry input.

Input Channel parameters#

Name	Type	Description	Default	Range
input_channels.detected_objects.associator_type	string	Association algorithm for this channel. 'bev': bird's-eye-view area scoring with GNN assignment. 'polar': polar-coordinate (range-bearing) scoring.	bev	['bev', 'polar']
input_channels.detected_objects.flags.can_spawn_new_tracker	boolean	Indicates if the input channel can spawn new trackers.	true	N/A
input_channels.detected_objects.flags.can_trust_existence_probability	boolean	Indicates if the input channel can trust the existence probability.	true	N/A
input_channels.detected_objects.flags.can_trust_extension	boolean	Indicates if the input channel can trust the object size(extension).	true	N/A
input_channels.detected_objects.flags.can_trust_classification	boolean	Indicates if the input channel can trust the object classification.	true	N/A
input_channels.detected_objects.flags.can_trust_orientation	boolean	Indicates if the input channel can trust the object orientation.	true	N/A
input_channels.detected_objects.optional.name	string	The name of the input channel.	detected_objects	N/A
input_channels.detected_objects.optional.short_name	string	The short name of the input channel.	all	N/A
input_channels.lidar_clustering.associator_type	string	Association algorithm for this channel. 'bev': bird's-eye-view area scoring with GNN assignment. 'polar': polar-coordinate (range-bearing) scoring.	bev	['bev', 'polar']
input_channels.lidar_clustering.flags.can_spawn_new_tracker	boolean	Indicates if the input channel can spawn new trackers.	true	N/A
input_channels.lidar_clustering.flags.can_trust_existence_probability	boolean	Indicates if the input channel can trust the existence probability.	true	N/A
input_channels.lidar_clustering.flags.can_trust_extension	boolean	Indicates if the input channel can trust the object size(extension).	true	N/A
input_channels.lidar_clustering.flags.can_trust_classification	boolean	Indicates if the input channel can trust the object classification.	true	N/A
input_channels.lidar_clustering.flags.can_trust_orientation	boolean	Indicates if the input channel can trust the object orientation.	true	N/A
input_channels.lidar_clustering.optional.name	string	The name of the input channel.	detected_objects	N/A
input_channels.lidar_clustering.optional.short_name	string	The short name of the input channel.	all	N/A
input_channels.lidar_centerpoint.associator_type	string	Association algorithm for this channel. 'bev': bird's-eye-view area scoring with GNN assignment. 'polar': polar-coordinate (range-bearing) scoring.	bev	['bev', 'polar']
input_channels.lidar_centerpoint.flags.can_spawn_new_tracker	boolean	Indicates if the input channel can spawn new trackers.	true	N/A
input_channels.lidar_centerpoint.flags.can_trust_existence_probability	boolean	Indicates if the input channel can trust the existence probability.	true	N/A
input_channels.lidar_centerpoint.flags.can_trust_extension	boolean	Indicates if the input channel can trust the object size(extension).	true	N/A
input_channels.lidar_centerpoint.flags.can_trust_classification	boolean	Indicates if the input channel can trust the object classification.	true	N/A
input_channels.lidar_centerpoint.flags.can_trust_orientation	boolean	Indicates if the input channel can trust the object orientation.	true	N/A
input_channels.lidar_centerpoint.optional.name	string	The name of the input channel.	detected_objects	N/A
input_channels.lidar_centerpoint.optional.short_name	string	The short name of the input channel.	all	N/A
input_channels.lidar_centerpoint_short_range.associator_type	string	Association algorithm for this channel. 'bev': bird's-eye-view area scoring with GNN assignment. 'polar': polar-coordinate (range-bearing) scoring.	bev	['bev', 'polar']
input_channels.lidar_centerpoint_short_range.flags.can_spawn_new_tracker	boolean	Indicates if the input channel can spawn new trackers.	true	N/A
input_channels.lidar_centerpoint_short_range.flags.can_trust_existence_probability	boolean	Indicates if the input channel can trust the existence probability.	true	N/A
input_channels.lidar_centerpoint_short_range.flags.can_trust_extension	boolean	Indicates if the input channel can trust the object size(extension).	true	N/A
input_channels.lidar_centerpoint_short_range.flags.can_trust_classification	boolean	Indicates if the input channel can trust the object classification.	true	N/A
input_channels.lidar_centerpoint_short_range.flags.can_trust_orientation	boolean	Indicates if the input channel can trust the object orientation.	true	N/A
input_channels.lidar_centerpoint_short_range.optional.name	string	The name of the input channel.	detected_objects	N/A
input_channels.lidar_centerpoint_short_range.optional.short_name	string	The short name of the input channel.	all	N/A
input_channels.lidar_apollo.associator_type	string	Association algorithm for this channel. 'bev': bird's-eye-view area scoring with GNN assignment. 'polar': polar-coordinate (range-bearing) scoring.	bev	['bev', 'polar']
input_channels.lidar_apollo.flags.can_spawn_new_tracker	boolean	Indicates if the input channel can spawn new trackers.	true	N/A
input_channels.lidar_apollo.flags.can_trust_existence_probability	boolean	Indicates if the input channel can trust the existence probability.	true	N/A
input_channels.lidar_apollo.flags.can_trust_extension	boolean	Indicates if the input channel can trust the object size(extension).	true	N/A
input_channels.lidar_apollo.flags.can_trust_classification	boolean	Indicates if the input channel can trust the object classification.	true	N/A
input_channels.lidar_apollo.flags.can_trust_orientation	boolean	Indicates if the input channel can trust the object orientation.	true	N/A
input_channels.lidar_apollo.optional.name	string	The name of the input channel.	detected_objects	N/A
input_channels.lidar_apollo.optional.short_name	string	The short name of the input channel.	all	N/A
input_channels.lidar_pointpainting.associator_type	string	Association algorithm for this channel. 'bev': bird's-eye-view area scoring with GNN assignment. 'polar': polar-coordinate (range-bearing) scoring.	bev	['bev', 'polar']
input_channels.lidar_pointpainting.flags.can_spawn_new_tracker	boolean	Indicates if the input channel can spawn new trackers.	true	N/A
input_channels.lidar_pointpainting.flags.can_trust_existence_probability	boolean	Indicates if the input channel can trust the existence probability.	true	N/A
input_channels.lidar_pointpainting.flags.can_trust_extension	boolean	Indicates if the input channel can trust the object size(extension).	true	N/A
input_channels.lidar_pointpainting.flags.can_trust_classification	boolean	Indicates if the input channel can trust the object classification.	true	N/A
input_channels.lidar_pointpainting.flags.can_trust_orientation	boolean	Indicates if the input channel can trust the object orientation.	true	N/A
input_channels.lidar_pointpainting.optional.name	string	The name of the input channel.	detected_objects	N/A
input_channels.lidar_pointpainting.optional.short_name	string	The short name of the input channel.	all	N/A
input_channels.camera_streampetr.associator_type	string	Association algorithm for this channel. 'bev': bird's-eye-view area scoring with GNN assignment. 'polar': polar-coordinate (range-bearing) scoring.	bev	['bev', 'polar']
input_channels.camera_streampetr.flags.can_spawn_new_tracker	boolean	Indicates if the input channel can spawn new trackers.	true	N/A
input_channels.camera_streampetr.flags.can_trust_existence_probability	boolean	Indicates if the input channel can trust the existence probability.	true	N/A
input_channels.camera_streampetr.flags.can_trust_extension	boolean	Indicates if the input channel can trust the object size(extension).	true	N/A
input_channels.camera_streampetr.flags.can_trust_classification	boolean	Indicates if the input channel can trust the object classification.	true	N/A
input_channels.camera_streampetr.flags.can_trust_orientation	boolean	Indicates if the input channel can trust the object orientation.	true	N/A
input_channels.camera_streampetr.optional.name	string	The name of the input channel.	detected_objects	N/A
input_channels.camera_streampetr.optional.short_name	string	The short name of the input channel.	all	N/A
input_channels.camera_lidar_fusion.associator_type	string	Association algorithm for this channel. 'bev': bird's-eye-view area scoring with GNN assignment. 'polar': polar-coordinate (range-bearing) scoring.	bev	['bev', 'polar']
input_channels.camera_lidar_fusion.flags.can_spawn_new_tracker	boolean	Indicates if the input channel can spawn new trackers.	true	N/A
input_channels.camera_lidar_fusion.flags.can_trust_existence_probability	boolean	Indicates if the input channel can trust the existence probability.	true	N/A
input_channels.camera_lidar_fusion.flags.can_trust_extension	boolean	Indicates if the input channel can trust the object size(extension).	true	N/A
input_channels.camera_lidar_fusion.flags.can_trust_classification	boolean	Indicates if the input channel can trust the object classification.	true	N/A
input_channels.camera_lidar_fusion.flags.can_trust_orientation	boolean	Indicates if the input channel can trust the object orientation.	true	N/A
input_channels.camera_lidar_fusion.optional.name	string	The name of the input channel.	detected_objects	N/A
input_channels.camera_lidar_fusion.optional.short_name	string	The short name of the input channel.	all	N/A
input_channels.camera_lidar_fusion_irregular.associator_type	string	Association algorithm for this channel. 'bev': bird's-eye-view area scoring with GNN assignment. 'polar': polar-coordinate (range-bearing) scoring.	bev	['bev', 'polar']
input_channels.camera_lidar_fusion_irregular.flags.can_spawn_new_tracker	boolean	Indicates if the input channel can spawn new trackers.	true	N/A
input_channels.camera_lidar_fusion_irregular.flags.can_trust_existence_probability	boolean	Indicates if the input channel can trust the existence probability.	true	N/A
input_channels.camera_lidar_fusion_irregular.flags.can_trust_extension	boolean	Indicates if the input channel can trust the object size(extension).	true	N/A
input_channels.camera_lidar_fusion_irregular.flags.can_trust_classification	boolean	Indicates if the input channel can trust the object classification.	true	N/A
input_channels.camera_lidar_fusion_irregular.flags.can_trust_orientation	boolean	Indicates if the input channel can trust the object orientation.	true	N/A
input_channels.camera_lidar_fusion_irregular.optional.name	string	The name of the input channel.	detected_objects	N/A
input_channels.camera_lidar_fusion_irregular.optional.short_name	string	The short name of the input channel.	all	N/A
input_channels.detection_by_tracker.associator_type	string	Association algorithm for this channel. 'bev': bird's-eye-view area scoring with GNN assignment. 'polar': polar-coordinate (range-bearing) scoring.	bev	['bev', 'polar']
input_channels.detection_by_tracker.flags.can_spawn_new_tracker	boolean	Indicates if the input channel can spawn new trackers.	true	N/A
input_channels.detection_by_tracker.flags.can_trust_existence_probability	boolean	Indicates if the input channel can trust the existence probability.	true	N/A
input_channels.detection_by_tracker.flags.can_trust_extension	boolean	Indicates if the input channel can trust the object size(extension).	true	N/A
input_channels.detection_by_tracker.flags.can_trust_classification	boolean	Indicates if the input channel can trust the object classification.	true	N/A
input_channels.detection_by_tracker.flags.can_trust_orientation	boolean	Indicates if the input channel can trust the object orientation.	true	N/A
input_channels.detection_by_tracker.optional.name	string	The name of the input channel.	detected_objects	N/A
input_channels.detection_by_tracker.optional.short_name	string	The short name of the input channel.	all	N/A
input_channels.radar.associator_type	string	Association algorithm for this channel. 'bev': bird's-eye-view area scoring with GNN assignment. 'polar': polar-coordinate (range-bearing) scoring.	bev	['bev', 'polar']
input_channels.radar.flags.can_spawn_new_tracker	boolean	Indicates if the input channel can spawn new trackers.	true	N/A
input_channels.radar.flags.can_trust_existence_probability	boolean	Indicates if the input channel can trust the existence probability.	true	N/A
input_channels.radar.flags.can_trust_extension	boolean	Indicates if the input channel can trust the object size(extension).	true	N/A
input_channels.radar.flags.can_trust_classification	boolean	Indicates if the input channel can trust the object classification.	true	N/A
input_channels.radar.flags.can_trust_orientation	boolean	Indicates if the input channel can trust the object orientation.	true	N/A
input_channels.radar.optional.name	string	The name of the input channel.	detected_objects	N/A
input_channels.radar.optional.short_name	string	The short name of the input channel.	all	N/A
input_channels.camera_vru.associator_type	string	Association algorithm for this channel. 'bev': bird's-eye-view area scoring with GNN assignment. 'polar': polar-coordinate (range-bearing) scoring.	bev	['bev', 'polar']
input_channels.camera_vru.flags.can_spawn_new_tracker	boolean	Indicates if the input channel can spawn new trackers.	true	N/A
input_channels.camera_vru.flags.can_trust_existence_probability	boolean	Indicates if the input channel can trust the existence probability.	true	N/A
input_channels.camera_vru.flags.can_trust_extension	boolean	Indicates if the input channel can trust the object size(extension).	true	N/A
input_channels.camera_vru.flags.can_trust_classification	boolean	Indicates if the input channel can trust the object classification.	true	N/A
input_channels.camera_vru.flags.can_trust_orientation	boolean	Indicates if the input channel can trust the object orientation.	true	N/A
input_channels.camera_vru.optional.name	string	The name of the input channel.	detected_objects	N/A
input_channels.camera_vru.optional.short_name	string	The short name of the input channel.	all	N/A

Core Parameters#

Node

Name	Type	Description	Default	Range
publish_rate	float	Timer frequency to output with delay compensation.	10.0	N/A
world_frame_id	string	Object kinematics definition frame.	map	N/A
ego_frame_id	string	Vehicle's ego frame.	base_link	N/A
consider_odometry_uncertainty	boolean	If True, consider odometry uncertainty in tracking.	false	N/A
ego_source	string	Backend used to obtain the time-varying ego pose: 'tf' looks it up from the TF tree (legacy), 'odometry' interpolates it from the subscribed ~/input/odometry buffer (TF fallback on miss).	tf	['tf', 'odometry']
publish_on_timer	boolean	Publish trigger, independent of delay_compensation. false publishes tracks on measurement; true publishes them from the periodic timer at publish_rate.	false	N/A
delay_compensation	string	Object-export delay compensation applied to the published tracks (independent of publish_on_timer), ordered from least to most. 'none': no compensation, use the latest detection/update stamp; 'publish_delay': advance the detection stamp by the update->publish delay; 'odometry': compensate up to the newest buffered odometry stamp; 'full': full compensation to the current time.	none	['none', 'publish_delay', 'odometry', 'full']
tracker_configs.polygon_tracker.enable_velocity_estimation	boolean	Estimate velocity for polygon tracks.	true	N/A
tracker_configs.polygon_tracker.enable_motion_output.unknown	boolean			N/A
tracker_configs.polygon_tracker.enable_motion_output.car	boolean			N/A
tracker_configs.polygon_tracker.enable_motion_output.truck	boolean			N/A
tracker_configs.polygon_tracker.enable_motion_output.bus	boolean			N/A
tracker_configs.polygon_tracker.enable_motion_output.trailer	boolean			N/A
tracker_configs.polygon_tracker.enable_motion_output.motorcycle	boolean			N/A
tracker_configs.polygon_tracker.enable_motion_output.bicycle	boolean			N/A
tracker_configs.polygon_tracker.enable_motion_output.pedestrian	boolean			N/A
tracker_configs.polygon_tracker.enable_motion_output.animal	boolean			N/A
tracker_configs.polygon_tracker.enable_motion_output.hazard	boolean			N/A
tracker_configs.static_tracker.convert_polygon_to_bbox	boolean	Convert POLYGON shape to BOUNDING_BOX at publish time.	false	N/A
pruning_pedestrian_pair_min_iou	float	Minimum 1D IoU between two pedestrian trackers to mark the pair redundant.	0.1	≥0.0 ≤1.0
pruning_known_pair_min_iou	float	Minimum 2D IoU between two known-class trackers to mark the pair redundant; a small fragment contained in a larger object merges by containment below this.	0.1	≥0.0 ≤1.0
pruning_unknown_pair_min_giou	float	Minimum generalized IoU between two unknown trackers to mark the pair redundant. Negative values admit disjoint pairs, bounded by pruning_unknown_pair_max_gap.	0.1	≥-1.0 ≤1.0
pruning_unknown_pair_max_gap	float	Maximum boundary gap [m] for a disjoint unknown-unknown pair to stay merge-eligible.	1.0	≥0.0
publish_processing_time	boolean	Enable to publish debug message of process time information.	false	N/A
publish_processing_time_detail	boolean	Enable to publish debug message of detailed process time information.	false	N/A
publish_tentative_objects	boolean	Enable to publish tentative tracked objects, which have lower confidence.	false	N/A
publish_debug_markers	boolean	Enable to publish debug markers, which indicates association of multi-inputs, existence probability of each detection.	false	N/A
diagnostics_warn_delay	float	Delay threshold for warning diagnostics in seconds.	0.5	N/A
diagnostics_error_delay	float	Delay threshold for error diagnostics in seconds.	1.0	N/A
diagnostics_warn_extrapolation	float	Extrapolation duration threshold for warning diagnostics in seconds.	0.5	N/A
diagnostics_error_extrapolation	float	Extrapolation duration threshold for error diagnostics in seconds.	1.0	N/A

Association

Name	Type	Description	Default	Range
tracker_assignment.bounding_box.unknown.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.bounding_box.unknown.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.bounding_box.car.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.bounding_box.car.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.bounding_box.truck.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.bounding_box.truck.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.bounding_box.bus.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.bounding_box.bus.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.bounding_box.trailer.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.bounding_box.trailer.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.bounding_box.motorcycle.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.bounding_box.motorcycle.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.bounding_box.bicycle.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.bounding_box.bicycle.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.bounding_box.pedestrian.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.bounding_box.pedestrian.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.bounding_box.animal.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.bounding_box.animal.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.bounding_box.hazard.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.bounding_box.hazard.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.polygon.unknown.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.polygon.unknown.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.polygon.car.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.polygon.car.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.polygon.truck.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.polygon.truck.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.polygon.bus.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.polygon.bus.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.polygon.trailer.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.polygon.trailer.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.polygon.motorcycle.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.polygon.motorcycle.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.polygon.bicycle.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.polygon.bicycle.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.polygon.pedestrian.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.polygon.pedestrian.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.polygon.animal.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.polygon.animal.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.polygon.hazard.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.polygon.hazard.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.cylinder.unknown.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.cylinder.unknown.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.cylinder.car.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.cylinder.car.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.cylinder.truck.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.cylinder.truck.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.cylinder.bus.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.cylinder.bus.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.cylinder.trailer.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.cylinder.trailer.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.cylinder.motorcycle.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.cylinder.motorcycle.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.cylinder.bicycle.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.cylinder.bicycle.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.cylinder.pedestrian.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.cylinder.pedestrian.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.cylinder.animal.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.cylinder.animal.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_assignment.cylinder.hazard.create	N/A	Tracker type to spawn for new detections of this class, or "null" to explicitly reject (no tracker created, no association attempted).		N/A
tracker_assignment.cylinder.hazard.match	array	Ordered list of tracker types eligible for association. Omitted when create is "null".		N/A
tracker_profiles	object	Per-tracker-type association thresholds keyed by shape then label. Keys are tracker type names (e.g. general_vehicle_tracker). Every (tracker, shape, label) triple listed in tracker_assignment.match must have an explicit entry here.		N/A
unknown_association_giou_threshold	float	GIoU threshold for unknown-unknown association.	-0.8	N/A

Assumptions / Known limits#

See the model explanations.

Performance Benchmark & Unit Testing#

Overview#

Unit tests and benchmarks are included to evaluate tracker performance under varying detection loads and object types.

How to Run Locally#

1. Build with Tests#

colcon build --packages-select autoware_multi_object_tracker \
  --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo

2. Run the Performance Benchmark#

source install/setup.bash
./build/autoware_multi_object_tracker/test_multi_object_tracker

This runs the default test (SimulatedDataPerformanceTest and RealDataRosbagPerformanceTest) and outputs timing data.

3. Run Optional Profiling Modes/Tests#

PerformanceVsCarCount()
PerformanceVsPedestrianCount()
PerformanceVsUnknownObjectCount()
AssociationTest()

These optional profiling tests are compiled as disabled and can be run directly using GoogleTest options:

To run a specific option profiling test:

./build/autoware_multi_object_tracker/test_multi_object_tracker \
  --gtest_also_run_disabled_tests \
  --gtest_filter="*.*AssociationTest"

To run multiple profiling tests together (separate with :):

./build/autoware_multi_object_tracker/test_multi_object_tracker \
  --gtest_also_run_disabled_tests \
  --gtest_filter="*.*AssociationTest:*.*PerformanceVsPedestrianCount"

This allows you to evaluate scalability with object count and other scenarios without modifying the source code.

Rosbag Replay & Visualization#

Simulated Rosbag Output#

To record benchmark results for visualization in RViz:

Enable write_bag = true in runIterations()
Run the test; the output .db3 path is printed
Play rosbag and visualize using rviz(suggested to use a product one):

ros2 bag play <output_file>.db3
rviz2 -d <your_rviz_config>.rviz

Real Rosbag Input#

Set the path in runPerformanceTestWithRosbag() to a real .db3 file
Run the test
Visualize the tracking result in RViz

(Optional) Error detection and handling#

(Optional) Performance characterization#

Evaluation of muSSP#

According to our evaluation, muSSP is faster than normal SSP when the matrix size is more than 100.

Execution time for varying matrix size at 95% sparsity. In real data, the sparsity was often around 95%. mussp_evaluation1

Execution time for varying the sparsity with matrix size 100. mussp_evaluation2

(Optional) References/External links#

This package makes use of external code.

Name	License	Original Repository
muSSP	Apache-2.0	https://github.com/yu-lab-vt/muSSP

[1] C. Wang, Y. Wang, Y. Wang, C.-t. Wu, and G. Yu, "muSSP: Efficient Min-cost Flow Algorithm for Multi-object Tracking," NeurIPS, 2019