autoware_traffic_light_pipeline#

Single-node composition of the traffic light recognition pipeline.

Node (`traffic_light_recognition`)#

Input / Output#

Direction	Topic	Type
Subscribed	`~/input/image`	`sensor_msgs/msg/Image`
Subscribed	`~/input/camera_info`	`sensor_msgs/msg/CameraInfo`
Subscribed	`~/input/vector_map`	`autoware_map_msgs/msg/LaneletMapBin`
Subscribed	`~/input/route`	`autoware_planning_msgs/msg/LaneletRoute`
Published	`~/output/traffic_signals`	`tier4_perception_msgs/msg/TrafficLightArray`
Published	`~/output/rois`	`tier4_perception_msgs/msg/TrafficLightRoiArray`
Published	`/diagnostics`	`diagnostic_msgs/msg/DiagnosticArray`

Node parameters#

Name	Type	Description	Default	Range
build_only	boolean	Build the TensorRT engines for the whole-image detector and the classifiers and exit, without waiting for the vector map.	false	N/A
ml_model_path	string	Directory holding the ML artifacts the model_path / label_path values below are relative to. Supplied by the launch file, so this package's config file names no user-specific path.		N/A
whole_image_detector.model_path	string	Path to the detector's ONNX model, relative to ml_model_path.		N/A
whole_image_detector.label_path	string	Path to the detector's class label file, relative to ml_model_path.		N/A
whole_image_detector.score_threshold	float	Detection score threshold for the whole-image yolox detector.	0.35	N/A
whole_image_detector.nms_threshold	float	NMS threshold for the whole-image yolox detector.	0.7	N/A
whole_image_detector.precision	string	TensorRT engine precision. int8 is calibrated with the Entropy algorithm and a fixed clip value, so it needs no calibration image list.	fp16	['fp32', 'fp16', 'int8']
map_based_detector.min_timestamp_offset	float	Minimum timestamp offset when searching for the corresponding map->camera tf sample.	-0.04	N/A
map_based_detector.max_timestamp_offset	float	Maximum timestamp offset when searching for the corresponding map->camera tf sample.	0.0	N/A
car_classifier.model_path	string	Path to the classifier's ONNX model, relative to ml_model_path.		N/A
car_classifier.label_path	string	Path to the classifier's lamp label file, relative to ml_model_path.		N/A
car_classifier.precision	string	TensorRT engine precision for this classifier.	fp16	['fp32', 'fp16', 'int8']
car_classifier.mean	array	Per-channel mean subtracted from the input image. Must match the normalization the model at model_path was trained with, so change it together with model_path.	[123.675, 116.28, 103.53]	N/A
car_classifier.std	array	Per-channel standard deviation the input image is divided by. Must match the normalization the model at model_path was trained with, so change it together with model_path.	[58.395, 57.12, 57.375]	N/A
pedestrian_classifier.model_path	string	Path to the classifier's ONNX model, relative to ml_model_path.		N/A
pedestrian_classifier.label_path	string	Path to the classifier's lamp label file, relative to ml_model_path.		N/A
pedestrian_classifier.precision	string	TensorRT engine precision for this classifier.	fp16	['fp32', 'fp16', 'int8']
pedestrian_classifier.mean	array	Per-channel mean subtracted from the input image. Must match the normalization the model at model_path was trained with, so change it together with model_path.	[123.675, 116.28, 103.53]	N/A
pedestrian_classifier.std	array	Per-channel standard deviation the input image is divided by. Must match the normalization the model at model_path was trained with, so change it together with model_path.	[58.395, 57.12, 57.375]	N/A
classifier.over_exposure_threshold	float	ROI mean-intensity threshold above which the ROI is treated as over-exposed and classified as UNKNOWN. Shared by car_classifier and pedestrian_classifier: both classify ROIs cropped from the same camera image, so this is a property of the camera, not of either classifier individually.	0.85	N/A
classifier.under_exposure_threshold	float	ROI mean-intensity threshold below which the ROI is treated as under-exposed and classified as UNKNOWN. Shared by car_classifier and pedestrian_classifier for the same reason as over_exposure_threshold.	-0.83	N/A

Prerequisites#

The ML models used by this pipeline (traffic light detector / classifier) must be downloaded in advance to ~/autoware_data. See Manual downloading of artifacts for how to download them.

The default data_path launch argument (see below) points to ~/autoware_data, so no additional configuration is needed once the models are placed there.

How to launch#

ros2 launch autoware_traffic_light_pipeline traffic_light_recognition.launch.xml

Useful launch arguments:

Argument	Default	Description
`data_path`	`$(env HOME)/autoware_data`	Directory containing the downloaded ML artifacts
`camera_name`	`camera6`	Which camera namespace this instance subscribes to / publishes for
`build_only`	`false`	Exit after the TensorRT engine is built

Example, running against a second camera:

ros2 launch autoware_traffic_light_pipeline traffic_light_recognition.launch.xml camera_name:=camera7

How to test#

PACKAGE_NAME=autoware_traffic_light_pipeline
colcon build --packages-select $PACKAGE_NAME
colcon test --packages-select $PACKAGE_NAME --event-handlers console_cohesion+

The unit/integration tests also require the ML models under ~/autoware_data, since test_traffic_light_recognition_node brings up the node with the same ml_model_path default as the launch file above.